[CHORE]: Achieve 60% doctest coverage and add Makefile and CI/CD targets for doctest and coverage

[crivetimihai]

🧭 Chore Summary

Implement comprehensive doctest coverage across the entire codebase: make doctest and drive coverage to 60% while keeping all tests green and ensuring every public function/method/class has executable documentation examples.

Ensure all docstring / doctest follows: https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html

And doctest examples start with Examples:

🧱 Areas Affected

• Pre-commit hooks / linters
• Build system / Make targets (make doctest, make doctest-verbose, make doctest-coverage, make pre-commit)
• GitHub Actions / CI pipeline
• Runtime codebase (docstrings, examples, edge cases, error handling)
• Documentation generation (Sphinx integration)

---

⚙️ Context / Rationale

Doctest ensures that documentation stays synchronized with code. Every example in docstrings becomes an executable test, catching API changes, parameter mismatches, and stale documentation before they confuse users. This creates living documentation that's always accurate and serves as both usage examples and regression tests.

What is Doctest?
Doctest searches for text that looks like interactive Python sessions in docstrings, then executes those sessions to verify that they work exactly as shown.

Simple Example:

def add_numbers(a, b):
    """Add two numbers together.
    
    >>> add_numbers(2, 3)
    5
    >>> add_numbers(-1, 1)
    0
    >>> add_numbers(0, 0)
    0
    """
    return a + b

More Complex Example with Error Handling:

def divide_safely(numerator, denominator):
    """Divide two numbers with proper error handling.
    
    Basic division:
    >>> divide_safely(10, 2)
    5.0
    >>> divide_safely(7, 3)
    2.3333333333333335
    
    Edge cases:
    >>> divide_safely(0, 5)
    0.0
    >>> divide_safely(5, 0)
    Traceback (most recent call last):
        ...
    ValueError: Cannot divide by zero
    
    Type validation:
    >>> divide_safely("10", 2)
    Traceback (most recent call last):
        ...
    TypeError: Numerator must be a number
    """
    if not isinstance(numerator, (int, float)):
        raise TypeError("Numerator must be a number")
    if not isinstance(denominator, (int, float)):
        raise TypeError("Denominator must be a number")
    if denominator == 0:
        raise ValueError("Cannot divide by zero")
    return float(numerator) / float(denominator)

Advanced Example with State and Objects:

class BankAccount:
    """A simple bank account with deposits and withdrawals.
    
    Create account:
    >>> account = BankAccount("Alice", 100)
    >>> account.owner
    'Alice'
    >>> account.balance
    100
    
    Deposit money:
    >>> account.deposit(50)
    >>> account.balance
    150
    
    Withdraw money:
    >>> account.withdraw(30)
    >>> account.balance
    120
    
    Overdraft protection:
    >>> account.withdraw(200)
    Traceback (most recent call last):
        ...
    ValueError: Insufficient funds
    
    Transaction history:
    >>> len(account.transactions)
    3
    >>> account.transactions[0]
    'Deposit: +50'
    """
    def __init__(self, owner, initial_balance=0):
        self.owner = owner
        self.balance = initial_balance
        self.transactions = []
    
    def deposit(self, amount):
        self.balance += amount
        self.transactions.append(f"Deposit: +{amount}")
    
    def withdraw(self, amount):
        if amount > self.balance:
            raise ValueError("Insufficient funds")
        self.balance -= amount
        self.transactions.append(f"Withdrawal: -{amount}")

MCP Gateway Specific Examples:

class MCPConnection:
    """Manages connection state to an MCP server.
    
    Initialize connection:
    >>> conn = MCPConnection("ws://localhost:8080", timeout=30)
    >>> conn.url
    'ws://localhost:8080'
    >>> conn.timeout
    30
    >>> conn.is_connected
    False
    
    Connection lifecycle:
    >>> conn.connect()  # doctest: +SKIP
    >>> conn.is_connected  # doctest: +SKIP
    True
    >>> conn.ping()  # doctest: +SKIP
    'pong'
    
    Error handling:
    >>> conn = MCPConnection("invalid-url")
    >>> conn.validate_url()
    Traceback (most recent call last):
        ...
    ValueError: Invalid WebSocket URL format
    
    Connection metrics:
    >>> conn.get_stats()
    {'messages_sent': 0, 'messages_received': 0, 'uptime': 0}
    """
    def __init__(self, url, timeout=10):
        self.url = url
        self.timeout = timeout
        self.is_connected = False
        self._stats = {'messages_sent': 0, 'messages_received': 0, 'uptime': 0}
    
    def validate_url(self):
        if not self.url.startswith(('ws://', 'wss://')):
            raise ValueError("Invalid WebSocket URL format")
    
    def get_stats(self):
        return self._stats.copy()

class RequestRouter:
    """Routes requests to appropriate MCP servers based on patterns.
    
    Setup router:
    >>> router = RequestRouter()
    >>> router.add_route("/api/v1/*", "server1")
    >>> router.add_route("/tools/*", "server2") 
    >>> len(router.routes)
    2
    
    Route matching:
    >>> router.match_route("/api/v1/users")
    'server1'
    >>> router.match_route("/tools/calculator")
    'server2'
    >>> router.match_route("/unknown/path")
    
    Route conflicts:
    >>> router.add_route("/api/v1/*", "server3")
    Traceback (most recent call last):
        ...
    ValueError: Route pattern '/api/v1/*' already exists
    
    Pattern validation:
    >>> router.add_route("invalid-pattern", "server4")
    Traceback (most recent call last):
        ...
    ValueError: Route pattern must start with '/'
    """
    def __init__(self):
        self.routes = {}
    
    def add_route(self, pattern, server_id):
        if not pattern.startswith('/'):
            raise ValueError("Route pattern must start with '/'")
        if pattern in self.routes:
            raise ValueError(f"Route pattern '{pattern}' already exists")
        self.routes[pattern] = server_id
    
    def match_route(self, path):
        for pattern, server_id in self.routes.items():
            if self._matches_pattern(path, pattern):
                return server_id
        return None
    
    def _matches_pattern(self, path, pattern):
        if pattern.endswith('*'):
            return path.startswith(pattern[:-1])
        return path == pattern

---

📦 Related Make Targets

Target	Purpose
make doctest	Run doctest on all modules with summary report
make doctest-verbose	Run doctest with detailed output (-v flag)
make doctest-coverage	Generate coverage report for doctest examples
make doctest-check	Check doctest coverage percentage (fail if < 60%)
make pre-commit	Execute all hooks locally (includes doctest validation)
make docs-build	Build documentation with doctest examples
make docs-test	Test all documentation examples
make lint	Meta-target (includes doctest + type checks + style)
make test	Unit / integration tests
make smoketest	Minimal E2E sanity check

Bold targets are mandatory; CI must fail if doctest coverage is below 60% or any doctest fails.

---

📋 Acceptance Criteria

• make doctest exits 0 with 0 failures across all modules.
• make doctest-coverage reports 60% coverage for all public functions/methods/classes.
• make doctest-check passes coverage threshold validation.
• make pre-commit includes doctest validation without modifying working tree.
• make test and make smoketest pass with doctest integration.
• GitHub Actions enforces doctest requirements in CI pipeline.
• All public APIs have meaningful, executable examples in their docstrings.
• Error cases and edge conditions are documented with doctest examples.
• Changelog entry under "Documentation" or "Maintenance".

---

🛠️ Task List (suggested flow)

1. Baseline assessment

   python -m doctest mcpgateway/**/*.py > /tmp/doctest-report.txt
   find mcpgateway -name "*.py" -exec python -c "import doctest, sys; doctest.testmod(__import__(sys.argv[1].replace('/', '.').replace('.py', '')))" {} \;

   Identify modules without doctest coverage and catalog missing examples.

2. Makefile integration

   .PHONY: doctest doctest-verbose doctest-coverage doctest-check
   
   doctest:
   	python -m pytest --doctest-modules mcpgateway/
   
   doctest-verbose:
   	python -m pytest --doctest-modules mcpgateway/ -v
   
   doctest-coverage:
   	python -m doctest_coverage mcpgateway/ --threshold=100
   
   doctest-check:
   	python -c "import doctest_coverage; doctest_coverage.check_coverage('mcpgateway/', 100)"

3. Start with simple functions

   • Add basic input/output examples to utility functions
   • Cover happy path scenarios first
   • Run make doctest frequently to catch syntax errors

4. Expand to complex scenarios

   • Document error cases with Traceback examples
   • Show state changes in classes and methods
   • Include edge cases and boundary conditions

5. Error handling patterns

   def validate_email(email):
       """Validate email address format.
       
       Valid emails:
       >>> validate_email("user@example.com")
       True
       >>> validate_email("test.email+tag@domain.co.uk")
       True
       
       Invalid emails:
       >>> validate_email("invalid-email")
       False
       >>> validate_email("@missing-local.com")
       False
       >>> validate_email("missing-at-sign.com")
       False
       
       Type errors:
       >>> validate_email(None)
       Traceback (most recent call last):
           ...
       TypeError: Email must be a string
       """
   
   def validate_mcp_message(message):
       """Validate MCP protocol message structure.
       
       Valid message:
       >>> msg = {"jsonrpc": "2.0", "method": "ping", "id": 1}
       >>> validate_mcp_message(msg)
       True
       
       Missing required fields:
       >>> validate_mcp_message({"method": "ping"})
       Traceback (most recent call last):
           ...
       ValueError: Missing required field: jsonrpc
       
       Invalid JSON-RPC version:
       >>> validate_mcp_message({"jsonrpc": "1.0", "method": "ping", "id": 1})
       Traceback (most recent call last):
           ...
       ValueError: Invalid JSON-RPC version, must be '2.0'
       
       Invalid method name:
       >>> validate_mcp_message({"jsonrpc": "2.0", "method": "", "id": 1})
       Traceback (most recent call last):
           ...
       ValueError: Method name cannot be empty
       """

6. CI integration

   Add to the lint matrix in the existing GitHub Actions workflow:

   # Add to the existing lint matrix
   - id: doctest
     setup: pip install pytest doctest-coverage
     cmd: |
       python -m pytest --doctest-modules mcpgateway/
       python -m doctest_coverage mcpgateway/ --threshold=100
   
   - id: doctest-verbose  
     setup: pip install pytest
     cmd: python -m pytest --doctest-modules mcpgateway/ -v

   Or create a dedicated doctest job following the existing pattern:

   name: Documentation Tests
   
   on:
     push:
       branches: ["main"]
     pull_request:
       branches: ["main"]
   
   jobs:
     doctest:
       name: doctest
       runs-on: ubuntu-latest
       steps:
         - name: ⬇️  Checkout source
           uses: actions/checkout@v4
           with:
             fetch-depth: 1
   
         - name: 🐍  Set up Python
           uses: actions/setup-python@v5
           with:
             python-version: "3.12"
             cache: pip
   
         - name: 📦  Install project (editable mode)
           run: |
             python -m pip install --upgrade pip
             pip install -e .[dev]
   
         - name: 🔧  Install doctest tools
           run: pip install pytest doctest-coverage
   
         - name: 🔍  Run doctest
           run: |
             python -m pytest --doctest-modules mcpgateway/
             python -m doctest_coverage mcpgateway/ --threshold=100

7. Coverage measurement

   • Install doctest-coverage or similar tool
   • Set up automated reporting for missing doctest examples
   • Integrate with existing coverage tools

8. Advanced doctest features

   # Ellipsis for variable output
   >>> import uuid; str(uuid.uuid4())  # doctest: +ELLIPSIS
   '...-...-...-...-...'
   
   # Normalize whitespace
   >>> print("hello\n\nworld")  # doctest: +NORMALIZE_WHITESPACE
   hello
   world
   
   # Skip platform-specific tests
   >>> import sys
   >>> sys.platform  # doctest: +SKIP
   'linux'

9. Documentation integration

   • Configure MkDocs to include doctest examples in documentation
   • Set up mkdocs.yml with appropriate plugins for code execution
   • Update documentation in docs/docs/testing/ to reference doctest examples
   • Ensure examples render properly in generated MkDocs site

   Example mkdocs.yml configuration:

   plugins:
     - mkdocstrings:
         handlers:
           python:
             options:
               show_source: true
               show_docstring_examples: true

10. Final validation

    make doctest doctest-coverage doctest-check
    make test smoketest
    make docs-build docs-test

---

📖 References

• Python doctest module – Built-in testing via documentation · https://docs.python.org/3/library/doctest.html
• pytest doctest integration – Running doctests with pytest · https://docs.pytest.org/en/stable/how.html#doctest
• doctest-coverage – Measure doctest coverage · https://pypi.org/project/doctest-coverage/
• MkDocs Material – Documentation framework · https://squidfunk.github.io/mkdocs-material/
• mkdocstrings – Automatic API docs from docstrings · https://mkdocstrings.github.io/

---

🧩 Additional Notes

• Start simple: Begin with pure functions that have clear inputs/outputs before tackling stateful objects.
• Test the examples: Every doctest example should be copy-pastable into a Python REPL and work exactly as shown.
• Cover error cases: Don't just show happy path - demonstrate how functions handle invalid inputs.
• Keep examples realistic: Use domain-appropriate data rather than contrived foo/bar examples.
• Maintain consistency: Establish conventions for example formatting, variable names, and output representation.
• Performance considerations: Doctest examples run during import - keep them fast and avoid expensive operations.
• Mock external dependencies: Use doctest.testmod() options or # doctest: +SKIP for examples requiring external services.

Doctest Best Practices:

• Each public function/method/class should have at least one example
• Show typical usage patterns, not just minimal cases
• Include examples that demonstrate the function's purpose and value
• Use meaningful variable names and realistic data
• Test both success and failure scenarios where appropriate

[manavgup]

Working on infrastructure setup (Makefile, CI, tooling)