Testing Fundamentals

Master Core MCP Testing Concepts and Strategies

Learning Path Context

This page bridges the gap between the Quick Start tutorial and advanced testing approaches like How to Test MCP Servers.

Prerequisites: Complete the Quick Start guide and have a working MCP server.
What you'll learn: Core testing concepts, approach selection, and validation strategies.

Core Testing Concepts

MCP testing validates the complete lifecycle of Model Context Protocol communication. Understanding these core concepts helps you choose the right testing approach and build comprehensive test suites.

MCP Server Lifecycle

Every MCP server interaction follows this pattern:

Server Startup: Process spawned via command + arguments
Handshake: initialize → initialized sequence
Tool Operations: tools/list and tools/call methods
Graceful Shutdown: Process termination and cleanup

Validation Levels

Protocol Level

JSON-RPC 2.0 format, handshake sequence, method availability

Schema Level

Tool definitions, input schemas, response structures

Semantic Level

Business logic, data validation, error handling

YAML vs Programmatic Testing

MCP Aegis offers two complementary approaches. Choose based on your testing needs and team preferences.

YAML Approach

✅ Best For:

Declarative request/response validation
Pattern-based assertions (regex, arrays, types)
CI/CD integration and automated testing
Non-developer team members
Documentation-as-tests

yaml

description: "Tool validation example"
tests:
  - it: "should have well-documented tools"
    request:
      jsonrpc: "2.0"
      id: "validate-1"
      method: "tools/list"
      params: {}
    expect:
      response:
        result:
          tools:
            match:arrayElements:
              name: "match:type:string"
              description: "match:regex:.{20,}"
              inputSchema: "match:type:object"

Programmatic Approach

✅ Best For:

Complex conditional logic
Dynamic test generation
Performance measurements
Integration with existing test suites
Advanced error handling scenarios

javascript

import { connect } from 'mcp-aegis';

test('comprehensive tool validation', async () => {
  const client = await connect('./config.json');
  
  const tools = await client.listTools();
  
  // Dynamic validation based on tool types
  tools.forEach(tool => {
    assert.ok(tool.description.length >= 20);
    
    if (tool.name.includes('file')) {
      assert.ok(tool.inputSchema.properties.path);
    }
  });
  
  await client.disconnect();
});

Hybrid Approach Recommended

Most production projects benefit from both approaches: YAML for standard validation patterns and programmatic tests for complex scenarios. Start with YAML, add programmatic tests as complexity grows.

Common Testing Patterns

These patterns form the foundation of most MCP test suites. Master these before moving to advanced scenarios.

Essential Validations

1. Tool Discovery Validation

yaml

- it: "should discover all tools"
  request: { jsonrpc: "2.0", id: "1", method: "tools/list", params: {} }
  expect:
    response:
      result:
        tools: "match:arrayLength:3"  # Expect exactly 3 tools
    stderr: "toBeEmpty"

2. Schema Completeness

yaml

- it: "should have complete schemas"
  expect:
    response:
      result:
        tools:
          match:arrayElements:
            name: "match:type:string"
            description: "match:type:string"
            inputSchema:
              type: "object"
              properties: "match:type:object"

3. Successful Execution

yaml

- it: "should execute tool successfully"
  request:
    method: "tools/call"
    params: { name: "read_file", arguments: { path: "test.txt" } }
  expect:
    response:
      result:
        content:
          - type: "text"
            text: "match:type:string"
        isError: false

4. Error Handling

yaml

- it: "should handle invalid input"
  request:
    jsonrpc: "2.0"
    id: "error-1"
    method: "tools/call"
    params: { name: "read_file", arguments: { path: "/invalid/path" } }
  expect:
    response:
      result:
        isError: true
        content:
          - type: "text"
            text: "match:contains:not found"

Validation Strategies

Choose the right validation strategy based on your confidence level and testing goals.

🎯 Exact Matching

Use when you know the exact expected output

yaml

result:
  content:
    - type: "text"
      text: "File read successfully"
  isError: false

🔍 Pattern Matching

Use when output varies but follows patterns

yaml

result:
  content:
    - type: "text"
      text: "match:regex:File .+ read"
  isError: false

📋 Partial Validation

Validate only critical fields

yaml

result:
  match:partial:
    isError: false
    content:
      - type: "text"

🚫 Negative Testing

Ensure certain conditions don't occur

yaml

result:
  isError: "match:not:equals:true"
  content: "match:not:arrayLength:0"

Troubleshooting Basics

Common issues and debugging strategies for MCP testing.

Debugging Workflow

Use Query Command: Test tools interactively before writing tests
Enable Debug Mode: Add --debug flag to see MCP communication
Check Verbose Output: Use --verbose for detailed comparisons
Isolate Issues: Test one tool at a time
Verify Configuration: Ensure config paths and commands are correct

Common Issues & Solutions

🔴 Server Won't Start

Symptoms: Timeout errors, "server not ready"

Solutions:

Check file permissions: chmod +x server.js
Verify command path in config
Increase startupTimeout
Check readyPattern matches server output

🟡 Pattern Matching Fails

Symptoms: Expected vs actual mismatches

Solutions:

Use --verbose to see exact differences
Start with simpler patterns and build complexity
Check for extra whitespace or formatting
Use match:contains for partial matches

🟣 Tests Pass Individually but Fail Together

Symptoms: Buffer bleed, flaky test results

Solutions:

Use client.clearAllBuffers() between tests
Ensure proper test isolation
Check for state sharing between tests
Add delays if server needs time between calls

Testing Workflow

A proven workflow for building comprehensive MCP test suites.

🔄 Recommended Development Cycle

Interactive Testing: Use aegis query to explore your server
Basic YAML Tests: Start with tool discovery and simple calls
Pattern Refinement: Add validation patterns as you understand the output
Error Scenarios: Test edge cases and error conditions
Programmatic Tests: Add complex scenarios as needed
CI Integration: Automate tests in your deployment pipeline

bash

# Example development workflow

# 1. Explore your server interactively
aegis query --config config.json --method tools/list
aegis query read_file '{"path": "test.txt"}' --config config.json

# 2. Create basic YAML test
# Write basic-tools.test.mcp.yml with tool discovery

# 3. Run and refine
aegis basic-tools.test.mcp.yml --config config.json --verbose

# 4. Add error testing
# Create error-scenarios.test.mcp.yml

# 5. Integrate programmatic tests
# Create complex-scenarios.programmatic.test.js

# 6. Automate in CI
aegis "test/**/*.test.mcp.yml" --config config.json --json

Next Learning Steps

Now that you understand testing fundamentals, choose your path based on your goals:

🎯 Deep Dive into Patterns

Master advanced validation techniques

Basic Patterns Array Patterns Regex Patterns

🤖 AI Agent Testing

Enterprise-grade testing for AI agents

AI Agent Testing Guide Performance Testing Real-World Examples

You're Ready!

You now have a solid foundation in MCP testing concepts. You understand when to use YAML vs programmatic approaches, core validation patterns, and debugging strategies. Choose your next step based on your specific testing needs.