# Building an MCP-Powered QA Agent


---

Connecting AI to Playwright, APIs, test evidence, and CI/CD.

## Abstract

[AI-assisted software testing](https://geekyants.com/engineering/quality-assurance/automation-testing-services) is moving beyond code generation toward agents that can inspect applications, execute tests, analyze failures, and interact with engineering tools. Model Context Protocol (MCP) provides a standardized interface for [AI applications](https://geekyants.com/ai/ai-development-services) to discover and invoke tools exposed by external systems. The current MCP TypeScript SDK v2 exposes tools, resources, and prompts.

This article presents a code-first QA architecture using TypeScript, Playwright, Zod, structured outputs, permission boundaries, evidence collection, and human approval. The objective is controlled access to deterministic [QA](https://geekyants.com/engineering/quality-assurance) capabilities rather than unrestricted autonomy.

## Introduction

A modern QA workflow spans browser automation, [API validation](https://geekyants.com/engineering/quality-assurance/api-testing), logs, traces, test reports, test data, source control, and CI/CD. An AI model can reason across these systems, but reasoning alone does not give it safe or structured access to them.

[MCP](https://geekyants.com/blog/mcp-in-action-a-developers-take-on-smarter-service-coordination) provides a standardized protocol boundary between an AI host and external tools and data. The engineering challenge is therefore not simply connecting MCP to Playwright; it is designing a QA tool layer with explicit capabilities, validated inputs, controlled environments, observable outputs, and safe failure behavior.

This article focuses on a practical architecture: an MCP server exposes selected QA operations while [Playwright](https://geekyants.com/blog/from-manual-testing-to-ai-assisted-automation-with-playwright-agents) and API clients remain the deterministic execution layer. The AI can request an operation, but the server owns validation and authorization.

## Architecture

![The AI host connects through MCP to a QA server, branching into Playwright, API, and evidence components that feed QA results.](https://geekyants-v5-media.sgp1.cdn.digitaloceanspaces.com/media/2026/10/a2f02042-8382-41f1-bf20-e086c3385ce9.png align="center")

The MCP TypeScript SDK supports servers exposing tools, resources, and prompts. Tools are appropriate for actions and side effects; resources are appropriate for read-oriented data such as reports or configuration.

The SDK also supports stdio and Streamable HTTP transports. See the official MCP SDK documentation for the current v2 implementation.

## Project Structure

```text
qa-mcp-agent/
├── src/
│   ├── server.ts
│   ├── tools/
│   │   ├── playwright.ts
│   │   ├── api.ts
│   │   └── evidence.ts
│   ├── security/
│   │   └── permissions.ts
│   └── utils/
│       └── sanitize.ts
├── tests/
│   └── checkout.spec.ts
├── artifacts/
├── package.json
└── tsconfig.json
```

## Creating the MCP Server

For a current TypeScript implementation, use the v2 server package and Zod for input validation.

```bash
npm init -y
npm install @modelcontextprotocol/server zod @playwright/test
```

```typescript
import { McpServer } from "@modelcontextprotocol/server";

const server = new McpServer({
  name: "qa-mcp-server",
  version: "1.0.0"
});
```

The current v2 SDK documents @modelcontextprotocol/server as the server package and uses Zod schemas for typed validation.

### Expose a Controlled Playwright Tool

```typescript
import * as z from "zod/v4";

server.registerTool(
  "run_playwright_test",
  {
    title: "Run Playwright Test",
    description: "Execute an approved Playwright test file",
    inputSchema: { testFile: z.string() },
    outputSchema: {
      passed: z.boolean(),
      output: z.string()
    }
  },
  async ({ testFile }) => {
    validateTestFile(testFile);
    const result = await runPlaywrightTest(testFile);
    const output = { passed: result.passed, output: result.output };
    return {
      content: [{ type: "text", text: JSON.stringify(output) }],
      structuredContent: output
    };
  }
);
```

MCP tool schemas validate arguments before the handler executes, and structured outputs can be validated against an output schema. Clients can discover tools with listTools and invoke them with callTool.

### Keep Playwright Deterministic

```typescript
import { execFile } from "node:child_process";

function runPlaywrightTest(testFile: string) {
  return new Promise<{ passed: boolean; output: string }>((resolve) => {
    execFile(
      "npx",
      ["playwright", "test", testFile],
      { timeout: 120000 },
      (error, stdout, stderr) => {
        resolve({
          passed: !error,
          output: `${stdout}\n${stderr}`
        });
      }
    );
  });
}
```

The server should not pass arbitrary shell strings from the model into an operating-system shell. Use an allowlist and validated arguments.

### Permission Boundaries

```typescript
const ALLOWED_TESTS = new Set([
  "tests/login.spec.ts",
  "tests/checkout.spec.ts",
  "tests/cart.spec.ts"
]);

function validateTestFile(testFile: string) {
  if (!ALLOWED_TESTS.has(testFile)) {
    throw new Error(`Test file is not approved: ${testFile}`);
  }
}
```

The security boundary belongs in deterministic application code, not in the model prompt.

### API Testing as Another Tool

```typescript
server.registerTool(
  "validate_api",
  {
    title: "Validate API",
    description: "Send an HTTP request and validate the response",
    inputSchema: {
      method: z.enum(["GET", "POST", "PUT", "DELETE"]),
      url: z.string(),
      expectedStatus: z.number()
    }
  },
  async ({ method, url, expectedStatus }) => {
    const response = await fetch(url, { method });
    const output = {
      status: response.status,
      passed: response.status === expectedStatus
    };
    return {
      content: [{ type: "text", text: JSON.stringify(output) }],
      structuredContent: output
    };
  }
);
```

In production, URL allowlists, authentication handling, request-body validation, rate limits, and environment restrictions should be added before exposing mutation-capable API tools.

### Use Resources for Test Evidence

MCP resources are a better fit for read-oriented evidence than action tools. A QA server can expose resources such as test-results://latest or trace://checkout.

```typescript
server.registerResource(
  "latest-results",
  "test-results://latest",
  { title: "Latest Playwright Results", mimeType: "application/json" },
  async (uri) => {
    const results = await readLatestResults();
    return {
      contents: [{
        uri: uri.href,
        mimeType: "application/json",
        text: JSON.stringify(results)
      }]
    };
  }
);
```

### Failure Analysis

```typescript
server.registerTool(
  "analyze_test_failure",
  {
    title: "Analyze Test Failure",
    inputSchema: {
      testName: z.string(),
      error: z.string(),
      consoleErrors: z.array(z.string()),
      networkFailures: z.array(z.string())
    },
    outputSchema: {
      classification: z.enum([
        "APPLICATION", "TEST", "NETWORK",
        "DATA", "ENVIRONMENT", "UNKNOWN"
      ]),
      evidence: z.array(z.string())
    }
  },
  async (input) => {
    const result = await analyzeFailure(input);
    return {
      content: [{ type: "text", text: JSON.stringify(result) }],
      structuredContent: result
    };
  }
);
```

Separate observed evidence from AI-generated hypotheses. The AI can classify a failure and recommend an investigation path, but the underlying evidence should remain inspectable.

### The UNKNOWN State

```typescript
type Diagnosis =
  | "APPLICATION" | "TEST" | "NETWORK"
  | "DATA" | "ENVIRONMENT" | "UNKNOWN";

const analysisInstruction = `
Analyze only the supplied evidence.
Do not invent missing information.
If evidence is insufficient, return UNKNOWN.
Separate observed evidence from hypotheses.
`;
```

Allowing UNKNOWN is an important reliability control. It prevents incomplete logs from being converted into false certainty.

## A QA Workflow

![AI QA agent runs Playwright tests, reads results, validates APIs, and analyzes failures before human validation or a CI decision.](https://geekyants-v5-media.sgp1.cdn.digitaloceanspaces.com/media/2026/10/a2f02042-ce73-4405-8fc5-678c23e4805f.png align="center")

## Human Approval for High-Risk Operations

```typescript
type RiskLevel = "LOW" | "MEDIUM" | "HIGH" | "CRITICAL";

const TOOL_RISK: Record<string, RiskLevel> = {
  read_test_results: "LOW",
  run_playwright_test: "LOW",
  validate_api: "LOW",
  create_test_data: "MEDIUM",
  modify_test: "HIGH",
  delete_test_data: "CRITICAL"
};

function requiresApproval(risk: RiskLevel) {
  return risk === "HIGH" || risk === "CRITICAL";
}
```

The AI can request an action, but the policy layer determines whether it is permitted. Human approval is an explicit control point rather than a suggestion embedded in a prompt.

## Auditability

```typescript
interface AuditEvent {
  timestamp: string;
  agent: string;
  tool: string;
  arguments: unknown;
  environment: string;
  decision: "ALLOWED" | "BLOCKED" | "APPROVAL_REQUIRED";
  approvedBy?: string;
}
```

Every tool invocation should produce an auditable event so teams can determine which agent called which tool, against which environment, and with what authorization decision.

## Connecting the Client

```typescript
const { tools } = await client.listTools();

const result = await client.callTool({
  name: "run_playwright_test",
  arguments: {
    testFile: "tests/checkout.spec.ts"
  }
});
```

The MCP client API supports discovery and invocation of tools and access to resources. Playwright also provides an MCP server for browser automation, which can be used when the use case is direct browser interaction.

## Testing the MCP Server

```typescript
test("blocks unapproved test", async () => {
  await expect(
    runPlaywrightTest("tests/../../secret.spec.ts")
  ).rejects.toThrow("Test file is not approved");
});

test("requires approval for data deletion", async () => {
  const decision = authorize({
    tool: "delete_test_data",
    environment: "production"
  });
  expect(decision).toBe("BLOCKED");
});
```

The MCP server itself is production software. Its schemas, authorization rules, environment restrictions, error handling, and tool contracts should therefore be covered by automated tests.

## Evaluation: Avoid Unsupported Benchmark Claims

A strong implementation should be evaluated rather than supported by invented numbers. Compare conventional QA with MCP-assisted QA using the same scenarios and report the actual environment and measurement method.

| Metric | Measurement |
| --- | --- |
| Execution time | Repeated wall-clock runs |
| Diagnosis time | Failure to actionable hypothesis |
| Correct classification | Correct diagnoses / total failures |
| Human interventions | Manual approvals or interventions |
| Blocked unsafe actions | Policy violations prevented |
| Tool-call failures | Invalid or failed calls |
| Cost | LLM/tool/API cost per workflow |

If numerical performance results are reported, include the actual MCP/Node/Playwright versions, machine or CI environment, workload, number of runs, warm-up method, and measurement definition. Do not present illustrative figures as measured evidence.

## Production Architecture

![QA MCP server with tools, resources, prompts, and policy controls, showing separate read-only evidence and write-enabled tool paths.](https://geekyants-v5-media.sgp1.cdn.digitaloceanspaces.com/media/2026/10/a2f02043-1aca-47b4-96e6-8e22fde745ea.png align="center")

## Trade-offs

More tools increase capability but also increase the action space the model must select from. More autonomy can reduce manual effort but increases the impact of incorrect decisions. More evidence can improve diagnosis but can increase data exposure and model cost. More guardrails improve safety but can introduce friction for unusual workflows.

The engineering objective is therefore not maximum autonomy. It is useful autonomy within an explicitly defined risk boundary.

## Conclusion

MCP gives [QA engineers](https://geekyants.com/service/hire-quality-assurance-developers) a standardized way to expose testing capabilities to [AI systems](https://geekyants.com/engineering/ai-engineering), but the protocol itself does not define a safe QA architecture. That responsibility belongs to the implementation.

![AI planning flows through MCP and authorization controls to QA tool execution, evidence collection, and human-reviewed analysis.](https://geekyants-v5-media.sgp1.cdn.digitaloceanspaces.com/media/2026/10/a2f02043-65d9-42e6-8906-ddd18149ed75.png align="center")

A production-oriented QA MCP server should expose narrow, validated capabilities; keep browser and API execution deterministic; separate tools from read-only evidence; enforce environment and permission boundaries outside the model; require human approval for high-impact actions; and record enough evidence to make agent behavior auditable.

The important shift is from “AI controls the test environment” to “AI operates through a controlled QA interface.” That distinction provides the foundation for building AI-assisted testing systems that remain useful, observable, and governable as their autonomy increases.

[![](https://cdn.hashnode.com/uploads/covers/5fae473f8f0a02655347989e/7c6d14f3-ba81-4d75-bca9-3cad73adeb24.png align="center")](https://geekyants.com/hire)

## References

MCP TypeScript SDK v2 documentation; MCP client tool/resource APIs; MCP resources and tools documentation; Playwright MCP documentation. These are the current primary technical references used for the implementation.