What Is QAmcp? The Beginner's Guide to Testing MCP Servers Like an AI App Would
CodexFuse Team · 2026-11-05T13:10:36.050Z · 17 min read min read
What Is QAmcp? The Beginner's Guide to Testing MCP Servers Like an AI App Would
What Is QAmcp?
If you are asking what is qamcp, the short answer is that it is a free, open-source tester that checks an MCP server the way an AI app would use it. It connects to your server like a client, reads every tool, safely runs the read-only ones, and reports pass or fail with evidence. This guide is qamcp explained in plain English: what it checks, how its 0–100 score works, and how to run your first test.
Every result is checked by code, not by an AI model. A result shows pass or fail, the time in milliseconds, the exact match, or the word that set it off. If a report flags a possible leaked secret, you can hash the suspected value and compare it against the hash in the error text without pasting the secret itself — a free SHA-256 hash generator does that comparison in seconds.
QAmcp runs from your own machine, so there is nothing to install on a server to test one. Many MCP servers do run on rented machines, and that setup is usually VPS hosting: you control the machine, and QAmcp connects to it over the network like any AI app would. The rest of this guide walks through the checks, the score, and the GitHub Action gate.
| Fact | Detail |
|---|---|
| What it is | A qamcp mcp server tester that connects like an AI app |
| Command | npx qamcp-cli <url> (Node 20 or newer) |
| Price | CLI, browser test, GitHub Action, and directory free; MIT license |
| Score | 0–100, plain arithmetic; 90+ solid, 70–89 needs fixes, below 70 likely struggles |
| Maker | Abhiram Reddy, solo founder |
| Dates | Repo created Oct 13, 2026; Visalytica launch listing Oct 29, 2026; second Show HN Nov 4, 2026 |
QAmcp is only 23 days old as a public repo — created October 13, 2026 — and the October 29 date is the Visalytica launch listing, not the day the code first existed. It appeared on Show HN first on October 21, then again on November 4, one day before this article. All facts here are as of November 5, 2026.
Why Do MCP Servers Need Testing?
MCP is an open-source standard for connecting AI applications to external systems. The project describes it as a USB-C port for AI applications: one standard plug that lets an app reach data sources, tools, and workflows.
An MCP server exposes tools, resources, and prompts that an AI app calls on a user's behalf. A vague tool description, a missing safety label, or a hidden instruction in a description changes what the AI does — and the user never sees the description that steered it.
Testing the server before your users meet it is the same habit that quality teams apply everywhere else. TestGrid ONE takes the same idea — let software check software — and applies it to whole agent-built applications, while QAmcp focuses on the MCP server itself.
The failure modes are concrete, not theoretical. A tool that claims success but returns nothing wastes an agent's turn.
A leaked secret in an error message hands a key to anyone who reads the log. A description that says "don't tell the user" is an instruction smuggled into a tool the AI trusts.
What Does QAmcp Check?
QAmcp runs a fixed set of 15 checks on every server it tests. The table below lists them in the README's own words, shortened where the original runs long.
| Check | What it looks at |
|---|---|
| Connects | Current and older MCP connection types, or a local program (stdio) |
| Newest MCP version | On the July 2026 version, or still on old SSE |
| Lists its tools | Duplicate or invalid names; a broken tool that stops AI apps loading the server |
| Every tool is described | Missing or vague descriptions, undescribed inputs, look-alike tools |
| Safety labels | Labels, and labels the tool itself contradicts |
| No hidden instructions | Known tricks: <IMPORTANT> blocks, "don't tell the user", invisible characters |
| Good inputs | Runs safe tools; catches tools that hang or claim success but return nothing |
| Bad inputs | Missing and wrong-type inputs |
| Fast enough | Connect, list, and tool times |
| No secrets leaked | Keys, passwords, server folders, stack traces in error messages |
| Login setup | Only when a server needs a login: the OAuth details AI apps need |
| Resources and prompts | Only when a server has them: each opens, content matches its type |
| Tools work together | Only when tools chain: one tool's answer passed to the next |
| Best practices (tips, not scored yet) | Common tool names that clash, mixed naming styles, idempotentHint and openWorldHint, oversized tool lists, undeclared outputSchema |
| Origin header (not scored yet) | Only for local servers: checks the Origin header as the spec requires |
Two of the 15 checks are tips, not scored yet: best practices and the Origin-header check. They still appear in the report, so you can fix them before they become scored checks. The other 13 decide the score.
The checks that apply depend on the server. A server with no login skips the login check, and a server with no resources skips that check too. Checks that could not run do not count either way, so the score is points earned out of the points actually tested.
How the 0–100 Score Works
The score is plain arithmetic, so you can check it yourself. A pass earns all points for a check, a warning earns half, and a failure earns none.
| Check group | Points each |
|---|---|
| Connects; Every tool is described | 15 |
| Newest MCP version; Lists its tools; Safety labels; No hidden instructions; Good inputs; Bad inputs | 10 |
| Login setup; Resources and prompts; Tools work together (only when they apply) | 10 |
| Fast enough; No secrets leaked | 5 |
When every check applies, the denominator is 130 points: 30 from the two 15-point checks, 60 from the six 10-point checks, 30 from the three conditional 10-point checks, and 10 from the two 5-point checks. Most servers skip at least one conditional check, so the denominator is usually smaller.
| Finding | Maximum score |
|---|---|
| Hidden instructions in a tool, resource, or prompt description | 40 |
| A secret leaked in an error message | 50 |
| A broken tool definition that can stop AI apps loading the server | 50 |
| A tool that crashes, hangs, or claims success but returns nothing | 70 |
The lowest ceiling wins. When a ceiling applies, the report says so in plain words, for example: "Would be 85, but capped at 50 because…" A server can pass most checks and still score 40 if one description hides an instruction.
The rough guide in the README is simple: 90 or above is solid, 70 to 89 works but has things to fix, and below 70 means AI apps will likely struggle or there is a safety problem. Two sample reports show the range: Context7's public server scored 100 out of 100, while the deliberately broken demo server scored 38 — a 62-point gap between a clean server and a rigged one.
How to Test MCP Server Quality With QAmcp in 6 Steps
You need Node 20 or newer and the server's URL. The whole first test takes about a minute once Node is installed.
- Install Node 20 or newer if it is not already on your machine.
- Run npx qamcp-cli https://your-server.com/mcp — replace the URL with your server's address.
- Read the score and the coverage line, for example "Tested 10 of 10 checks · ran 5 of 9 tools".
- Open the flagged findings first — hidden instructions and leaked secrets cap the score, so they matter most.
- Try the demo if you have no server yet: npx qamcp-cli --demo starts a deliberately broken server on your computer, tests it, and writes qamcp-demo-report.html in about 30 seconds.
- Save a shareable report with --json report.json or --html report.html, and re-run after every fix to watch the score move.
The package and command are qamcp-cli, not qamcp — npx qamcp-cli is the command to type. If you install it once with npm install -g qamcp-cli, the command shortens to qamcp. A local server runs with --stdio instead of a URL, for example npx qamcp-cli --stdio "npx -y @your-org/your-server".
Before you test a server you do not trust, add --look-only to call no tools at all, or --before-install to read only what the server says about itself. Both give you a safe first look without running anything the server offers.
What QAmcp Never Runs
QAmcp's safety rule is the reason the tool is safe to point at a stranger's server. By default it runs only tools the server labels read-only, and it never runs a tool that sounds destructive, like delete_account, in any mode. This is mcp server safety testing with a hard boundary: the tester never becomes the thing that changes your data.
If you own the server, --owner also runs unlabeled tools whose names clearly only read, such as get_, list_, and search_. Tools whose name or description says they write, send, or create are still skipped. For a program on your computer, QAmcp starts it a second time for a moment to check that it prints only MCP messages, and says so in the report.
Nothing leaves your machine except the requests to the server under test. There is no AI judging, no tracking, and no telemetry. The browser test on the QAmcp website runs on the project's test service, and its report is kept so you can share the link.
The QAmcp GitHub Action: Failing the Build When the Score Drops
The qamcp github action turns a one-off test into a gate. The workflow uses QAmcp/qamcp-cli@v0 with your server URL and a fail-under threshold of 80. Every pull request gets a comment with the score, and the build fails if the server got worse.
A pre-commit hook does the same check earlier: npx qamcp-cli --install-hook --stdio "npm start" makes every git commit test your server first. If it scores below 80 — or cannot be tested at all — the commit stops and tells you why. Skip it once with git commit --no-verify.
QAmcp also remembers each server's tools on your computer in a .qamcp folder, which git ignores. The next test shows every change since last time, with behavior-changing edits first — a renamed input, a newly required field, a changed safety label — and rewording last. In CI, --fail-on-tool-change stops the build on a behavior change compared with your --baseline.
The exit codes are part of the contract: 0 means fine, 1 means a required check failed, 2 means wrong options, and 3 means the server could not be tested — it is down, or it needs a login. A 3 is a signal to add a key with --header or log in with --login, then test again.
QAmcp vs MCP Inspector vs mcpspan
Three tools sit next to QAmcp, and each answers a different question. MCP Inspector is the official tool for connecting to a server and calling its tools by hand.
QAmcp runs a fixed set of checks automatically and gives a repeatable score. The two work well together: explore by hand, then score on every change.
The servers being tested are the same ones AI apps call every day. If you use Claude Code, you already rely on MCP servers — learning how to use Claude Code means learning which servers it can reach, and QAmcp tells you whether those servers behave.
| Tool | Question it answers | How it works |
|---|---|---|
| QAmcp | Does this server behave correctly? | Automatic scored checks; pass or fail with evidence |
| MCP Inspector | What does this server do when I call it? | Manual connection and hand-run tool calls |
| mcpspan | Which tools ran, and how did they perform? | Self-hosted analytics; docker compose dashboard |
The useaitools.me article from October 24, 2026 puts the pair neatly: "MCP analytics tells you which tools fire, a protocol test tells you they behave." mcpspan watches usage after the fact, and QAmcp checks correctness before your users meet the server. Neither replaces the other.
What Does QAmcp Cost?
The tester itself is free. The browser test, the command-line tool, the GitHub check, and the directory cost nothing, and the code is MIT licensed. The paid part is the monitor, which re-tests your server on a schedule and emails you when something breaks.
| Tier | Price | What you get |
|---|---|---|
| Free | $0 | 1 server; up check every hour; full test once a day; email alerts; no card required |
| Pro | $19/month founding price ($29 struck through) | 3 servers; up check every 5 minutes; full test every hour; email and Slack alerts; teammates; login servers |
| Agency | $99/month ($149 struck through), coming soon | 25 servers; 5 seats; reports with your logo — not yet available |
The founding Pro price is $19 a month for the first 50 customers, kept as long as they stay subscribed. Against the $29 list price that is $10 a month saved, or $120 over a year. The Agency tier is listed as "soon" with a "tell me when it's ready" button, so treat it as announced, not live.
The monitor's example alert shows why the schedule matters: a server dropping from 92 to 61 at 2:14 in the morning, with a fetch_notes description change and a get_order safety-label change. A daily manual test would find that hours later; the monitor finds it when it happens.
The Name Traps: QAmcp Is Not qa-mcp, qmcp, or QA-MCP
QAmcp shares its neighborhood with three similar names, and every one of them is a different project. If you install the wrong one, you get a different tool entirely — so check the name before you type it.
| Name | What it actually is | How to tell it apart |
|---|---|---|
| qa-mcp (joaopedro17) | Lets Claude create TestRail test cases from Jira or Azure DevOps stories | Commands are npx qa-mcp init, start, and doctor; needs TestRail and Jira accounts |
| qmcp (gabiteodoru) | MCP server for q/kdb+ databases | Installs with pip install qmcp; connects AI assistants to q servers |
| qmcp (quaternionmedia) | FastAPI server for tool discovery, invocation history, and human-in-the-loop steps | Listens on 127.0.0.1:3141 with a SQLite record; runs with uv run qmcp serve |
| QA-MCP (Atakan-Emre) | Generates standardized QA test cases and syncs them to Jira/Xray | Tools include testcase_generate and testcase_to_xray; installs with pip install qa-mcp |
None of the four tests MCP servers. qa-mcp and QA-MCP generate test cases for humans, and the two qmcp projects are MCP servers themselves — one for databases, one for human-in-the-loop workflows. The tester in this article is QAmcp, the qamcp-cli package from QAmcp on GitHub.
The confusion is easy to understand: "QA" and "MCP" are both common words in this space, and "qmcp" is one letter away from "qamcp". The safe check is the package name. If the install line does not say qamcp-cli, it is not this tool.
When QAmcp Fits — and When It Does Not
QAmcp fits anywhere an MCP server meets users you do not control. If your server is public, listed in a directory, or shipped to customers, a scored test on every change is cheap insurance. If your server is a weekend experiment on your own machine, the demo mode and --look-only give you the same report without the ceremony.
| Situation | Verdict |
|---|---|
| You publish or ship an MCP server | Use it — run the GitHub Action on every pull request |
| You list a server in a public directory | Use it — the score is a public claim you can back up |
| You connect to a third-party server | Use --look-only or --before-install first — check before you trust it |
| You build MCP servers for clients | Use it — the monitor's Pro tier covers 3 servers with alerts |
| You only run a private server for yourself | Optional — the demo and one-off runs are free, but the gate adds little |
| You need load testing or a full security audit | Not this tool — QAmcp checks behavior and safety labels, not scale or deep vulnerabilities |
The clearest fit is the team that ships a server and forgets it. Tool descriptions drift, a dependency changes a label, and an AI app starts behaving differently with no code change on your side. A score that drops on a pull request catches that drift at the moment it happens.
What QAmcp Is Not
QAmcp is a behavior tester, not a load tester. It checks whether tools answer correctly and safely; it does not tell you how many requests per second your server survives. That is a different test with different tools.
It is not a security audit either. It catches known hidden-instruction tricks and leaked secrets in error messages, but it does not review your code, your authentication design, or your infrastructure. A score of 100 means the server behaves well under these checks — not that it is invulnerable.
It is not a protocol, and it does not change how MCP itself works. Transports underneath MCP keep evolving — learning what MCP over MOQT is shows one direction that work is heading — but QAmcp tests the server on top of whatever transport it uses. The checks stay the same even as the wire changes.
Finally, it is not a popularity contest. A star count on a three-week-old repo says nothing about whether the checks are good; the checks are good if they catch real problems on real servers. Judge the tool by running it against a server you know, not by its star count.
Sources
Every fact in this article comes from a source fetched on November 5, 2026. Vendor claims are the vendor's own words.
QAmcp GitHub repo and README — https://github.com/QAmcp/qamcp-cli
QAmcp website, pricing, and monitor tiers — https://qamcp.dev
Visalytica listing for QAmcp — https://www.visalytica.com/tool/qamcp
Show HN, November 4, 2026 — https://news.ycombinator.com/item?id=50366061
Show HN, October 21, 2026 — https://news.ycombinator.com/item?id=50186988
What Your Agent Did, and How You Would Prove It — https://useaitools.me/blog/what-your-agent-did-and-how-you-prove-it-2026
my2cents.ai open-source agents digest — https://www.my2cents.ai/news/2026-10-22-open-source-agents/
VibeHacker digest entry for QAmcp, Show HN, Nov 4, 2026 — blurb verified from the VibeHacker search index on November 5, 2026; no dedicated page URL was retrievable.
Model Context Protocol introduction — https://modelcontextprotocol.io/introduction
qa-mcp by joaopedro17 — https://glama.ai/mcp/servers/joaopedro17/qa-mcp
qmcp by quaternionmedia — https://glama.ai/mcp/servers/quaternionmedia/qmcp
qmcp on PyPI, the q/kdb+ server — https://pypi.org/project/qmcp/0.7.0/
QA-MCP by Atakan-Emre — https://glama.ai/mcp/servers/Atakan-Emre/McpTestGenerator
Frequently Asked Questions
What is QAmcp?
QAmcp is a free, open-source tester for MCP servers. It connects the way an AI app does, runs a fixed set of checks, and reports a 0–100 score with evidence. The command is npx qamcp-cli <url>, and it needs Node 20 or newer.
Is QAmcp free?
Yes. The browser test, the command-line tool, the GitHub Action, and the directory are free, and the code is MIT licensed. Only the monitor has paid tiers: Pro is $19 a month at the founding price, and the Agency tier is not yet available.
Does QAmcp run tools that change data?
No. By default it runs only tools the server labels read-only, and it never runs a tool that sounds destructive, like delete_account, in any mode. With --look-only it calls no tools at all.
What is a good QAmcp score?
The README's rough guide is 90 or above for solid, 70 to 89 for works-but-fix-things, and below 70 for likely to struggle or a safety problem. A hidden instruction caps the score at 40, so always read the findings, not just the number.
How is QAmcp different from MCP Inspector?
MCP Inspector is the official tool for connecting to a server and calling its tools by hand, which is ideal for exploring and debugging. QAmcp runs a fixed set of checks automatically and gives a score you can repeat on every change. They work well together.
