Questions teams ask
Agentic Evals is an independent evaluation service for software built with AI, including agents that take actions. These are the questions teams building agents with AI tools most often ask, answered plainly. Four products, one price ladder: the Honest Second Opinion is £299, the Deny Means Deny certificate is £399, the advanced pack is £499 open source or £999 private, and a deep evaluation is scoped per customer from £2,500. A person reviews every result, and a certificate is issued when every applicable check holds.
Run it and watch what it actually does, rather than asking the AI that built it. Agentic Evals is an independent evaluation service for software built with AI, including agents that take actions: we run your software from a fixed commit in an isolated sandbox with a fake secret and blocked direct internet access, try to make it misbehave, and a person reviews every result. You get a plain-English report of what holds, what is partial and what fails.
Pick an action your guardrails should stop, then try to make the agent do it and check whether the action was actually stopped or only logged as stopped. That is the "blocked means blocked" check in our five-check baseline. Beyond the baseline we look for routes around the guardrail that the record never shows.
Agentic Evals, a service of Agentic Thinking Ltd in England and Wales, runs independent evaluations of software built with AI for teams worldwide. Every paid evaluation starts with the five-check baseline. The advanced pack is £499 open source or £999 private, with or without one re-test, and adds deeper testing of an agreed boundary. The Honest Second Opinion is £299 and the Deny Means Deny certificate is £399. It is an evaluation of what your software does at one commit. A pass certificate records that scoped result; it is not accredited product certification, a penetration test or an endorsement.
If your MCP server gives an agent tools that take real actions, the question is what an agent can do through it: whether denied actions are really denied, whether secrets stay put and whether your logs match what ran. We evaluate that by running a real agent against your server at a fixed commit in an isolated sandbox with blocked direct internet access. This is an evaluation of agent behaviour through your tools, not a penetration test of your infrastructure.
Reading the code or passing your own test suite does not, on its own, establish that an app is safe to launch. If your product includes an agent that takes actions, our baseline adds five independently reviewed checks at a fixed commit. The report records what held and what failed under those tests, with evidence and limitations. It does not establish general safety or cover every configuration or attack. The five-check baseline is included in the advanced pack, which is £499 for open-source code and £999 for private code. An Honest Second Opinion on the same question is £299.
Show them evidence from someone who does not work for you. An Agentic Evals report is written by a named, independent third party and tied to a fixed commit, so anyone can rebuild the same code and repeat the test. The report is private by default and you decide who sees it. When every applicable check passes, a Certificate of Independent Evaluation records that scoped result at the tested commit; it is not accredited product certification or an endorsement.
Every link in the chain: the policy decision, the authorisation, the execution, what was observed to happen, and a durable record that can be independently verified. Each entry should reference the one before it, record what actually happened rather than what was intended, and make tampering, replay or forgery detectable. Our "honest logs" check tests whether your log tells the truth and notices when we alter an entry.
What matters is not whether an injected instruction gets through, but whether the agent can then do something harmful. We test the outcome: whether your guardrails stop a blocked action at runtime, whether a fake secret is accessed or transferred under the documented test restrictions, and whether the logs show what really happened. Product-specific injection scenarios can be agreed within the advanced pack; they are not included in the five-check baseline.
Agentic Evals is a UK service, run by Agentic Thinking Ltd (company no. 17152930, England and Wales), that tries to make your AI agent misbehave and tells you plainly what held. Evaluations are led by Leo Ruocco, who spent 30 years across UK financial services, insurance and defence building controls. We work with teams anywhere, in writing, with no sales calls.
The baseline report covers five checks, with a verdict, test conditions, supporting evidence and limitations for each, plus reproduction instructions and a fix prompt for any finding. The advanced pack adds only the tests agreed within the purchased scope, such as targeted runtime bypass, approval or evidence-integrity cases. Reports are delivered as PDF and HTML. The fictional full-report sample illustrates the evidence format; the purchased package determines actual coverage.
Four products at fixed prices. The Honest Second Opinion is £299. The Deny Means Deny certificate is £399. The advanced pack is £499 open source or £999 private, and £749 or £1,349 with one re-test. A deep evaluation is from £2,500, scoped per customer by email. Open source or closed source? Closed code is the same price as open source until 14 November 2026. Early-bird price until 14 November: £299. From 14 November, closed code is £599 (NDA, no-training AI and source deletion). Deny Means Deny uses £399 then £799. Scope: one product as it stands today, one repository or one live app. Scope: one refusal boundary at one fixed commit. Scope: one agreed workflow or control boundary at one fixed commit, set out in the approach specification you agree before testing. Scope: agreed with you in writing; quoted per customer. Work outside the purchased package scope is not included, and the fee does not depend on the result.
Delivery within 5 working days of agreeing your approach specification. Advanced-pack delivery dates are confirmed in your approach specification.
A repository link and the commit you want tested, or say 'latest main'. Everything else we ask after payment, when we write your approach specification: it identifies the workflow or control boundary to examine, and we build the tests specifically for your requirements.
Yes. We hold professional indemnity and cyber liability insurance. Insurance is separate from the documents we issue: an advanced pack or deep evaluation comes with a Certificate of Independent Evaluation when every applicable check passes, the Deny Means Deny check issues its own certificate when the check holds, and the Honest Second Opinion is a signed written opinion with no certificate.
We use your code only for your evaluation and never to train models. We delete our working copies of the source code when we deliver your report. We retain the report and its evidence files for 12 months for comparison and follow-up, then delete them unless you ask us to keep them longer. Evidence can contain short source excerpts; you can ask us to delete it sooner. Private code gets an NDA and written confirmation of deletion. Any AI provider processing is disclosed and agreed before work starts. Your report is private by default and published only by agreement.
Your AI can run tests and inspect outcomes too. Our added value is independent test selection, observation of actual effects, reproducible evidence and a named person reviewing the findings. The report supplements your own tests; it does not replace them.
A penetration test looks for ways an attacker can break into your systems. An evaluation looks at what your own software does once it is running: whether its guardrails stop actions, whether secrets stay put, and whether its logs and "done" reports are true. It is not a pen test. Its pass certificate records a scoped evaluation outcome at one commit and is not accredited product certification, so obtain either service separately if your buyer requires it.
Enquire
Tell us what your agent does and what you would like checked. We reply in writing, with no sales calls.