Skip to content

LLM output evaluation providers

36 published entries state LLM output evaluation on their AI testing pages. Between them they name 9 AI product types.

Checks of model answers for quality and hallucination against a rubric, often with human review.

hallucination: Model output that is fluent but not supported by facts or sources.
AI product type
Company type
Country (HQ or delivery)

Showing 36 of 36. Default order: coverage score, then name. How the score works.

Entries on this page: LLM output evaluation providers
RankCompany typeAI products (as stated)ServicesVerified
1DeviQAPure-play QA companyPoland, Warsaw
LLM applicationsAI agentsRecommender and ranking systemsPredictive ML modelsComputer visionVoice AI
82010162026-11-09
2AbstractaPure-play QA companyUnited States, Jacksonville
LLM applicationsAI agentsPredictive ML models
102008152026-11-09
2ImpactQAPure-play QA companyUnited States
LLM applicationsAI agentsComputer visionVoice AIRecommender and ranking systems
9152026-11-09
4BetterQAPure-play QA companyRomania, Cluj-Napoca
LLM applicationsAI agentsVoice AI
92018142026-11-09
4QATestLabPure-play QA companyCyprus
LLM applicationsAI agentsPredictive ML models
10142026-11-09
4TestDevLabPure-play QA companyLatvia, Riga
LLM applicationsAI agentsVoice AIComputer visionPredictive ML modelsRecommender and ranking systemsAutonomous and embedded AI
52011142026-11-09
4TestingXpertsPure-play QA companyUnited States
LLM applicationsAI agentsVoice AI
10142026-11-09
4TestlioCrowdtesting platformUnited States, Austin
LLM applicationsAI agentsPredictive ML modelsRecommender and ranking systems
82012142026-11-09
4TestriqPure-play QA companyIndia, Mira-Bhayandar
LLM applicationsAI agentsPredictive ML models
92010142026-11-09
10ITRex GroupIT services firm with a QA practiceUnited States, Sacramento
LLM applicationsAI agentsComputer visionPredictive ML modelsRecommender and ranking systems
62009132026-11-09
11QualityAI (formerly Qualitest)Pure-play QA companyUnited States, Santa Clara
LLM applicationsAI agentsPredictive ML modelsRecommender and ranking systems
7122026-11-09
11SunTec IndiaData and evaluation vendorIndia, New Delhi
LLM applicationsDocument AIPredictive ML models
71999122026-11-09
14LionbridgeData and evaluation vendorUnited States, Waltham
LLM applicationsVoice AIComputer visionRecommender and ranking systems
51996112026-11-09
14QA MadnessPure-play QA companyPoland, Warsaw
LLM applicationsAI agentsRecommender and ranking systems
7112026-11-09
14QAwerkPure-play QA companyUkraine, Kyiv
LLM applicationsAI agents
72015112026-11-09
14Testrig TechnologiesPure-play QA companyUnited States, Dallas
Predictive ML models
82015112026-11-09
18Indium SoftwareIT services firm with a QA practiceUnited States, Cupertino
AI agentsLLM applicationsRecommender and ranking systems
6102026-11-09
18KiwiQAPure-play QA companyAustralia, Bella Vista
LLM applications
72009102026-11-09
18ShaipData and evaluation vendorUnited States, New York
LLM applicationsAI agentsVoice AI
6102026-11-09
23AppSierraIT services firm with a QA practiceIndia, Noida
LLM applicationsAI agentsPredictive ML models
592026-11-09
23CodoidPure-play QA companyIndia, Chennai
LLM applicationsAI agents
5201292026-11-09
23QASourcePure-play QA companyUnited States, Pleasanton
LLM applicationsAI agents
5200292026-11-09
29AIDX TechAI assurance and audit specialistSingapore, Singapore
LLM applicationsAI agentsVoice AIComputer vision
482026-11-09
29ApplauseCrowdtesting platformUnited States
LLM applicationsAI agentsVoice AI
482026-11-09
29SG AnalyticsIT services firm with a QA practiceIndia, Pune
LLM applications
5200782026-11-09
34CentificData and evaluation vendorUnited States, Redmond
AI agentsComputer vision
472026-11-09
40AppenData and evaluation vendorAustralia
LLM applications
562026-11-09
40InnodataData and evaluation vendorUnited States, Ridgefield Park
LLM applicationsGenerative media
462026-11-09
40KainosIT services firm with a QA practiceUnited Kingdom, Belfast
LLM applications
31001-500062026-11-09
50Digital Divide DataData and evaluation vendorUnited States, New York
LLM applicationsVoice AI
352026-11-09
50Knovel EngineeringIT services firm with a QA practiceSingapore, Singapore
LLM applications
452026-11-09
50QAlifiedPure-play QA companyUruguay, Montevideo
LLM applications
352026-11-09
61TolokaData and evaluation vendorNetherlands
LLM applicationsAI agents
242026-11-09
74SamaData and evaluation vendorUnited States, San Francisco
LLM applicationsGenerative media
132026-11-09
74TesvanPure-play QA companyArmenia, Sevan
LLM applications
132026-11-09
74VeriServePure-play QA companyJapan, Tokyo
LLM applications
232026-11-09

Product types covered by vendors stating this service

Count of entries on this page that state each product type.

Product types stated by entries offering LLM output evaluation
Product typeLLM output evaluation
LLM applications34
AI agents22
Voice AI10
Computer vision7
Predictive ML models11
Recommender and ranking systems9
Generative media2
Document AI1
Autonomous and embedded AI1

Scope and standards referenced

Hallucination and factuality checks, answer quality rubrics, LLM-as-judge with human review.

The directory lists which vendors state this service. It does not test whether a vendor applies these standards.

Questions

What does LLM testing cover?

LLM testing covers answer quality and factual accuracy, hallucination, retrieval quality for RAG systems, regression across prompt and model versions, adversarial robustness, latency and cost. 36 published entries state LLM output evaluation.

How is an LLM application tested before release?

Teams build an evaluation set from real use cases, score answers against a rubric, run adversarial tests and load tests, and set acceptance thresholds. The same set is rerun after every prompt, model or data change.

How is LLM testing different from LLM evaluation?

Evaluation measures quality on a dataset with metrics or graders. Testing uses those measurements to accept or reject a release and adds functional, security and performance checks around the model. In practice vendors sell both together.

How do testers handle non-deterministic LLM output?

They assert on meaning instead of exact text, run each case several times and use score thresholds. Settings such as temperature are fixed during tests, and results are tracked as rates across runs.

What is hallucination testing?

Hallucination testing checks whether a model states things that are not supported by its sources or by facts. Answers are compared with reference answers or retrieved documents, and unsupported claims are counted as failures.

Request a listing or correction