Ask ChatGPT, Gemini and Claude the same health question and the answers look almost identical: a confident intro, a numbered list, and "consult a professional" at the end.
The real differences are underneath. One spots an important exception. One invents a source. One gives an answer that's technically right but impossible to actually use.
That's what the AI Lab is for. Not to crown a favourite model, not to secretly advertise anyone. Just to show what a normal person would actually get, what checks out, and what still needs a human professional.
Here are my rules.
1. Question first, no favourite
I start with a practical question a real adult would ask about training, food, recovery or sleep. Specific enough to check. "How do I get healthier?" is useless. "How should a beginner split walking and strength work across a week?" gives claims I can verify.
I write the scoring rules before I run the test. Otherwise it's too tempting to change the rules once I see which answer I like.
2. Same prompt for everyone
I record the exact prompt, the product, the model name the provider shows, the date, and the relevant settings. If one model searched the web and the others didn't, I say so. If a product doesn't tell me which model it uses, I don't guess.
Every model gets a fresh chat, so old conversations can't influence the answer. I keep the original output. If I shorten something for the article, I mark it, and it never changes the meaning.
Follow-up questions are recorded separately. A model that only gets it right after three corrections did not do as well as one that got it right first time.
3. Sounding smart is not evidence
I check the important health claims against sources outside the AI answer: public health agencies, government health institutes, recognised medical organisations, systematic reviews, original research.
A citation gets no points just for looking academic. I open it. Does it exist? Does it say what the model claims? Does it apply to the people in the question?
4. Safety, but not panic
The best answer isn't the longest or the most cautious. It's the one that flags the safety issues that actually change the decision, without turning every normal question into an emergency.
I look for sensible boundaries: symptoms that need urgent help, conditions that need an adapted plan, and situations where general advice isn't enough. And whether the model admits when the evidence is thin.
5. A score belongs to one test
When I give scores, they apply to that prompt, that date, those settings. AI products change constantly, and the same question can give a different answer tomorrow. A model that's great at training plans can be bad at supplements. So you won't read "X is the smartest AI" here.
Categories can include relevance, factual accuracy, source quality, safety, honesty about uncertainty, and how usable the answer is. The scoring is explained before the result.
What I will never do
- invent model outputs,
- claim access I didn't have,
- hide a sponsorship,
- rewrite a weak answer to make it look better,
- treat "all the AIs agree" as proof. They can all repeat the same popular mistake.
You should be able to check me
Every test leaves a trail: prompt, date, model info, the relevant output, the scoring rules and the sources I used to verify. You're welcome to disagree with my judgement. But you should be able to see exactly how I got there.
The full rules are on how I test AI answers. The tests so far:
- Claude vs Gemini: Which Beginner Fitness Plan Survived a Source Check?
- GPT vs Gemini vs Claude: Which AI Gave the Safest Water Advice?
- GPT vs Gemini vs Claude: Is Brisk Walking Enough After 50?