Reading the HalluHard Paper: Why Claude Hallucinates More in Long Conversations, and 4 Implementation Patterns for Claude Code/MCP
Past three conversational turns, even the strongest models hallucinate ~30%. Reading the HalluHard paper against 1000 hours of Claude Code in production: what actually breaks long conversations, plus four concrete fixes — context compression, verification turn, reset & re-prompt, memory+retrieval hybrid — with implementation code.
Past three conversational turns, even the strongest models hallucinate ~30%. Reading the HalluHard paper against 1000 hours of Claude Code in production: what actually breaks long conversations, plus four concrete fixes — context compression, verification turn, reset & re-prompt, memory+retrieval hybrid — with implementation code.
Reading the HalluHard Paper: Why Claude Hallucinates More in Long Conversations, and 4 Implementation Patterns for Claude Code/MCP
Version note: This article is based on information as of May 2026. HalluHard is by Fan, Delsad, Flammarion, and Andriushchenko (arXiv: 2602.01031, 2026). The evaluation includes Claude Opus 4.5, GPT-5.2, and Gemini 3.x. The implementation patterns assume Claude Code v2.x and MCP 0.5. Numbers and model lineups change quickly, so refer to the original paper for the latest.
I started my career as a semiconductor process engineer, where TCAD simulations taught me daily that a single mis-set boundary condition can move the result by an order of magnitude. So when I first saw the HalluHard numbers, I had a strange feeling of déjà vu. A reasoning loop blowing up the moment boundary conditions degrade looks structurally the same in semiconductor processes and in LLM conversations. Below I cross-check the paper's findings against what I have actually hit while running Claude Code for around a thousand hours, and document the four mitigation patterns that hold up in practice.
TL;DR: What You'll Learn
| Topic | What you'll learn |
|---|---|
| HalluHard experimental design | 950 seed questions × 3-turn conversations across legal/research/medical/coding |
| The headline number | Even Claude Opus 4.5 with web search hits ~30% hallucination by turn 3, with 3–20% of bad references persisting |
| Why it happens | Self-conditioning — the model treats its own previous output as ground truth |
| What I see in practice | Claude Code fabricates non-existent file paths and API arguments in long sessions |
| Pattern 1 | Context Budget Management (compaction + summarization) |
| Pattern 2 | Verification Turn (grep / file-lookup fact-check loops) |
| Pattern 3 | Reset & Re-prompt (snapshot state every n turns, resume fresh) |
| Pattern 4 | Memory + Retrieval hybrid (CLAUDE.md + MCP fetch) |
| 1000-hour takeaway | Conversation design beats waiting for a smarter model |
The "Turn 3 Cliff" HalluHard Exposes
I don't read every new paper as it drops. I picked up HalluHard because a Japanese surgeon I follow on note.com summarized it, his post pulled 59 likes, and I went straight to arXiv that evening.
The core of the paper:
- 950 seed questions split into legal 250 / research 250 / medical 250 / coding 200
- For each seed, the authors generate two follow-up questions conditioned on the model's prior answer, giving a 3-turn conversation
- Metrics: hallucination rate, error-propagation rate, and self-correction rate
- Even Claude Opus 4.5 with web search ends turn 3 around a 30% hallucination rate
- The key finding: 3 to 20% of incorrect references reappear verbatim in later turns
That last number was what made me sit up. It's exactly the long-Claude-Code-session feeling — around turn 50 the assistant says "I'll update the file lib/auth/session.ts we proposed earlier" and I respond, "There is no such file." The paper basically measured that.
Why Long Conversations Generate More Lies
The way I decompose it:
1. Self-conditioning. An LLM also reads its own prior outputs. A guessed function name in turn 1 enters turn 2 as if it were settled fact. This is structurally similar to TCAD, where an initial boundary error gets amplified through time-stepping.
2. Attention dilution. As the token budget grows, the share of attention available for the actually-important instructions (system prompt, original user goal) shrinks. Honest "let me re-confirm the requirement" turns become rarer late in the session, and I suspect this is a chunk of why.
3. No native verification. Even human meetings drift after three hours — "wait, who decided that?" An LLM has no built-in habit of double-checking its own claims. Unless the app layer adds verification, distribution skew accumulates over time.
In semiconductor process work I have repeatedly seen the field confidently report "the equipment is bad" while TCAD re-simulation showed it was a boundary-condition issue. Confident misreports are only detectable through a second lens. LLM hallucinations follow exactly the same shape.
What Fabrication Actually Looks Like in Claude Code
To make it concrete, here are the patterns I hit most often over a thousand hours:
| Symptom | Typical turn range | Tell |
|---|---|---|
| Editing a file that doesn't exist | 30–80 | "I'll update src/utils/foo.ts we discussed earlier" |
| Re-pitching an option I rejected | 20–60 | "Let's use the auth flow I previously proposed" |
| Invented SDK arguments | any | client.messages.create({ tools_choice: ... }) etc. |
| Version drift | long sessions | Mixing Claude Code v1 and v2 behavior |
| Promoting its own guess to "decision" | 50+ | "Per our decision to use approach A…" (we never decided) |
None of this is fixed by "a smarter model". It is fixed by conversation design and external verification. The HalluHard "30% at turn 3, then propagation" curve maps onto my lived experience almost exactly.
Pattern 1: Context Budget Management
This is the highest-leverage one. Split the long conversation explicitly into a fact layer (CLAUDE.md, knowledge/decisions/, external files) and a dialogue layer (compacted every n turns).
A trimmed version of the compaction script I actually run:
// scripts/compact-session.ts
import Anthropic from '@anthropic-ai/sdk'
import { readFile, writeFile } from 'node:fs/promises'
const client = new Anthropic()
type Turn = { role: 'user' | 'assistant'; content: string }
const COMPACT_PROMPT = `
You are summarizing a long Claude Code session.
Output a JSON with:
- decisions: facts the user explicitly confirmed (verbatim if possible)
- open_questions: items still undecided
- rejected: ideas the user rejected (must NOT be re-suggested)
- artifacts: file paths actually created or edited (verify before listing)
Do NOT include speculative file names, function signatures, or numbers.
If unsure, omit the item rather than guess.
`
export async function compact(turns: Turn[]): Promise<string> {
const transcript = turns.map((t) => `<${t.role}>\n${t.content}\n</${t.role}>`).join('\n\n')
const res = await client.messages.create({
model: 'claude-opus-4-7',
max_tokens: 4000,
system: COMPACT_PROMPT,
messages: [{ role: 'user', content: transcript }],
})
const text = res.content
.filter((c) => c.type === 'text')
.map((c) => c.text)
.join('\n')
await writeFile('knowledge/session-summary.json', text, 'utf-8')
return text
}
The crucial field is rejected. The HalluHard "propagation" finding is exactly what this is designed to stop: keep a persistent reject list at the top of every turn, and rejected ideas almost stop returning.
Pattern 2: Verification Turn
Physically verify the LLM's proposal with code before accepting it. In Claude Code, hooks are the cleanest way.
# .claude/hooks/pre-edit-verify.yml
on: PreToolUse
match:
tool: Edit
run: |
TARGET="$CLAUDE_TOOL_INPUT_file_path"
if [ ! -f "$TARGET" ]; then
echo "BLOCK: file not found: $TARGET" >&2
exit 2
fi
This three-liner alone kills the "edit a fictional file" hallucination. For SDK arguments I lean on grep against type defs:
// scripts/verify-sdk-call.ts
import { execSync } from 'node:child_process'
// Check whether a proposed argument exists in the real SDK types
export function verifyAnthropicArg(arg: string): boolean {
const grep = execSync(`grep -r "${arg}" node_modules/@anthropic-ai/sdk/dist/`, {
encoding: 'utf-8',
})
return grep.trim().length > 0
}
The point of this pattern is delegating the final say to grep and the type system, not to the LLM's self-report. You trust the model's probability distribution only inside the perimeter that the file system and type checker have already validated.
Pattern 3: Reset & Re-prompt
More of an operational habit. I deliberately fold the conversation every 30 to 50 turns, snapshot state, and resume in a new session.
# scripts/snapshot-and-restart.sh
#!/usr/bin/env bash
set -euo pipefail
# 1) Pin session summary to knowledge/
claude -p "compact this session into knowledge/session-summary.json" --no-stream
# 2) Pin work-in-progress git diff
git diff > knowledge/wip-diff.patch
git status --porcelain > knowledge/wip-status.txt
# 3) Resume "from the current state" in a fresh session
claude --resume-from knowledge/session-summary.json
I used to assume longer conversations meant deeper context. HalluHard convinced me the marginal return of a long session is negative. Write the context to disk and restart. It's the same discipline I learned in investment reviews: better to sleep on a deal and re-read the memo in the morning than to push through three straight hours of debate.
Pattern 4: Memory + Retrieval Hybrid
The last pattern: keep facts outside the conversation. CLAUDE.md, knowledge/, or MCP fetch — the medium doesn't matter, but the principle does: re-fetch the grounding evidence from outside the model every time.
// scripts/retrieve-fact.ts
import { readFile } from 'node:fs/promises'
import { glob } from 'glob'
export async function pullFact(query: string): Promise<string> {
const files = await glob('knowledge/decisions/**/*.md')
const matches: string[] = []
for (const f of files) {
const body = await readFile(f, 'utf-8')
if (body.toLowerCase().includes(query.toLowerCase())) {
matches.push(`### ${f}\n${body}`)
}
}
return matches.join('\n\n---\n\n').slice(0, 6000)
}
Wrap this as an MCP server and expose it as mcp__zyl0__pull_fact. Every time Claude says "let me confirm the prior decision," a confirmed fact is injected from outside the probability distribution. The most stable design I have found is one that does not trust the LLM's internal memory at all.
Gotchas
- Over-compaction creates a different hallucination. If you compress the summary too aggressively, Claude starts re-inventing the omitted premises. Keep decisions verbatim.
- Strict verification hooks can halt development. My first cut blocked every
Editat PreToolUse and broke new-file creation. Separate policies forWritevsEdit. - MCP fetch cost is easy to miss. Fetching external files every turn raises both tokens and latency. I instruct Claude in the system prompt to fetch decisions only when needed.
- Reset thresholds are project-dependent. Currently ~50 turns for writing, ~30 for coding, ~80 for research. Better still: cut at task boundaries, not turn counts.
1000-Hour Takeaway: Conversation Design Drives Yield
The most uncomfortable insight from HalluHard is that "just wait for a smarter model" is structurally a losing bet. If Opus 4.5 with web search hallucinates at 30%, 5.0 won't fix the root, because self-conditioning is a structural property of the chat protocol, not a parameter you can scale away.
When I traced yield problems in semiconductor process work, what actually moved the needle was building a TCAD environment to add a second lens beyond the field's self-report. For an LLM, the equivalent of TCAD is CLAUDE.md, git, and grep. Don't lean on the conversation alone — bring in the file system and the type checker as the second lens. Yield improves visibly.
The note.com post that pointed me to HalluHard ended with the line "the longer the conversation, the more the errors stick around." It matches my lived experience exactly. So my current operating rule is just one sentence: when in doubt, fold the conversation and write to disk. It is the same discipline I learned in investment reviews: when in doubt, write the memo. Don't carry the argument verbally.
Next Issue Ideas
- Idea 1: A data-updated follow-up — Revisit the thesis with fresh numbers and separate what strengthened from what broke.
- Idea 2: A practical template edition — Turn the article into a checklist readers can reuse for investing, pricing, or technical operations.
- Idea 3: The bear-case test — Define the conditions that would invalidate this view and map the indicators to watch next.
Related Posts
- GPT-5.5 Is Finally "AI That Works at Work" — But Mythos Is Still Far Ahead — A deep comparison of OpenAI's newly announced GPT-5.5 against Claude Mythos Preview and Gemini 3.1 Pro — strategic div…
- Running AI Agents as an Organization: A 12-Subagent, Neon-Backed Postgres Task Board Architecture for a One-Person Company — The era of letting one AI do everything is over.
- Claude Code 2026 Feature Roundup: Putting 1M Context, Skills, Hooks, Subagents, MCP, and Plan Mode to Real Work — Six Claude Code features ordered by adoption priority — 1M context, Skills, Hooks, Subagents, MCP, and Plan Mode — wit…
References
The factual and numerical assumptions in this article are anchored to primary or public materials that readers can revisit during AdSense review and future updates.
- Anthropic Claude Code documentation
- arXiv computer science papers
- Model Context Protocol documentation
This article is for informational purposes only and does not constitute investment advice or a recommendation of any specific stock, service, or contract structure. The author may hold positions or interests related to companies or services mentioned. Generative AI was used for parts of research, translation, and proofreading, with final review by ZYL0. See the disclaimer for details.
