LLM Audit report
The LLM Audit answers a simple question: when an AI engine talks about my brand, is it telling the truth?
It cross-references LLM answers against your Google Search Console data, your structured content, and an LLM-as-judge evaluator to produce a confidence-graded audit report.
Open it from the left sidebar → LLM Audit.
What it checks
For every prompt where your brand is mentioned, the audit evaluates:
| Check | Question answered |
|---|---|
| Factual accuracy | Are the claims about your brand correct (services, locations, hours, products)? |
| Citation match | Are the cited URLs pages you actually own, or third-party? |
| Position truthfulness | Does the LLM's claim about your position/ranking match your real SERP position from GSC? |
| Entity recall | Did the LLM use your canonical brand name or a stale alias? |
| Recency | Is the information from the last 12 months, or is the LLM citing outdated pages? |
Each check returns a verdict (Pass / Watch / Fail) and a short evidence snippet.
Confidence grading
Every audit row carries a confidence badge — High, Medium, or Low — produced by our LLM-as-judge evaluator. The grading is qualitative rather than a fixed numeric threshold: the judge weighs how many independent signals (GSC, on-page content, citation match, recency) agree with the LLM's claim and how strong each signal is.
- High — multiple independent signals agree (GSC + on-page + judge).
- Medium — partial agreement or limited evidence.
- Low — single signal or the judge expressed uncertainty.
Treat Low rows as leads to verify manually rather than as definitive findings.
Requirements
The audit needs ground truth to compare against:
- Google Search Console must be connected and a property selected for the site.
- At least one completed measurement cycle in the last 30 days.
- Your brand entity must be configured (canonical name + aliases) so the judge knows what to look for.
Missing GSC will limit the audit to on-page and judge signals only.
How to use the report
- Sort by Fail → Watch → Pass to surface the most damaging errors first.
- Open a Fail row to read the judge's evidence and the original LLM answer side by side.
- Decide on a fix:
- Factual error → update your website (about page, locations, hours, pricing).
- Wrong citation → improve the page the LLM should be citing, or add canonical/structured-data hints.
- Outdated info → publish a fresh page or update the existing one with a visible "last updated" date.
- Re-run the audit after your next measurement cycle to confirm the fix.
Notes
- The audit does not consume prompt quota — it reuses LLM runs you already paid for.
- Running an audit does not consume AI Chat tokens — audit runs are included in your plan at no extra cost.
- Audit history is retained for the standard plan retention window (3 / 4 / 5 years depending on tier).