--- title: "Law Firm AI Search Indexing: Four Retrieval Pipeline Tests - Carlos Arias" description: "Law firm AI search indexing, tested in an afternoon. Four checks that tell you whether ChatGPT and Claude can retrieve your practice pages at all." url: "https://carlosarias.com/blog/law-firm-marketing/law-firm-ai-retrieval-pipeline" --- [Law Firm Marketing](/blog/categories/law-firm-marketing) # Law Firm AI Search Indexing: Four Retrieval Pipeline Tests Law firm AI search indexing, tested in an afternoon. Four checks that tell you whether ChatGPT and Claude can retrieve your practice pages at all. [Carlos Arias](/blog/authors/carlos-arias) · September 21, 2026 · 7 min read ![Sweeping black brush strokes on cream textured paper, with a small red square seal in the lower right corner.](/_astro/cover.DnxI5tiu_Z1knVb.webp) *Sweeping black brush strokes on cream textured paper, with a small red square seal in the lower right corner. AI-generated illustration by Carlos Arias .* Law firm AI search indexing has one question underneath it, and it is not whether you rank. It is whether a retrieval system can find your practice-area page at all when a person describes their matter to an assistant. You can test that yourself this afternoon, free, in under an hour. Four checks, in a fixed order, because each one only means something if the one before it passed. Most principals skip to the last question: does ChatGPT name my firm. That is the citation question, and it sits at the end of a pipeline with three earlier stages. Fail stage one and no amount of copywriting touches stage four. Legal has less room to wait this out than other verticals. Martindale-Avvo’s 2026 State of the Legal Consumer report put AI Overview trigger rates for legal queries near 78%, the highest of any category they measured. One vendor’s sample. Treat it as a direction. ## Retrieval is not citation, and the distinction is the whole article An answer engine does four things in sequence. It retrieves a candidate set of pages. It extracts a passage from them. It compresses that passage against every competing passage. Then it decides whether to attach a visible link. Chris Green’s reporting at Search Engine Land is the clearest public accounting of what sits behind ChatGPT in particular: a discovery index, a reading cache that holds full copies of pages already fetched, plus a small number of pages opened live mid-answer. Three layers, and your estate-planning page can be absent from all of them while holding position two in Google. Green also ran 1,000 prompts across 9,946 runs and found the retrieval source itself is not stable. One pipeline handled 88.1% of primary searches, but 11.6% of prompts switched source across repeats, and when the source switched, URL overlap between runs fell roughly 45%. Read that number twice. It is the reason a single test run is worthless, and the reason any vendor showing you one screenshot of ChatGPT naming your firm is showing you a coin flip. ## Test one: the snippet test that settles law firm AI search indexing This is the cheapest diagnostic in the discipline and almost nobody runs it. The method comes from Chris Green at Search Engine Journal: take a distinctive snippet of text from the page, put it in quote marks, and ask the assistant to find exact matches for it on the web. If your URL comes back, retrieval is not your problem. The snippet choice is where firms get this wrong. It has to be a sentence that exists nowhere else on the internet. “We fight aggressively for injured Floridians” appears on ten thousand pages and proves nothing. A sentence like “Florida’s four-year negligence deadline under section 95.11(3) was shortened to two years for causes of action accruing on or after March 24, 2023” is specific enough that a match is a real signal. If your practice page contains no sentence that unusual, you have already learned something, and it is not a good thing. Run it as two variants, in this order, and log both: - Variant A, no URL given. Paste only the quoted snippet and ask which page on the web contains it. This tests the index. - Variant B, URL given. Paste your URL and ask the assistant to read the page and quote the sentence back. This tests fetchability. A pass on B with a fail on A is the pattern that gets misread. It means the machine can reach your page when handed the address, and cannot find it when a client describes their problem. Nobody hands ChatGPT your URL. That is the whole point of the exercise. Run each variant five times, in ChatGPT with search enabled and in Claude with web search enabled, before you conclude anything. Non-determinism is not an excuse for sloppy testing, it is the reason for repetition. ## Test two: what the crawler sees with JavaScript switched off Vercel’s analysis of AI crawler behavior found GPTBot and ClaudeBot fetch JavaScript files and never execute them. Googlebot renders. The retrieval crawlers, largely, do not. curl -sL https://yourfirm.com/practice-areas/wrongful-death/ | grep -c "statute" Run that against your three most valuable practice pages. If your governing statute and your deadline language are missing from the raw response, they do not exist to the systems your prospective clients now ask first. Architecture, not writing. The full four-pillar version is in machine-first architecture for law firm websites. ## Test three: a prompt panel built on matter types, not keywords Now you test demand-side, and this is where most firms quietly discover their content targets the wrong thing. Write 30 to 50 prompts in the language a person in trouble actually types. Not “boca raton wrongful death lawyer.” Something closer to “my husband died in a forklift accident at work in Florida and the company says workers comp is the only option.” Each prompt maps to one matter type you actually want. If you want commercial landlord disputes and not residential evictions, the panel has to contain that distinction, because ChatGPT’s fan-out behavior breaks one question into multiple synthetic sub-queries before retrieving anything. Your page is competing against passages, not against homepages. Log three states per run. Cited with a link. Named without a link. Absent. That middle state matters more than people assume. A named firm gets typed into Google an hour later and arrives as branded organic, which nothing in your analytics will ever attribute correctly. The measurement side is its own problem, and it is worked through in what you can and cannot measure. ## Test four: which bots you are actually admitting This is the stage that gets blocked by accident. OpenAI runs three separate agents with three separate robots.txt strings. GPTBot collects training data, OAI-SearchBot builds the index ChatGPT search cites from, and ChatGPT-User fetches a page when a user’s question triggers it. Anthropic mirrors the split. ClaudeBot trains. Claude-SearchBot indexes for search. Claude-User fetches when a person’s question points at your page. Conflate them and you lose the thing you wanted to keep. A September 2026 sweep of the Tranco top 5,000 found 535 sites disallowing GPTBot, and 238 of them, 44.5%, disallowing OAI-SearchBot alongside it. The search bot went out with the training bot. Those distinctions are the policy lever. A firm can allow OAI-SearchBot and Claude-SearchBot while disallowing GPTBot and ClaudeBot, which is a coherent position: be findable, decline to be training data. Anthropic states all three of its bots honor robots.txt, including the user-triggered one. OpenAI draws that line differently for ChatGPT-User. Know which line applies before you write a rule. Then check the layer above robots.txt. Open your server logs and grep for OAI-SearchBot and Claude-SearchBot over the last 90 days. Zero hits with a permissive robots.txt means something in front of your origin is refusing them, usually a WAF rule or a bot-management default nobody documented. Often nobody chose it. Cloudflare has blocked AI crawlers by default since 1 July 2025 and asks every newly onboarded domain whether to allow them. Roughly a fifth of web traffic passes through that one default. ## What a non-retrieval result actually tells you Map the result to the structural fix. Do not skip the diagnosis and buy content. One of the four rows below is a writing project. The other three are engineering. - Fails variant B (cannot fetch by URL). Access problem. Robots.txt, WAF, bot management, or a 403 to non-browser user agents. Fix infrastructure before anything else. - Passes B, fails A (fetchable, not indexed). You are not in the discovery index. The usual causes are client-side rendering and text so generic there is nothing distinctive to index. Orphan pages do it too. - Passes A, absent from the prompt panel. Indexed, not selected. This is a content-substance problem, and it is the only one of the four where rewriting is the right answer. - Named without a link, repeatedly. You are in the model’s understanding of the market but not in the citation set. Usually an entity-consistency problem between your bar profile and your Google Business Profile. ## The fixes that do nothing Publish an llms.txt and wait. Ahrefs studied 137,000 sites and found 97% of those files were never fetched by anything. It is a file that makes a vendor look busy. Sprinkle FAQ schema on pages that have no substance under the markup. Schema describes content. It does not create it. Commission twelve “AI-optimized” blog posts while the practice pages still render client-side. That is the most expensive way to stay invisible. There is a correct sequence for the whole program, local foundation first and AI eligibility after, and it is set out in the 2026 law firm SEO guide. ## Where to start Run variant A on your single most valuable practice page. Five times. It takes fifteen minutes and it will tell you more than the last two quarterly decks. Nobody can promise that a model will name your firm. You can only be retrievable or not. That part is engineering you own. Keep the log either way, because the comparison you run next quarter is the only thing that makes any of these numbers mean something. If what you find does not match what you have been told, hi@carlosarias.com reaches us. Say which page you tested and what came back. Tags [#Law Firm SEO](/tags/law-firm-seo/)[#AI Search](/tags/ai-search/)[#Agentic AI](/tags/agentic-ai/)[#Law Firm Marketing](/tags/law-firm-marketing/) #Technical SEO Share Written by [Carlos Arias](/blog/authors/carlos-arias) Marketing Engineer for law firms. I combine digital marketing, software, data, automation and AI to improve the whole system — from first click to signed case. On this page - Retrieval is not citation, and the distinction is the whole article - Test one: the snippet test that settles law firm AI search indexing - Test two: what the crawler sees with JavaScript switched off - Test three: a prompt panel built on matter types, not keywords - Test four: which bots you are actually admitting - What a non-retrieval result actually tells you - The fixes that do nothing - Where to start ## Continue reading [Law Firm Marketing](/blog/categories/law-firm-marketing) · September 20, 2026 [### Measuring AI Search Visibility for Law Firms: Your Report Still Leads With Rankings](/blog/law-firm-marketing/ai-search-measurement-law-firm-marketing/) Measuring AI search visibility for law firms starts where the ranking report stops. What to track, what to drop, and the one number nobody can attribute. Carlos Arias · 7 min [Law Firm Marketing](/blog/categories/law-firm-marketing) · September 11, 2026 [### SEO for Law Firms in 2026: Local First, Then Practice Pages, Then AI Citations](/blog/law-firm-marketing/seo-for-law-firms-2026/) Law firm SEO in 2026 starts with the map pack, then practice-area pages that convert, then eligibility for AI citations. Timelines, intake, and what agencies get wrong — with no ranking guarantees. Carlos Arias · 14 min [Law Firm Marketing](/blog/categories/law-firm-marketing) · September 6, 2026 [### Rapid-Response SEO for Law Firms: Miami Airport Crash](/blog/law-firm-marketing/did-your-marketing-agency-get-you-in-front/) What rapid-response SEO for law firms is, how I wire it, and the limits no vendor can engineer away. Miami's runway overrun is the example. Carlos Arias · 8 min ## Stay in the loop. One email when it’s worth it — new posts and updates, no spam. Thanks — check your inbox to confirm. Free. Unsubscribe in one click.