Generative Engine Optimization (GEO)

Overview

AI answer engines read the web, write a short answer and show a few source links. GEO is the work of making a site eligible for that retrieval and easy to quote. It is not a separate algorithm to game. The vendors' own documentation says so, and this skill is built on what they publish.

What the primary sources actually establish, as of late 2026:

• Google says AI Overviews and AI Mode have no extra technical requirements: a page must be indexed and eligible to show with a snippet. Google tells site owners to ignore "chunking" and special AI text files for Google Search. Search Console has a Generative AI performance report.
• OpenAI, Anthropic and Perplexity each publish separate crawlers for search, for training and for user-triggered fetches. Allowing the search crawler while blocking the training crawler is a documented, supported choice.
• Microsoft reports citations in Copilot and Bing AI answers through the AI Performance view in Bing Webmaster Tools (public preview since February 2026).
• llms.txt is a community proposal (llmstxt.org). No engine in this list documents using it for ranking or citation.

So an audit has three layers, and the report must keep them apart:

1. Gates: facts you can verify and that decide whether an engine can see the page at all (robots.txt, HTTP status, rendered HTML, indexability).
2. Documented practice: guidance the vendors publish (helpful original content, structured data that matches visible text, clear authorship).
3. Heuristics: widely repeated advice with no vendor backing (passage length, answer-first layout, brand-mention volume). Useful, but label it as a guess.

Nobody outside the vendors can measure how often an engine cites a page, because answers vary by prompt, user, location and time. Do not produce a "GEO score out of 100"; any such number is invented weights.

Instructions

1. Scope

Ask for: the site URL, the business type, the 5-10 questions customers ask that the site should answer, which engines matter, and whether the owner wants to opt out of AI training (a separate decision from search visibility). Check what they already have: Search Console, Bing Webmaster Tools, server logs or a CDN bot report.

2. Gate 1: can the crawlers fetch it?

Fetch /robots.txt and evaluate each agent under RFC 9309 rules: the most specific User-agent group wins, and within a group the longest matching path wins. Run this stdlib script and read the output next to the file itself, because urllib.robotparser is simpler than Google's parser (check wildcards by hand):

import sys, urllib.request
from urllib.robotparser import RobotFileParser

BOTS = ["OAI-SearchBot", "GPTBot", "ChatGPT-User", "Claude-SearchBot", "ClaudeBot",
        "Claude-User", "PerplexityBot", "Perplexity-User", "Googlebot",
        "Google-Extended", "Bingbot", "Applebot", "Applebot-Extended"]

site, path = sys.argv[1].rstrip("/"), (sys.argv[2] if len(sys.argv) > 2 else "/")
req = urllib.request.Request(site + "/robots.txt", headers={"User-Agent": "geo-audit/1.0"})
rp = RobotFileParser()
rp.parse(urllib.request.urlopen(req, timeout=20).read().decode("utf-8", "replace").splitlines())
for bot in BOTS:
    print(f"{bot:18} {'allow' if rp.can_fetch(bot, site + path) else 'BLOCK'}")

Classify by what each vendor documents, not by a tier list:

Verify names against the vendor pages (listed in Guidelines) before advising a change: agent names have been added and retired.

Other things that block crawlers while robots.txt looks fine: a WAF or bot-protection challenge, geo blocks, rate limits, and login or cookie walls. Request a key page with each user-agent string using curl and compare status codes. Vendors publish IP ranges (OpenAI, Anthropic, Perplexity) for allow-listing; verify those rather than trusting the header. Anthropic warns that blocking by IP can stop its bots from reading robots.txt.

3. Gate 2: is the content in the HTML?

curl -sL -A "Mozilla/5.0 (compatible; OAI-SearchBot/1.0)" -o page.html https://northwind-bikes.example/guides/chain-wear
python3 - page.html <<'EOF'
import sys
from html.parser import HTMLParser

class Visible(HTMLParser):
    skip, words = 0, 0
    def handle_starttag(self, tag, attrs): self.skip += tag in ("script", "style", "noscript")
    def handle_endtag(self, tag): self.skip -= tag in ("script", "style", "noscript")
    def handle_data(self, data): self.words += 0 if self.skip else len(data.split())

p = Visible(); p.feed(open(sys.argv[1], encoding="utf-8", errors="replace").read())
print(p.words, "visible words in the server HTML")
EOF

Compare the word count and the presence of the main heading, answer text, prices and author with the rendered page. Googlebot renders JavaScript; there is no equivalent public commitment from the other crawlers, so treat client-only content as invisible to them. An empty <body> shell is a critical finding. Also check: HTTP 200 on the canonical URL, no stray noindex or nosnippet (a nosnippet or low max-snippet limits what Google can quote), a canonical tag that points to itself, and content not hidden behind tabs that load on click.

For Google specifically, confirm the page is indexed with URL Inspection. Eligibility for AI features equals being indexed and snippet-eligible.

4. Content: write what can be quoted

Documented guidance (Google): original, helpful content for people, clear authorship, accurate and current. Heuristics that fit how answers are assembled, to be offered as suggestions:

• Put a plain statement of the answer directly under the question heading, then the detail. Engines often issue several sub-queries per question (Google calls this query fan-out), so cover the sub-questions a buyer would ask: price, limits, alternatives, how it compares.
• Make each section understandable alone: name the subject instead of "it", give units, dates and the source of every figure.
• Prefer facts the model cannot guess: your own measurements, policies, specifications, version numbers, dated changes.
• Use real HTML structure: headings, lists, and <table> for comparisons, not images of tables.
• Show the date updated and the author, and keep both true.

There is no verified ideal passage length. Do not report one as a standard.

5. Entities and structured data

Structured data helps engines tie the page to a real thing. Google states that markup must match visible content. Check in the server HTML (not injected late by scripts, though Google can read JavaScript-generated markup):

• Organization with name, url, logo, and sameAs links to profiles you really own (LinkedIn, GitHub, Crunchbase, Wikipedia or Wikidata only if the entity genuinely has one).
• Article / BlogPosting with author (a Person with a profile URL), datePublished, dateModified.
• Product, SoftwareApplication, LocalBusiness where they fit, with the same prices and hours the page shows.
• Check the current Google structured data gallery before recommending FAQ or HowTo markup: Google has retired most of those rich results, so add them only if the content needs them, not for AI.

Validate with the Rich Results Test and schema.org's validator. Hand the JSON-LD writing to the schema-markup skill.

6. Off-site presence (correlation only)

Engines also lean on what other sites say. Look for accurate listings and reviews on the sites buyers use, third-party comparisons, documentation mentions, and community threads. Studies correlating brand mentions with AI visibility exist, but they show correlation, and sample choices differ. Recommend earning real mentions. Never fake reviews or buy mentions; Google's guide says chasing inauthentic mentions is not as helpful as it seems.

7. llms.txt (optional)

Per the proposal: a Markdown file at /llms.txt with an H1 name, a blockquote summary, and sections of links with short notes; ## Optional marks skippable links. Google says it neither helps nor harms Search. It is cheap and can serve coding agents and docs tooling that choose to read it. Offer it last, after the gates. Never present it as a ranking factor.

8. Measure what can be measured

• Google: Search Console, Generative AI performance report.
• Microsoft: Bing Webmaster Tools, AI Performance (citations, cited pages, grounding queries, a sample).
• All engines: server or CDN logs filtered by the agent names above (search crawlers show up here; referral hits from chat products appear as referrers such as chatgpt.com or perplexity.ai).
• Manual prompt sampling: run each target question several times on different days and record whether the site is cited. Report it as a sample with the date, never as a rank.

9. Report

Deliver: (a) gate results as pass/fail with evidence, (b) documented-practice findings, (c) heuristics clearly marked, (d) ordered fixes with the exact robots.txt lines or JSON-LD, (e) the measurement plan, (f) a short list of things nobody can verify.

Examples

Example 1: Hardware retailer invisible in ChatGPT

Audit northwind-bikes.example. Customers say ChatGPT never mentions us.

Findings:

1. robots.txt has User-agent: OAI-SearchBot then Disallow: /, copied from a 2023 "block AI" snippet. Gate failure: no ChatGPT search results.
2. /guides/chain-wear returns 200 but the HTML has 40 words: the guide is a client-rendered React shell. Gate failure for every crawler except Googlebot.
3. Organization JSON-LD exists with no sameAs; articles have no Person author.
4. The wear-limit answer sits in paragraph four under a heading "Introduction".

Fix list, in order:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

(keep the GPTBot block only if the owner confirmed a training opt-out), then server-render /guides/*, retitle the heading "How do you tell if a bike chain is worn?", put the 0.5% / 0.75% elongation limits first, add the Person author and sameAs links to the shop's real profiles. Measurement: Bing AI Performance and a weekly five-prompt sample. Nothing here is a promise of citation.

Example 2: Docs site asking about llms.txt

Our docs are on docs.ledgerlane.io. Should we add llms.txt and block GPTBot?

Answer in three lines. llms.txt: optional; Google says it has no effect on Search, a docs generator may already produce one, and coding agents may fetch it, so add a short one linking the quickstart, API reference and changelog. GPTBot: independent of search; blocking affects OpenAI training only, so it is the owner's policy call. Verify first: OAI-SearchBot and Claude-SearchBot are allowed and the API reference renders without JavaScript. The llms.txt:

# Ledgerlane Docs

> Ledgerlane is an invoicing API for freelancers. These docs cover authentication, invoices, webhooks and rate limits.

## Docs
- [Quickstart](https://docs.ledgerlane.io/quickstart): create a key and send a first invoice
- [Webhooks](https://docs.ledgerlane.io/webhooks): event types, retries, signature check

## Optional
- [Changelog](https://docs.ledgerlane.io/changelog): dated API changes

Guidelines

• Sources to re-check before each audit: Google Search Central "AI features and your website" and "Optimizing for generative AI"; OpenAI "Overview of OpenAI Crawlers" (platform.openai.com/docs/bots); Anthropic Help Center "Does Anthropic crawl data from the web"; Perplexity "Perplexity Crawlers" (docs.perplexity.ai/guides/bots); Apple "About Applebot"; Bing Webmaster blog on AI Performance; RFC 9309; llmstxt.org; schema.org.
• Fix gates first; they cost little and decide everything else. Content rewrites come after.
• Keep search and training decisions apart. Blocking a training crawler does not remove a site from that vendor's search, and blocking a search crawler does.
• A crawler that is allowed may still be unable to read the page behind a WAF. Test with real requests.
• Never promise citations, rankings or traffic. State the evidence level of every recommendation.
• Do not invent a Wikipedia page, reviews, author credentials or statistics. Recommend creating the real thing.
• Vendor behavior changes; this is a 2026 snapshot, and facts here were taken from the vendor pages listed above.
• Skip this skill for pure ranking problems with no AI angle; use seo-audit.