How to Evaluate an AEO Agency: Four Questions

AEO is new enough that the market is full of repositioned SEO offers, and the buyer's problem is telling apart the practitioners who understand answer engines from the ones who renamed their existing service. The good news: the distinction is detectable in a single conversation, because real AEO knowledge shows up as specific claims about mechanism, and its absence shows up as vagueness dressed in new vocabulary. This is a buyer's guide to running that test.

The evaluation reduces to one question asked four ways: does this agency understand that AEO is a contest for selection, not position? Everything below is a way to surface the answer.

How do you measure success?

This is the fastest tell. An agency that answers in rankings and keyword positions is measuring the wrong thing, because AI citations are probabilistic and do not have stable positions. A real practitioner talks about share of model rather than rank, about how often you are cited across repeated queries, and about reading citation patterns over time to distinguish a stable win from a volatile or decaying one. If the measurement story is a rank-tracking dashboard with "AI" in the title, the underlying method is unchanged.

What do you diagnose before writing anything?

A weak agency proposes content first, because content is what they sell. A strong one diagnoses first, because most AEO failures are not content problems. The single most common catastrophic issue is that engines cannot read the site at all: JavaScript-rendered content arriving as an empty shell. If an agency has not asked how your site renders, whether your schema parses, and whether engines currently cite you before pitching an article package, they are selling output without diagnosis, and the content will underperform because the substrate is broken. Ask what their audit checks. A real one covers readability, structure, entity, and current citation standing, not just "content gaps."

How do you handle entity and authority?

Citations depend on the engine trusting who you are, which is entity work, not content work. A practitioner who understands AEO talks about your entity home, consistency across sources, and how engines describe your brand. An agency that only discusses on-page content is missing the layer that determines whether the content gets trusted at all. Entity fluency is a strong positive signal, because it is the part of AEO that repackaged SEO does not include.

What do you refuse to promise?

This question is the most revealing, because the honest answer sounds worse than the dishonest one. A practitioner who understands AEO will refuse to guarantee a specific citation count or "AI ranking," because the systems are probabilistic and no one controls them. An agency that guarantees AI visibility outcomes is either misunderstanding the systems or misrepresenting them, and both are disqualifying. The willingness to say "I cannot promise you the number, I can promise you the work that moves the odds" is a trust signal precisely because it costs the agency the easy sell. Be suspicious of confidence that exceeds what the technology allows.

Two more practical checks

Ask them to audit a page live. A real practitioner can look at a URL and name specific, mechanism-level problems (this renders client-side, this has no entity anchor, this buries the answer). Someone reselling SEO will produce generic observations. And ask what they would not do, a strong agency will tell you which fixes are outside content (server-side rendering, developer work) rather than pretending content solves everything, which matches how the real signal set actually distributes across technical, structural, and entity layers.

Here is what the difference sounds like in practice. You hand both agencies the same product page. The repackaged-SEO agency says: "the content is thin, the keyword density is low, and you need more backlinks, we recommend a content package." The practitioner says: "this page renders its spec table client-side, so engines see an empty div where your differentiators live; the answer to the page's own target question first appears at word four hundred; and your brand has no entity home, so nothing corroborates who is making these claims. Fix rendering first, nothing else pays until then." One of these is a diagnosis. The other is a quote wearing one.

The one-line filter

If you remember one thing: real AEO expertise is specific about mechanism, and repackaged SEO is vague in new vocabulary. The practitioner who can explain exactly why a page does or does not get cited, and who refuses to promise what the systems will not let anyone promise, is worth more than the agency with the confident dashboard and the guarantee. The contrarian truth for buyers is that the agency willing to tell you what it cannot do is usually the one that understands the field, because honesty about the limits is itself the clearest evidence of expertise. The confident guarantee is the tell that they have not understood what they are selling.

Sources

• Google, AI features and your website: the official position that no special trick unlocks AI visibility, only quality and readability. developers.google.com
• Princeton, GEO: Generative Engine Optimization: the research grounding the measurement of generative-engine visibility. arxiv.org/abs/2311.09735
• Website AI Score, share of model vs rank tracking: the measurement question that exposes repackaged SEO. View article
• Website AI Score, AEO scoring signals: the full signal set a real audit covers. View article
• Website AI Score, empty-shell rendering audit: the diagnosis a real agency runs before proposing content. View article