What to Check Before You Hire an SEO or AI Optimization Agency: Seven Tests You Can Run in Fifteen Minutes
An agency that sells search visibility has a website of its own. Before you sign, run the tests below on it. Every one is something you can do yourself, in a browser, with no account, and every one has a clear pass or fail. If an agency fails the test for the thing it sells you, you have learned the most important fact about them before the first invoice.
Fifteen minutes. The order matters: the first test is the one that ends the conversation if it fails.
Why you test the agency and not the pitch
Every agency pitch is built on the same three claims: they understand what search engines and AI engines read, they do the technical work properly, and they can show you results. None of those claims can be checked from the pitch. All three can be checked from the agency’s own website, because that site is the one property where the agency chose every setting, wrote every line, and had no client to blame.
What follows came out of a real case. In September 2026 we audited a marketing agency in a large American city that sells a service it calls AI optimization. Its sales page says it makes it “trivial for crawlers and LLM indexers” to reach content, and describes “proprietary frameworks for monitoring AI visibility.” We are not naming the agency, because the point is not that one firm; the point is that every finding below was on the public internet, findable in minutes, and nobody had looked. We include the measurements with each test so you can see what a failure looks like in practice.
Test one: fetch their site as an AI crawler
What it checks: whether the crawlers that decide AI citation can read the agency’s own site. For OpenAI, Perplexity and Anthropic, the crawler that decides whether a business can be cited in an answer is a different crawler from the one that collects training data, and both can be refused by a firewall rule that never appears in robots.txt. Cloudflare, which sits in front of a large share of small-business websites, has blocked AI crawlers by default for new domains since 1 July 2025, according to its own announcement.
Why it matters: an agency selling AI visibility while its own site refuses the AI crawlers has either never run this check or does not know what the setting does. Neither is what you are paying for.
Put the agency’s address into the free AI Crawler Check. It requests the site once as each named crawler and shows the HTTP status each one gets, next to the robots.txt rule that applies. A 200 for a browser and a 403 for the crawlers is the failure.
In the case we audited: from a datacenter address, which is where AI crawlers actually run, every crawler we tried received a 403 while an ordinary browser received a 200. The site’s robots.txt allowed everything. The block was at the edge, invisible in the file, on the site of a company selling crawler access.
Test two: read their structured data
What it checks: whether the machine-readable facts an agency puts in its pages can be read by a machine. Structured data is written in a strict format called JSON-LD. One comment left inside the block, one trailing comma, one pair of single quotes, and every parser stops at that point and reads nothing. The page looks perfect in a browser. The block is dead.
Why it matters: structured data is the single most common deliverable on an AI optimization invoice. If an agency cannot ship it correctly on its own homepage, it is not shipping it correctly on yours.
Put the agency’s homepage into the free Structured Data Check. It reports every block on the page as readable or not, says why when it is not, lists what each block declares, and flags a FAQ question that exists in the code but not on the page. For the separate question of whether a page qualifies for rich results in Google, use Google’s Rich Results Test, which is the authority on that.
In the case we audited: the agency’s homepage carried two structured-data blocks. Both were unreadable. Someone had labelled the sections inside them with comments, which JSON does not allow, so neither block existed as far as any engine was concerned. The agency’s own service page, one click away, carried a large and perfectly valid block. The fault was not knowledge. It was that nobody had ever checked the homepage.
Test three: search their own service in their own city
What it checks: whether the agency ranks for the thing it sells, where it sells it. Not for its own name, which any business with a website ranks for. For the search a new customer would type: the service and the city.
Why it matters: an agency cannot have been holding back its best work for clients. If it could rank a business for “AI optimization agency” in its own city, the first business it would rank is itself.
Open a private browser window and search “[their service] [their city]”, then “best [their service] [their city]”, then “[their service] near [their city]”. Note whether the agency appears on page one for any of them, and who does. Then search the agency by name, which it should win. If it wins only its own name, that is the result.
In the case we audited: we ran five customer-shaped searches for the agency’s service and city. It was not on page one for any of them. Page one belonged to a national directory, two out-of-state agencies and a local advertising firm. For its own name it was first, as expected.
Test four: read the dates in their sitemap
What it checks: whether the site is being worked on. A sitemap lists every page a site wants search engines to know about, and each entry can carry the date the page last changed. Read across a whole site, those dates are a work log. Read on an agency’s own site, they are a work log for the people who will be sending you a monthly invoice for work.
Why it matters: there is one pattern to know. Some systems stamp every page with the same date every time the sitemap is rebuilt, so the whole site appears to have changed on one day. That is a build stamp, not a record, and it means the log tells you nothing. An agency whose sitemap is a stamp cannot show you its own work log, and you should ask how it plans to show you yours.
Put the agency’s address into the free Last Updated Check. It reads the sitemap and lays out the dates month by month, and it says plainly when the dates are a stamp. Nine empty months on a site that sells monthly work is a fair question to ask on the first call.
In the case we audited: 106 pages in the sitemap, and 99 percent of them carried the same date. A stamp. The nine months before it were empty. The blog posts carried no dates at all.
Test five: open their llms.txt
What it checks: whether a file the agency probably sells says anything. llms.txt is a plain text file at the root of a site, written for language models. It is optional, and it is worth being honest about its weight: Google has said publicly that it does not use the file, and a large-scale study of the files that exist found most are never requested by anything. So absence is not a fault. Presence is only worth something if the file has content.
Why it matters: a file that exists and is empty is a claim of readiness with nothing behind it. It is the kind of thing that gets added to a checklist, ticked, and never opened.
Open theiragency.com/llms.txt in a browser. Absent is fine. A page of text describing the business is fine. A blank screen at a working address is the failure. The Structured Data Check above reports this too, and it tells the empty case apart from the case where the address returns a web page instead of a text file.
In the case we audited: the file existed, was listed in the sitemap, and was zero bytes long.
Test six: run their homepage through a mobile speed test
What it checks: whether the agency’s own site meets the performance bar it will put in your audit. Every agency audit includes page speed, because Google’s own PageSpeed Insights is free and produces a number.
Why it matters: speed is the one deliverable with a published, public, third-party measurement. There is no room to argue about it, which is why an agency that fails it on its own site has no answer.
Put the agency’s homepage into PageSpeed Insights and read the mobile Largest Contentful Paint figure. Google treats 2.5 seconds as good. Run it twice, because a single simulated run can be an outlier. Two runs both over ten seconds are not an outlier.
In the case we audited: 24.9 seconds on mobile, twice. The homepage HTML alone weighed 166 kilobytes before a single image loaded.
Test seven: find the number on their sales page and ask for its source
What it checks: whether the agency’s claims are claims or evidence. Sales pages for AI visibility are full of percentages. Some are real and cited. Many are animated counters with no source. Ask.
Why it matters: an agency that cannot source its own numbers will report your results the same way. Then ask the second question, which is the one that matters more: ask to see the dashboard. An agency that says it monitors AI visibility should be able to open its own monitoring, on a screen, in the meeting, and show you what it looks like for an existing client. If the answer is a PDF sent later, you have learned what your reporting will be.
“Where does that figure come from?” and “Can you show me the dashboard you would give me, live, for a client you already have?” Two questions. The pause before the answer is the data.
In the case we audited: the service page named a percentage with no source, the sources it did link were a schema tooling vendor’s explainer pages, and the “proprietary frameworks for monitoring AI visibility” were described but not shown anywhere on the site.
What the agency had going for it
This is not a hit piece, and the audit did not read like one. The agency’s Google Business Profile was excellent: hundreds of reviews at a high rating, the right category, an address and phone that matched the website exactly. On the parts of a business a client can see in a meeting, it was strong. On the parts that live on the wire, where the engines look, it was failing the tests for the service it sells. That gap is the whole reason to run these tests before you sign rather than after.
What to do with the results
- A crawler block on the agency’s own site ends the conversation if the agency sells AI visibility. There is no version of that finding that is fine.
- Unreadable structured data and an empty llms.txt are questions, not verdicts. Ask how the agency checks its deliverables. The answer you want names a test, not a person.
- Not ranking for its own service in its own city is the finding to put in writing. Ask what the agency would do for itself, and why it has not.
- Ask for the dashboard before you ask for the price. Evidence you can open any time is the difference between an agency you can verify and one you have to trust.
Run the same seven tests on your own site while you are at it. If you fail test one, fix that before you hire anyone, because nothing else an agency does will matter until the engines can read you. The technical guide covers how.
If you already pay an SEO or AI optimization agency and want the same tests run on your own site, with the technical items, your Google profile and where you actually rank, that is the SEO Audit. If you want to know whether the AI engines name your business at all when a customer asks, that is the AEOCheck report.
Questions, answered
How do I check if an SEO agency is legitimate?
Test its own website against the things it sells. Fetch it as an AI crawler, read its structured data, search its service in its city, read its sitemap dates, open its llms.txt, run it through a mobile speed test, and ask for the source of any number on its sales page. A legitimate agency passes the tests for the service it sells you. The tests take about fifteen minutes and need no account.
What is a red flag when hiring an AI optimization agency?
The largest single one: the agency’s own website refuses the AI crawlers. An agency that sells AI visibility while blocking the crawlers that decide AI citation either has not run the check on itself or does not understand what the setting does. Close behind it: structured data that cannot be parsed, and a monitoring dashboard the agency describes but will not show you live.
Should an SEO agency rank for its own keywords?
For its own service in its own city, yes, or it should have a good answer for why not. It will always rank for its own name, which proves nothing. The search that matters is the one a new customer types: the service and the city. An agency that cannot rank itself for that has not shown it can rank you for yours.
Does an empty llms.txt file hurt a website?
Not directly. Google has said it does not use the file, so an empty one costs nothing in rankings. What it tells you is about process: the file was added to a checklist, ticked, and never opened. That is worth knowing about the people who will be maintaining your site.
How can I see whether an agency actually does the work I pay for?
Two ways. Read your own sitemap dates month by month, which shows what changed on the site and when, and ask for a dashboard that logs work against the items in your contract and that you can open any time without requesting a report. An agency that cannot offer the second is asking you to take the first on trust.
Written by Tyson McKay, founder of AEOCheck. Every measurement in the case above was taken from the public internet on 14 September 2026 using the free tools linked in each test; we publish how we check things, including our own claims. Last verified 14 September 2026.