Every technical SEO audit template has a section for crawlability. It checks robots.txt, it checks status codes, it checks whether the important pages return 200 to a GET request from a verified Googlebot IP. That section has been the same for a decade because for a decade it was sufficient.
It is no longer sufficient, and the reason is a throwaway detail Google published in August: its crawlers do not only speak GET and POST. They also send HEAD, OPTIONS, PUT, PATCH and DELETE. The volume is small, under 1.5% of everything Google's crawlers send. The consequence is not small at all, because of where those requests come from and what almost every production edge does when it sees one.
This is a piece about a rounding error that decides what version of your page gets indexed.
The number nobody expected to matter
The disclosure came from Gary Illyes, posting on LinkedIn and picked up by Search Engine Roundtable on August 7. Two facts, stated plainly. Google's crawlers send methods beyond GET and POST. Those methods stay under 1.5% of total crawler request volume.
The second fact is what made most people scroll past. One and a half percent sounds like noise, and in aggregate it is. But aggregate is the wrong unit here. That 1.5% is not spread evenly across your site like a tax. It concentrates on exactly the pages that ask a browser to do something, which in a modern application means the pages that carry your product, your pricing, your search, your configurator and your booking flow.
The explanation Illyes gave for the traffic matters more than the number. His phrasing was that JavaScript on the page is initiating these requests, and rendering does its rendering things. Read that carefully. Google is not probing your server with a DELETE to see what happens. Google is executing your code, and your code is making the call.
“These are not Google's requests. They are your requests, made from Google's renderer, and your edge is rejecting them as though a stranger sent them.”
That inversion is the whole story. A preflight OPTIONS call before a cross-origin fetch. A HEAD request from a library checking whether an asset exists before loading it. A PATCH from an analytics or personalization SDK writing state. A PUT to a session endpoint. None of that is exotic. It is the ordinary behavior of a front end built in the last five years, and it runs when Googlebot renders the page just as it runs in a user's browser.
Why Googlebot HTTP methods reach your origin at all
Rendering is a two step process and the second step is where this lives. Google fetches your HTML with a GET. Later, sometimes seconds later and sometimes considerably longer, the page goes into the rendering queue, a headless browser loads it, and the JavaScript executes. Every network call your application makes during that execution is a real request that leaves Google's infrastructure and arrives at your origin.
Practitioners internalized this for GET requests years ago. The client side data fetch that populates a product grid is understood as a crawl dependency, and if it returns a 403 the grid is empty in the index. The same logic applies without modification to every other method your application uses, and hardly anyone has extended the audit to cover them.
| METHOD | WHAT TYPICALLY TRIGGERS IT IN A FRONT END | WHAT A DEFAULT EDGE POLICY DOES |
|---|---|---|
| OPTIONS | CORS preflight before any cross-origin request carrying custom headers or a non-simple content type | Frequently rejected or challenged, which kills the request that follows it |
| HEAD | Asset existence checks, media probing, download size checks before a fetch | Often allowed, sometimes stripped of headers the client needs |
| PUT | Session or preference writes, draft saves, cart state in headless commerce | Blocked by default in most managed WAF rule sets |
| PATCH | Partial state updates from personalization, consent and analytics SDKs | Blocked by default, often with no logging at the origin |
| DELETE | Clearing a session, removing a cart item, tearing down a temporary resource | Blocked, and frequently rate limited or flagged as an attack signature |
Look at the middle column and notice how much of it is infrastructure you did not write. Consent management platforms, tag managers, A/B testing tools, session replay, chat widgets and personalization engines all make calls of this shape. The team that installed them was not thinking about crawl behavior, and the team that wrote the firewall policy was not thinking about the tag manager.
This is the same structural gap we described in the search dependency framework: the things that break indexing are increasingly owned by nobody, because they sit between two teams who each assume the other one checked.
What your edge does with a method it does not recognize
Open your CDN or WAF configuration and find the allowed methods list. In a managed rule set it is usually GET, HEAD and POST, with everything else returning 405 Method Not Allowed. In an API gateway it is per route and typically strict by design. In a hardened enterprise config, PUT, PATCH and DELETE are often not just rejected but treated as hostile, because for two decades an unexpected DELETE against a web server was a reasonable thing to treat as hostile.
That default was correct when the only thing sending a DELETE to your marketing site was somebody testing an exploit. It is no longer correct now that the request originates from your own application code, executed by the renderer that decides what your page contains.
Non-GET, non-POST requests per day at various crawl volumes, using Google's stated 1.5% ceiling (illustrative arithmetic)
The bars are arithmetic, not measurement, and the real figure for any given site sits somewhere under that ceiling. The point is the shape. At enterprise crawl volume, a category of request you have never audited is arriving in the thousands per day and being rejected by a rule written before your current front end existed.
There is a second layer to this that most teams hit first. Bot management and AI crawler controls at the edge classify traffic by identity, and Googlebot has been caught in those nets repeatedly this year, most visibly when Cloudflare's defaults started returning errors to Googlebot on sitemap requests. Method filtering is a different mechanism with the same outcome, and a site can be hit by both at once while its owner is looking at neither.
The failure is silent in every tool you use to check
Here is why this has stayed invisible. Every diagnostic in the standard kit fetches a URL with GET and reports what came back. That is true of the URL Inspection tool, of the rich results test, of your crawler of choice, and of the curl command you run when something looks wrong.
The result is a class of indexing defect that survives every routine check and gets attributed to something else. We have watched teams spend a quarter on content revisions for pages whose actual problem was one rejected preflight call, which is the same pattern of misdiagnosis that made the bot blocking teardown worth writing up in the first place.
Auditing Googlebot HTTP methods without a week of work
This does not require a project. It requires an afternoon and access to two systems.
Start in the browser, not the crawler. Load your five highest value templates with the network panel open, filter to requests where the method is not GET, and write down what you find. On most sites the list is short and boring: one or two OPTIONS preflights, occasionally a PATCH from a consent or analytics tool. On a headless commerce or heavily personalized template it will be longer. That list is your test set, and until you have it you are guessing.
Then reproduce each one against production the way the renderer does, from outside your network, without a browser session, with no referer and no cookies. Send it and record the status. Anything that is not the status the browser received is a live defect. Confirm the same thing from a verified Googlebot address range if your bot rules discriminate by identity, using forward confirmed reverse DNS rather than a user agent string, which anyone can set.
Then pull edge logs rather than origin logs, filter to verified Google crawler traffic, and group by method and status. If you retain enough history you will see immediately whether the 405s are real and where they concentrate. If you do not retain it, start, because this is not the last question you will want to ask of that data. This kind of instrumentation is the unglamorous half of every technical audit we run, and it finds more revenue than the interesting half does.
One caution on remediation. The fix is to allow the specific methods your application actually uses on the specific routes that use them, verified against the list you built in step one. It is not to open every method site wide. A blanket allow turns a rendering problem into a security problem, and the people who wrote the restrictive default were not wrong about why it existed.
What to change this week
Three things, in order, none of which need a planning cycle.
Get the method inventory for your top templates. Ten minutes each in the network panel, and it converts an abstract risk into a concrete list you can hand to whoever owns the edge configuration. Most teams discover their marketing site makes exactly two non-GET calls and both are preflights, at which point the whole question resolves in one rule change.
Ask your security or platform team a specific question rather than a general one. Not is Googlebot allowed, which will always come back yes. Ask which HTTP methods are permitted on the production hostname, per route, and what the response is for the rest. The answer is a configuration file, and it either matches your inventory or it does not.
And treat rendering dependencies as a standing inventory rather than a one time check. Every tag added by a marketing team, every SDK added by a growth team and every new component added by an engineering team can introduce a network call that the crawler will make and your edge may refuse. For SaaS products where the pricing and product pages carry the most client side machinery, that inventory belongs in the release checklist next to the performance budget.
The 1.5% is not the interesting number. The interesting number is what share of your indexable pages depend on a request your own infrastructure is configured to reject, and until somebody checks, the honest answer is that nobody at your company knows.
See where you are cited today
A free snapshot audit of your rankings and AI citations before we ever talk.
Tyler leads work at the intersection of SEO and generative engines at Something Inc., helping B2B brands get ranked and cited across every major AI engine.