Something Inc.LoginSchedule a free consultation
TECHNICAL SEO

The 2MB cutoff your SEO tool can't see

Google documents a hard 2MB fetch limit on every page it crawls, and quietly revised that same documentation again on October 29. The Search Console tool built to check your page runs a different crawler with a different ceiling, and will tell you everything is fine when it isn't.

JBJosh BernsteinManaging Partner · NOV 22, 2026 · 10 MIN READ
TL;DR · 60 SECONDSGoogle's documentation says Googlebot fetches at most 2MB of any page, HTTP headers included, and quietly updated that same page on October 29, 2026. Spotibo published a test in February showing a 3MB page gets truncated mid-word during indexing with zero warning in Search Console. But Search Console's own URL Inspection and Test Live URL tools run a different crawler on a different ceiling, so they'll show you the complete, untruncated page even when the index copy was cut off. The gap is rare, under 1% of pages by one count, but it concentrates on exactly the large template and category pages enterprise sites run.

On October 29, Google quietly edited a blog post it published seven months earlier, clarifying that its 2MB crawl limit counts HTTP response headers, not request headers. It's a small correction. It's also a reminder that this limit is real, enforced, and still not well understood by the people whose job it is to catch it before it costs them traffic.

The limit is 2MB. Past it, Googlebot doesn't reject your page, it doesn't flag it, and it doesn't tell you. It just stops reading. The bigger problem is that the main tool SEOs use to check their own pages for exactly this issue was built on a different crawler, with a different ceiling, and it will hand you a clean bill of health on a page that's already being cut off in the index.

What Google's crawl limit actually says

Googlebot fetches at most 2MB of any individual URL, PDFs excluded, according to Google's own crawling explainer, published March 31, 2026 under Gary Illyes' byline. That 2MB figure covers the entire raw HTML response, including the HTTP headers that precede it. Google revised that same post on October 29, 2026, to clarify a detail that matters for anyone measuring byte counts by hand: the 2MB counts HTTP response headers, the ones the server sends back, not the request headers the browser or crawler sends out. Any crawler that doesn't declare its own limit defaults to 15MB, which is a separate, unrelated ceiling for unnamed bots, not a downgrade from some prior Googlebot number.

“Googlebot doesn't reject the page. It stops the fetch exactly at the 2MB cutoff, and any bytes after that threshold simply don't exist to Googlebot.”

Two details in the official post matter more than the headline number. First, the truncated fragment isn't discarded, it's passed to indexing and rendering as if it were the whole file, so a page that gets cut off mid-sentence still gets indexed, just wrong. Second, subresources get their own separate counters: a linked CSS or JavaScript file doesn't eat into the parent page's 2MB budget, but each of those files is capped at 2MB in its own right. A page built from clean, externalized assets is largely protected by that separation. A page with everything crammed inline is not.

The test that caught Google doing it

Spotibo ran the test most SEOs only argue about. On February 9, 2026, developer Janko Januska published a set of files to Google and tracked exactly what happened to each one, with the test pages left live for anyone to check. The point of the exercise wasn't to confirm the number Google had already put in writing. It was to see what actually happens to the file once it crosses that line, because a documented limit and an observed, reproducible failure mode are two different kinds of evidence, and only one of them tells you what your own pages are at risk of.

TEST FILEWHAT HAPPENEDSEARCH CONSOLE'S RESPONSE
3MB HTML pageIndexed copy cut off at roughly 2MB, around line 15,210, mid-wordReported "Page is indexed," no warning
16MB HTML pageGoogle never processed itIndexing request returned a generic error, crawl data showed N/A
2.5MB imageIndexed normally, appeared in Image Search within two daysNo issue, images aren't bound by the HTML limit
CSS/JS files over 15MBSuspected silent truncation, unconfirmedNo errors reported either way

The 3MB test page is the one that matters. Google indexed it, Search Console called it healthy, and the version actually sitting in Google's index stops roughly two-thirds of the way through the file, mid-word, with no flag anywhere in the interface telling the site owner that happened.

The tool everyone uses to check this says everything is fine

DebugBear founder Matt Zeunert ran a parallel investigation a week later and reached the opposite result using Search Console's own tools. Across five separate checks, Test Live URL on an oversized HTML document, Test Live URL on a page with a large external JavaScript file, a dedicated test page submitted through URL Inspection, and two more built from 2MB-plus DOM content generated by JavaScript, every single one came back showing the complete, untruncated page.

Read those two results side by side and they look like a contradiction. One published test proves truncation happens, with a line number and a dead link to the cut page. Another, equally careful test, run through Google's own verification tooling, finds nothing. Both are accurate. They're measuring two different crawlers.

Why the two tests disagree

Search Console's URL Inspection and Test Live URL features run on a separate crawler Google calls Google-InspectionTool, and both Spotibo and the German technical SEO site seo-kreativ independently report that it operates on a materially higher fetch ceiling than the indexing-time Googlebot fetch the 2MB limit actually governs. That's why Zeunert's tests kept coming back clean: he was running the right test on the wrong crawler.

The practical consequence is that the verification layer and the production layer disagree, and the verification layer wins the argument by default because it's the one with the interface. Search Console tells you a page is indexed. It does not tell you whether the copy it indexed is complete. Those are two different claims, and right now only one of them is checkable from inside Google's own tools.

2MB
Googlebot's documented per-URL HTML fetch limit, headers included
0.82%
of 44.5M pages analyzed by Seobility that exceed 2MB
33KB
median mobile HTML size, HTTP Archive Web Almanac 2025
151KB
90th percentile HTML size, per John Mueller's own read of that data

Who this actually hits

Almost nobody runs into this limit by accident, and that's exactly why it's dangerous. Seobility's pass across 44.5 million pages found only 0.82% exceeding 2MB, and separate HTTP Archive data puts the median mobile page at roughly 33KB with 90% of pages under 151KB. Most sites will never see this number in a report.

The pages that do cross it share a profile: large category and listing pages with hundreds of inline product cards, templates carrying megabytes of base64-encoded images instead of linked files, enterprise navigation markup repeated in full on every page, and JSON-LD blocks dumped inline at full size instead of trimmed or deferred. That's not a random sample of the web. That's the enterprise e-commerce and publisher footprint this site's own audits spend most of their time inside, which makes a quiet, unverifiable truncation bug considerably more expensive than a 0.82% prevalence rate suggests.

There's a second-order effect worth naming directly. A page built from heavy inline markup usually crossed 2MB gradually, one added widget or one new product-card variant at a time, with no single commit that looks dangerous on review. Nobody ships a page that's deliberately 3MB. It accumulates, which means the fix is rarely one bad deploy to revert. It's a budget to defend going forward, the same way page weight and Core Web Vitals became release-gate metrics once teams stopped treating them as someone else's problem.

How to check your own site

Checking for this requires measuring the actual delivered document instead of asking Search Console whether it's happy, because Search Console's happiness is not the question that matters here.

1Measure the raw response, not the rendered DOMPull the page with curl or view-source and check the byte count of the HTML document itself, headers included. DevTools' network tab showing final rendered size tells you nothing about what Googlebot actually fetched.
2Audit your largest templates firstCategory pages, search results pages, and any template that concatenates many items into one document are the highest-risk shape. Check your three largest templates by rendered HTML size before touching anything else.
3Externalize anything inline and heavyBase64 images, large inline CSS or JS blocks, and full-size inline JSON-LD all eat the same 2MB budget as your actual content. Moving them to linked files gives each one its own separate counter instead.
4Don't trust "Page is indexed" as proof of completenessThat status confirms Google has a copy. It says nothing about whether the copy is whole. Spotibo's 3MB test page carried that exact status while missing a third of its content.
5Treat Search Console's live tests as directional, not definitiveURL Inspection and Test Live URL run on Google-InspectionTool, not the indexing-time Googlebot fetch. A clean result there is reassuring. It is not confirmation that your indexed copy matches what you shipped.

Do this next

Pull the three largest templates on your site, check their actual delivered HTML size with curl, and stop relying on Search Console's green checkmarks to tell you the whole story. If you're running technical seo services for a site with large catalog or listing templates, this is a five-minute check that catches a bug none of the standard tooling will ever surface on its own, which is exactly the gap a proper audit earns its fee by closing. We've written before about how Google shares crawl capacity across every crawler hitting your site, about the rendering gap between Googlebot and the AI crawlers that don't execute JavaScript, and about Google's own published ranges for how long crawling and indexing actually take. This bug sits underneath all three: it's not a ranking problem, it's a measurement problem, and it stays invisible for exactly as long as you keep trusting the tool that can't see it.

What is Google's 2MB crawl limit?Googlebot fetches at most 2MB of any URL's HTML, HTTP headers included. Bytes past that point are never fetched, rendered, or indexed, per Google's March 2026 "Inside Googlebot" post.
Does Search Console warn me if my page gets truncated?No. Spotibo's published test showed a 3MB page truncated at 2MB during indexing while Search Console reported it as fully indexed, with no flag anywhere in the interface.
Why didn't DebugBear's tests find any truncation?DebugBear tested through Search Console's Test Live URL and URL Inspection tools, which run on a separate crawler (Google-InspectionTool) with a different, higher fetch ceiling than the indexing-time Googlebot fetch.
Does the 2MB limit apply to CSS and JavaScript files too?Yes, each external resource gets its own separate 2MB counter and doesn't count against the parent page's budget. A 3MB inline script block does count against the page itself.
Are PDFs affected by the 2MB limit?No. Google's documentation sets a separate 64MB limit for PDFs, well above the 2MB figure that applies to HTML.
How many sites actually exceed 2MB of HTML?Seobility's analysis of 44.5 million pages found just 0.82% over 2MB, and HTTP Archive data puts the median mobile page at around 33KB. It's rare, but concentrated on large template-driven sites.
What kind of pages are most at risk?Large category and listing pages, templates with hundreds of inline items, base64-encoded images instead of linked files, and oversized inline JSON-LD or navigation markup.
What's the fastest way to check my own pages?Pull the page with curl or view-source and measure the raw HTML byte count, headers included. Don't rely on DevTools' rendered size or a clean Search Console status as proof.

See where you are cited today

A free snapshot audit of your rankings and AI citations before we ever talk.

JB
Josh BernsteinMANAGING PARTNER, SOMETHING INC.

Josh leads work at the intersection of SEO and generative engines at Something Inc., helping B2B brands get ranked and cited across every major AI engine.

Keep reading

All insights →
TECHNICAL SEOTechnical SEO Services: Scoping For Two CrawlersA technical SEO engagement in 2026 has to serve two crawler populations with opposite capabilities. Googlebot executes JavaScript, runs 4.5 billion fetches a month and reports its findings back to you in Search Console. The AI crawlers execute nothing, work from stale URL inventories, and report nothing at all. The most-cited measurement of that second group is a Vercel and MERJ study from December 2024, which found that ChatGPT burned 34.82 percent of its fetches on 404 pages against Googlebot's 8.22 percent. Nobody has published a comparable re-test since. That is the honest state of the evidence, and it decides what technical SEO services can promise, what they must refuse to promise, and which half of the scope you should be paying a premium for. Here is the scope that holds up, the comparison table buyers should ask for, and the parts your own engineers should keep.JOSH BERNSTEIN · 12 MINTECHNICAL SEOSEO Audit Services: What To Buy In 2026Five credible studies answered the same question about AI citations and rankings and came back with 12%, 17%, 38%, 76% and 84%. That spread is the most useful finding in search right now, and it decides what a 2026 SEO audit has to measure, what it has to stop pretending to know, and what the remediation actually costs.TYLER TRUFFI · 13 MINTECHNICAL SEOThe AI Training Opt Out Is Not The One You WantOn October 30, 2026, Google said it plans to honour a shared AI training opt out in robots.txt from 2027, and Microsoft published a full replacement for NoArchive and NoCache that splits AI Training from AI Grounding. The two announcements are being read as one thing: a line you add to robots.txt to opt out of AI. That reading is wrong in both directions. The shared token covers model training, which was never what cost publishers traffic. The grounding preference, which decides whether your pages feed an AI answer, cannot be set in robots.txt at all on Bing, and Google's generative search opt out remains a separate control in Search Console. Bing is blunt about the price of the grounding lever: pulling it will significantly reduce brand visibility in AI experiences. Here is what each of the four controls actually does, the deadlines that now sit in the calendar, and the setting almost every commercial site should land on.JOSH BERNSTEIN · 11 MIN
Free consultation

Let us be the last SEO agency you ever work with

A 30 minute call and a free audit of your SEO and GEO position. You keep the findings either way.