What are conditional requests?
A server can label each version of a page with an ETag (a quoted code such as "5f2a1c") and a Last-Modified date. When a crawler visits again, it sends the label back: If-None-Match with the ETag, or If-Modified-Since with the date. If the page is unchanged, the server answers 304 Not Modified without a body. If it changed, the server sends the full page with HTTP 200.
Google’s crawlers support both validators. When a page sends both, Google uses the ETag, and Google recommends the ETag for its crawlers because dates are easier to get wrong. Google has said that only a small share of its fetches can currently be served from cache, so many sites could save crawling and bandwidth here. A page without validators is not broken: crawlers simply download it in full each time.
Google treats a 304 as “the content is the same as last time it was crawled”. A server or CDN that answers 304 for an ETag the page never had, or for a date long before the last change, can keep crawlers on an outdated copy of the page.
How this tool works, step by step
Read robots.txt and the page
The checker reads robots.txt and honours it for DeepSpider, then requests the page once with DeepSpider’s own user agent, following up to 5 redirects. It records the ETag, Last-Modified, Cache-Control, Vary and any CDN cache headers. Only HTML pages that answer 200 are tested, as in a DeepSpider audit.
Send a made-up ETag
It sends If-None-Match with an ETag the page cannot have. The right answer is 200. If the answer is 304, it repeats with another made-up ETag to rule out a one-off glitch.
Send the page’s own ETag
It sends the real ETag (or the real date when there is no ETag). A 304 means conditional requests work. After a 304 it requests the full page again and compares the content, title, canonical and noindex, so a 304 for a changed page is caught.
Test the date
It sends If-Modified-Since with the page’s own Last-Modified (304 or 200 are both fine) and with 1 January 1998, which must get 200. A 304 for the old date is repeated once.
Compare with the audit
Steps 2 and 3 run the same code as the Incorrect 304 responses check in a DeepSpider audit, and the audit’s rule judges them. Step 4 and the ETag stability check are extra observations of this tool.
How to read the report
Conditional requests work
A matching ETag or date gets 304, made-up ones get 200. Nothing to change.
Nothing to compare
No ETag or Last-Modified. Not an error; add an ETag if you want crawlers to revalidate instead of downloading.
Works, but unreliable
An ETag that changes on every request, a date in the future or in the wrong format. 304s may never happen or be unreliable.
Wrong 304
A 304 for a made-up ETag, for an old date, or for a page that changed. Fix this first.
“ETag is not used for 304 responses” means the server ignores conditional headers and always sends the full page. That is allowed; it only saves nothing. A weak ETag (starting with W/) is fine: If-None-Match uses weak comparison, and servers often weaken ETags when they compress a page. CDN headers such as cf-cache-status, x-cache or age show how the CDN edge that answered DeepSpider handled the request.
How to fix caching problems
Answer 304 only when the ETag in If-None-Match equals the page’s current ETag, or when the page has not changed since the If-Modified-Since date. Otherwise send 200 with the page.
Derive the ETag from the page content or its version, so it changes whenever the page changes and stays the same when it does not. Avoid timestamps, random values or per-server values.
Many wrong 304s come from a CDN or proxy rule, not the origin. Test the origin directly, then review the CDN’s caching and revalidation settings and purge it after changes.
Last-Modified must be a real HTTP date such as “Tue, 06 Oct 2026 08:00:00 GMT” and never later than the response time.
If you add one validator, add an ETag. Google’s crawlers use it when both are present.
Run the checker again after a deploy or CDN change, and include a page that you have just edited.
Frequently asked questions
Does a 304 improve rankings?
No. Caching saves crawling and bandwidth; Google has not said it affects rankings. A wrong 304, however, can keep an outdated copy of the page in use.
Does this show what Googlebot received?
No. The checker shows how the server and CDN answer DeepSpider now, from its server in the EU. Other CDN locations can answer differently, and the check says nothing about Google’s past crawls or what is in Google’s index. Use Search Console’s URL Inspection for Google’s view.
Why does the checker send Cache-Control: no-cache?
It asks caches between DeepSpider and the server to check with the origin instead of answering from a stored copy, as the DeepSpider audit does. Without it, a CDN could answer with an old stored response that hides how the origin behaves.
My page has no ETag. Is that a problem?
No. Crawlers download the page in full each time. If the site is large or changes rarely, an ETag can save crawling and bandwidth.
Why is my ETag different on every request?
The ETag probably includes a timestamp, a random value or something that differs between servers. A crawler’s stored ETag then never matches, so 304 never happens. Base it on the content instead.
Is 200 for If-Modified-Since with my own date wrong?
No. A server may always answer with the full page. Only a 304 for a date before the page changed is wrong.
Does the checker use Googlebot’s user agent?
No. It uses DeepSpider’s own user agent and honours robots.txt for DeepSpider. Servers that treat crawlers differently can answer Googlebot in another way.
Where to check the underlying guidance
Read Google’s Crawling December: HTTP caching, the caching section of Overview of Google crawlers and fetchers, HTTP status codes, Search Engine Journal’s report on Google recommending ETags, and the HTTP standards RFC 9110 (conditional requests) and RFC 9111 (caching).