Boneyard Tools

Robots Meta Tag Generator

Pick the crawl and indexing rules you want and get a ready-to-paste robots meta tag plus the equivalent X-Robots-Tag header. Toggle noindex, nofollow, snippet and preview limits, and an unavailable-after date, all updated live.

How to generate a robots meta tag

  1. Choose whether crawlers may index the page and follow its links.
  2. Add any extra directives: noarchive, nosnippet, snippet and preview limits.
  3. Copy the robots meta tag into your <head>, or copy the X-Robots-Tag header for non-HTML files.

Examples

Keep a page out of search but still follow links

index: off, follow: on
<meta name="robots" content="noindex, follow" />

Allow indexing with a large image preview and no text snippet limit

max-image-preview: large, max-snippet: -1
<meta name="robots" content="index, follow, max-snippet:-1, max-image-preview:large" />

Frequently asked questions

What is the difference between the robots meta tag and robots.txt?

robots.txt controls crawling: it tells bots which URLs they may request. The robots meta tag controls indexing and display: it lets a page be crawled but tells search engines not to index it, follow its links, show a snippet, and so on. A page blocked in robots.txt can never read its own robots meta tag, so to reliably keep a page out of search you allow crawling and add a noindex meta tag instead.

What is the difference between the robots meta tag and the X-Robots-Tag header?

They carry the same directives. The meta tag goes in the HTML <head>, so it only works for HTML pages. The X-Robots-Tag is an HTTP response header, so you can apply the same rules to PDFs, images, and other non-HTML files that have no head section. This tool outputs both from one set of options.

What does noindex, nofollow mean?

noindex tells search engines not to include the page in their index, so it should not appear in results. nofollow tells them not to follow the links on the page or pass ranking signals through them. You can mix and match: noindex, follow keeps a page out of search while still letting crawlers discover linked pages.

What do max-snippet, max-image-preview and max-video-preview do?

They cap how much of your content can appear in results. max-snippet sets the maximum text snippet length in characters (-1 means no limit, 0 means no snippet). max-image-preview can be none, standard, or large. max-video-preview sets the maximum video preview length in seconds. They are honored by Google and several other engines.

Is my input sent to a server?

No. The tag and header are built entirely in your browser with JavaScript. Nothing you toggle or type is uploaded, logged, or stored anywhere.

Where do I put the robots meta tag?

Place it inside the <head> element of the page you want to control, before the closing </head> tag. For files that are not HTML, send the X-Robots-Tag header from your server or CDN config instead.

Related tools