AI search

Google publishes a guide to let its AI agents past your firewall. Check your website isn't blocking them

Google-Agent visits websites to act for users. A new Google help document explains how to allow it in CDNs, firewalls and bot-detection tools.

· Published

Google publishes a guide to let its AI agents past your firewall. Check your website isn't blocking them

Google has published a new guide for website owners on letting its AI agents into their sites. Search Engine Roundtable reported on Tuesday, November 17, that Google added a help document to its crawling documentation, with a changelog entry that reads: "Published a technical guide on how to ensure that Google-Agent has access to your website."

Google describes the agent this way: "Google-Agent is the name used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request." In plain terms, this is not Google's search crawler indexing your pages. It is software visiting your site because a person asked Google to do something there, such as compare products, check details or fill in a form.

According to Search Engine Roundtable, the new document covers three things:

  • How Google-Agent identifies its requests, so your systems can tell it apart from other traffic.
  • Allowing it through your security tools. Google's guidance talks about explicitly allowing Google-Agent traffic in your content delivery network (CDN), web application firewall or bot-detection service, while keeping your existing security controls in place.
  • Troubleshooting when the agent can't get through.

Search Engine Roundtable says the guide is meant to help publishers that want to enable Google-Agent for their website. It is a separate name from Googlebot, the crawler that powers Google Search.

What this means for your business

When Google writes a step-by-step guide for getting past firewalls, it suggests agents are already running into blocks. Many small business websites sit behind bot protection they never set up themselves: a CDN setting, a hosting plan's security layer or a WordPress plugin. Those tools are built to stop automated traffic, and an AI agent is automated traffic. If it hits a CAPTCHA or a block page, the task can fail quietly. The customer may never see your error page. They may just get an answer that doesn't include you, or a competitor's booking link.

This is the same shift we have covered in recent days from the other side. OpenAI has launched ChatGPT Wallet for agent checkout and is rolling out Business Agents in Business Hub. Now Google is preparing site owners for its own agents. Across platforms, more customers will reach you through an assistant acting on their behalf, and your website has to work for that visitor too.

The businesses most affected are the ones where the action happens on the website: online stores, restaurants that take reservations, salons, dental offices and service companies with booking or quote forms. A site that agents can read but not use is only half visible.

The right answer is not to turn security off. Bot protection blocks scrapers, spam and fake form fills, and you still need it. The goal is narrower: let verified Google-Agent traffic through and keep blocking everything else. That is what Google's guide is written for. It is also part of how we approach AI search visibility: being cited by an assistant only helps if the assistant can then complete the task on your site.

What to do this week

  1. Find out what bot protection you have. Ask your web host, developer or agency which CDN, firewall or security plugin sits in front of your site and whether it challenges automated visitors.
  2. Share Google's Google-Agent guide with whoever manages it, and ask them to confirm that verified Google-Agent requests are allowed while other rules stay on.
  3. Check your logs for requests that identify as Google-Agent, and look for blocks or challenges on those requests.
  4. Test your key flows by hand: booking, contact form, quote request and checkout. If a human needs three tries to finish, an agent will likely fail. Our conversion optimization reviews start with exactly these pages.
  5. Don't switch off protection wholesale. Make one targeted change, then watch spam and form submissions for a week.

Sources

Languages: Español · 日本語 · Português · Bahasa Indonesia · Français · Deutsch · Nederlands

All news