GuidesFound

SEO automation: what to let a tool change

One SEO tool says it “runs on autopilot and deploys optimizations on its own.” That’s fine for a meta description that runs long. It’s a problem for the robots.txt rule someone added on purpose.

Josh Pigford, founder of FirstIndex7 min read
The short answer

SEO automation is software that finds SEO problems and fixes some of them. It’s good at small, mechanical fixes: 69% of the 291 startup homepages we checked needed one. It can’t tell a mistake from a choice, like a noindex on a page you haven’t launched or an AI crawler blocked on purpose. Check each change before it goes live.

What can SEO automation do?

It does three jobs, and most tools do one or two of them:

  • FindCrawl your site for broken links and missing tags, pull your Search Console numbers, track your rankings and count your mentions in AI answers.
  • DraftWrite the brief, the title, the description, the schema and a list of internal links to add.
  • ShipPut the change on your live site. This is where it can go wrong.

Finding and drafting are safe to hand to software, because a person reads the result before anything changes. Shipping is different. A tool that ships on its own is betting it understood why your site looks the way it does. In a Reddit thread on the best SEO automation people had seen, the top answer said it “wasn’t content generation. It was content maintenance.” Further down, someone warned: “You have to create a log of what’s been done or the ai will constantly ruin its own work.”

What SEO problems do startup sites have?

On November 15, 2026, we checked the homepages and robots.txt files of 291 startup sites, the same random sample of active B2B companies in a public copy of Y Combinator’s directory we’ve used for our other tests. We sorted every problem into two groups. A mechanical fix is one a tool can make without undoing anything you chose, like adding a missing tag. The other group is anything that looks like a mistake but could be on purpose.

201 of the 291 homepages (69%) had at least one mechanical fix to make, and the typical site had one. 37 (13%) had something in the second group:

What we found on 291 startup homepages, by kind of problemShare of 291 startup homepages with each problem, checked November 15, 2026. Mechanical fixes: no structured data 35%, meta description over 160 characters 27%, no canonical tag 22%, no og:image 20%, title over 60 characters 16%. Might be on purpose: AI crawlers blocked in robots.txt 9%, homepage redirects to another domain 3%, canonical points to another page 1%, homepage set to noindex 1%. Bars start at 0%. Mechanical: a tool can fix it No structured data35% Description over 160 characters27% No canonical tag22% No og:image for link previews20% Title over 60 characters16% Might be on purpose: needs a decision AI crawlers blocked in robots.txt9% Homepage redirects to another domain3% Canonical points to another page1% Homepage set to noindex: 2 sites, under 1%
Share of 291 startup homepages with each problem, from the raw HTML and robots.txt, November 15, 2026. A site can have more than one.

Two of the most common mechanical items, a description over 160 characters and a title over 60, are rules of thumb, not Google rules. Leave them out and it’s still 145 sites, half the sample. And we read the HTML a site sends before any JavaScript runs, so a site that adds its tags in the browser can look worse here than it is.

Which SEO fixes are safe for a tool to make?

The mechanical ones, and they’re common:

  • Structured data: 103 sites had none (35%)A tool can add Organization or SoftwareApplication schema from facts you give it once.
  • Meta description: 79 over 160 characters (27%), 12 missingGoogle may show it as the snippet “when it describes the page better than other parts of the content.”
  • og:image: 59 missing (20%)The picture a link shows when someone shares it in Slack or on LinkedIn.
  • Title: 48 over 60 characters (16%), 3 missingGoogle may show a different title than yours. It picks the one it shows using “a number of different sources.”
  • No sitemap found: 48 sites (16%)A sitemap is the list of your pages you hand to Google. These sites had none in robots.txt or at /sitemap.xml.
  • Alt text: 20 sites had an image with noneOnly 142 of 13,082 images (1.1%), so this one is mostly done. Google uses alt text “to understand the subject matter of the image.”

Low risk isn’t no risk. A title and a description are words about your product, and a tool writes them from what it can read on the page. They’re quick to check, and worth a look before they go live.

Which changes need a person to decide?

Anything that could undo something you chose on purpose. These are rare, and to a tool each one looks exactly like a bug:

  • AI crawlers blocked: 25 sites (9%)A tool selling AI visibility wants to delete the block. But the owner may not want their pages used to train AI models. That’s their call.
  • Homepage redirects to another domain: 8 sites (3%)Usually a rebrand or a move. “Fixing” it splits the site in two.
  • Canonical points to another page: 3 sites (1%)It might be a copy-paste error, or the page might really be a duplicate.
  • Homepage set to noindex: 2 sites (under 1%)A site still in stealth, or a launch nobody finished. Only someone at the company knows which.

Take a closer look at the AI crawler blocks. All 25 sites blocked Common Crawl’s crawler, and most also blocked the ones OpenAI, Google and Anthropic use for AI training. None blocked the crawlers OpenAI and Anthropic use for AI search:

Which AI crawlers 291 startup sites block in robots.txtNumber of 291 startup sites whose robots.txt blocks each crawler from the whole site, November 15, 2026. Archive and training crawlers: CCBot 25, GPTBot 19, Google-Extended 19, ClaudeBot 18. AI search crawlers: OAI-SearchBot 0, Claude-SearchBot 0. Bars start at 0 sites. Archive and AI training crawlers CCBot (Common Crawl)25 sites GPTBot (OpenAI)19 Google-Extended (Gemini training)19 ClaudeBot (Anthropic)18 AI search crawlers OAI-SearchBot and Claude-SearchBot: 0 sites
Sites whose robots.txt blocks each crawler from the whole site, of 291 startup homepages, November 15, 2026.

OpenAI treats them as separate switches, so a site can “allow OAI-SearchBot in order to appear in search results while disallowing GPTBot” to keep its content out of training. Deleting those 25 blocks does nothing for ChatGPT search. It only undoes the owner’s call on training. To see what your own file allows, our robots.txt checker shows which of 14 search and AI crawlers can read a page.

Some auto-fix tools can make exactly these changes. Search Atlas’s OTTO SEO, the tool quoted at the top of this page, changes canonicals and redirects, and deploys on its own unless you turn on its approval mode. Ahrefs lists noindex tags, canonicals and redirects among what its Patches feature can change, with “publish when ready” and a rollback. Alli AI says it deploys titles, schema and internal links “across your whole portfolio without waiting on engineering.” If you use one, turn the approval step on and read what it wants to ship.

Does Google penalize automated SEO?

Not for being automated. Google’s line is about why the content exists: “Appropriate use of AI or automation is not against our guidelines.” What breaks its rules is using automation “with the primary purpose of manipulating ranking.”

The trap is volume. Google calls it scaled content abuse when “many pages are generated for the primary purpose of manipulating search rankings and not helping users,” and one of its examples is using AI tools “to generate many pages without adding value for users.” Since March 2024 it applies no matter whether a person, a tool or both made the pages. A tool that publishes a hundred near-copies of a page in a week is the risky kind of automation, whatever the pages say.

One automated fix is easy to get wrong: blocking a page in robots.txt to get it out of Google. Google says robots.txt “is not a mechanism for keeping a web page out of Google.” Use noindex for that, and leave the page crawlable so Google can see it.

Should you use a tool, build a workflow or have it done for you?

There are three ways to get the work done, and they differ in who reads each change before it ships:

  • Done for you: FirstIndexWe do the full plan for software companies. Real changes ship to your site every day, and we check each one before it goes live. It’s $2,000 a month. Hiring the full plan out costs about $25,000 a month at an SEO agency. 30+ software companies already work with us, and every page on this site, this one included, shipped through the same process we run for clients.
  • An auto-fix toolFast on the mechanical fixes. Somebody on your team becomes the approver.
  • A workflow you buildMake, Gumloop, n8n or Zapier wired to an AI model and your CMS. You choose every step, and you build and keep up the checks yourself.

FirstIndex is the right fit if you run a software company and want to get named when buyers ask AI, without running SEO yourself. It isn’t the right fit if you want a dashboard and plan to run it yourselves. Here’s what hiring out SEO costs, line by line.

Questions people ask about SEO automation

Can ChatGPT do SEO?

It can draft the parts that are words: a title, a description, schema, an outline. It doesn’t put them on your site, and someone still has to read them first.

Will SEO be replaced by AI?

No. AI answers are built from pages the engines can find. Google says a page must be indexed to show as a link in AI Overviews or AI Mode, and ChatGPT search finds sites with its own crawler. Software is taking over more of the work, but the work is still SEO.