SEO GetsSEO Gets
Back to blog

Google Indexing Script

Get indexed in few hours. No tricks, no hacks, just a simple open source script

It seems like somewhere between 2022 and 2023, Google has changed the way they index new pages. It used to be that if I published a new page to my site with enough internal links pointing to it, it would get discovered and indexed within a few days.

That no longer seems to be the case, at least not on the sites I own.

What I've noticed that Google is now only indexing new pages if:

  1. they have backlinks pointing to them;
  2. the site owner ask for the page to be indexed via Google Search Console;

That actually makes sense if you think about the amount of new pages being published every day, especially now that there's a gazillion of services automatically writing content using AI.

Manually requesting indexing takes around 1 minute, so submitting a handful of pages is not that time-consuming.

Here's how you can index your pages using GSC:

  1. Open Google Search Console
  2. Enter the URL you want to request index
  3. Wait a bit and then click on Request Indexing
  4. Wait between 24-48 hours to see it on Google
  5. It's that simple! 🎉
Requesting manual indexing on Google

What about Bulk Indexing?

I recently wanted to index 26 pages and I really didn't want to do it manually. It wouldn't take that long, but next week I'd likely have a few dozen more pages to index, and the week after that, and so on.

Manually requesting indexing is not a scalable solution, so I started looking for alternatives.

If you work in the SEO space you must have seen how the marked has flooded with SaaS platforms that can index your page in hours. Ilias tested a few of them and wrote an article about the 5 Best Website Indexing Tools that you can check out if you prefer a less technical solution.

Most of them are paid services with a monthly subscription model — what a surprise! As much as I dislike paying for something monthly, I don't mind if the service is constantly adding value to my business. But that's not the case here. I just need to submit a few pages once in a while.

After a bit of research, I eventually found the Indexing API that powers Google Search Console. To my surprise, the API is free, documented and officially supported by Google.

Ok, so what's the catch?

First you have to remember that Indexing != Ranking. Just because your page is indexed doesn't mean you'll get any traffic from Google. Using the Indexing API is not really giving you an unfair advantage, you still have to put in the work to rank your pages.

But anyway, here's an important quote from the official documentation:

Currently, the Indexing API can only be used to crawl pages with either JobPosting or BroadcastEvent embedded in a VideoObject. For websites with many short-lived pages like job postings or livestream videos, the Indexing API keeps content fresh in search results because it allows updates to be pushed individually.

In short, this API is meant to be used by Job Boards and Events platforms, supposedly because of the high volume of pages they have to add/remove due to the nature of their business.

But as reported by hundreds of site owners on Twitter, myself included, it works for ANY page, regardless of being a Job Post or Event page. It doesn't even need to have a Schema markup on it.

It's unclear if that's a bug or if the documentation is outdated, but the fact is that it works, and it works really well! Will it work next month or next year? No one knows, but for now, it's a great solution and as far as I know, it has been working for at least since 2022.

Here's an example of one of my sites that has about ~35 pages. All pages are on the sitemap and there's a bunch of internal links to each page. Up until last week, only 3 of those pages were indexed by Google without me having to do anything about it.

After using the API above, Google had indexed all pages in less than 48 hours 🤯

Report on recent indexed pages

Google Indexing Script

I decided to pack everything I learned into a simple script that you can use to index an entire site. The script is free and open source, so you can also modify it to fit your needs. It's available on my GitHub at goenning/google-indexing-script.

You can find the instructions on how to use it on the README. It's pretty straightforward, especially if you're familiar with Node.js and JavaScript.

Here's a quick summary of what the script does:

Find all site pages using sitemaps.
For each page, check if it's indexed or if already requested previously.
  ✅ If yes, skip it. We don't want to spam Google.
  ❌ If not, request indexing.

Here's the script output after executing it on one of my sites 82 pages were found, 79 were indexed, 1 did not need to be indexed and 2 were requested to be indexed.

Example of script output

That's it! 🥳

Don't forget to check out the SEO Analytics tool I'm building. It's like Google Search Console on steroids!

SEO Gets