Automate Google Search Title Extraction with Autom Search API

Of the many use cases scraping Google search, this one can be really handy for marketers & SEO.

This automation lets you scrape the title and all the info for a page that is ranking in Google SERPs which can be used to analyze what is ranking.

We will be using Autom Google Search API & n8n to build this simple automation. This way you can expand your research on SEO and topics. 

Why API you may ask though?

Can’t we do it manually? Well, doing it manually is prone to mistakes & it would take a lot of time. And therefore, using an API is a better option for use cases where you want to extract bulk data. 

Tools We Will Use To Build This Automation

1. Google Sheets (To Maintain Database)

2. Autom Google SERP API 

3. N8n

Let’s start building our workflow.

Building our Title & SERP Extraction Automation

Before starting to build this workflow, let’s first sign up to all the platforms. 

First, sign up to Autom’s API, you will get free credits to spin the API.

Similarly, you should sign up for free to n8n account, you get a 14-day free trial to test this platform, and later you can either self host, or get the paid plans for it.

Anyway, when you do sign up for both, the last tool in your arsenal you need is Google sheets to maintain our database. 

The format of it would look something like this below: -We will be taking Title, Snippet & Domain for a particular keyword.

Setting up the trigger

For every automation to start a trigger starts it and therefore for this automation, we have a chat based trigger set. In it we will give the kw for which we want to extract title & other details, that we will take in our spreadsheet at the very end.

So any input we give this to trigger, it returns the same, let’s test this out first before proceeding to next module. We will take this output for our API to take as one of the parameters.

Understanding Autom Search API & Adding Node for It in n8n

Before having a node for scraping Google search results in n8n, we will understand a little of how to use the Autom’s API.

Well if you understand APIs first hand, the documentation here will help you out. The API endpoint is “https://autom.dev/api/v1/google/search”.

The method used is “POST” and the necessary parameters are query & API_KEY. The query is sent through Body & the Key through headers. 

And that’s all for understanding the basic working of the API, you can send more parameters as per your specific needs to extract data. 

Let’s now use an HTTP node in n8n, to use this API.This is how you set up the endpoint, and the key.And in the Body, the content type is JSON and the API requires you to send a query in the body.

The value of the query parameter is taken from the chat trigger we have, I have also used the location parameter and its value is “NY”.

Let’s check the automation up until here, I will send a message in my trigger “SEO Tools” & let's see what HTTP node gives the output. As you can see on the right hand side that we are getting the organic results, this is the result from the first page and to keep this automation simple we are taking results from this page only. 

You cannot get all 100 results in one go from Google now after the removal of num=100 parameter.

And therefore to call results from the second page you have to use pagination, which is out of scope of this tutorial but you can always read the documentation to understand that. 

Since the result extracted is in Array, we need to split them out.

Using Split Node

Using this node, we will separate out each search result into different items. This way it will be easy for us to collect them later in our spreadsheet.Let’s test the output of this node. As you can see each item has been separated out & now the final step would be to collect it. 

Collecting Data into Our Google Sheets

Finally, we will collect this in our Spreadsheet, you can build this simple sheet, with headers of data points. 

I am taking ‘Title’, ‘Snippet’ & ‘domain’ for this demonstration purpose, however you can take other data points that are extracted as well. 

Here’s how this node is configured: -Let’s run this whole workflow once again and see how our data sheet gets populated. Now this is just one use case of implementing Google search API into your workflows, however for SEOs & Marketers, It can be used in many ways.

Finally, here is the blueprint for this workflow that you can implement right away in your n8n canvas. The only thing you will need is the API_KEY from Autom’s dashboard.

So if you haven’t tested out the free credits, you can do so by signing up for free from here.