Scraping URLs from Google Search Results

Domx path is working consistently for me on google searches. Try capturing the entire box at the top of a search result.

That has a DomX of: //div[@id='rso']/div[3]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]

The first Div increments with each result so I’ve inserted: //div[@id='rso']/div[$nSearchResultsRow.Number:toString$]/div[1]/div[1]/div[1]/div[1]/div[1]/div[1]

Make sure the only things you are using for the object properties are the HTML Tag, the DOMX Path, and maybe the HTML HasFrame.

I’ve placed the recorder action in a loop that goes 5 times. I start with the nSearchResultsRow

equal to 3 since that seems to be the first row and then increment from there in the loop.

To get the URL I’m grabbing the “HTML InnerText” property which for example looks something like this:

Best Vegan Chocolates: Ideal for Plant-Based TreatsDallmann Confectionshttps://dallmannconfections.com › collections › vegan-c..

So you would need to use the string tools to isolate the URL out of there!

Since the number of results will vary, you will need to error trap if you run out of rows and need to click the “More Options” button to expand the results, and then the Next button. Note that clicking next probably resets the rows so you’ll need to set variable back to 3 to start scraping again.