I noticed recently that Googlebot seems to be visiting a site I manage – it is visiting the site at a rate of hundreds of thousands of times a day! Most of these visits seem to be to the auto generated rss feeds that WordPress seems to create. However concatenating on to the end of the urls being visited are some parameters.
…thereby making the urls unique.
The parameters on the end are usually longer than the above example but there appears to be literally hundreds of the urls all slightly different and to various different categories and authors all being visited by various IP addresses that I can trace back to Google!
I can not find any of these urls in the sitemaps (these are generated by Yoast). I do not believe that there are any other plugins or functions generating sitemaps with or without the aforementioned rss feeds. I can not even find the feeds without the parameters listed in the sitemaps generated by Yoast.
These visits were not happening until recently. From what I can see this has only been happening the last 2 weeks. No changes were made to Yoast or to any plugins that I know of.
The IP addresses of the robots visiting the site seem to be legitimate Google IPs. I can’t understand why Google would be wasting its own bandwidth visiting these urls that are all almost identical.
Regardless though, the above is chewing through bandwidth at a rate of knots!
Any help or suggestion as to why this may be happening would be most appreciated.
Read more here: googlebot visiting WordPress feed urls