If you encounter a warning saying images cannot be crawled because of robots.txt restrictions or it says you have to wait three days, this article will explain the issue in more detail.
Diagnostics Tab
First, let’s understand the diagnostics tab. When you see errors and warnings in the diagnostics tab, and you have fixed all issues. However, it can take up to 30 days for the problems to clear up. This is because Google’s Bot for Google Shopping does not crawl every page every day. You will notice that after two days, your issue count will drop lower and lower. So do not panic that you don’t see it fall within the next day, wait at least two days and check the original error count vs. the new one. If it’s dropping down, you have done an excellent job.
Understanding the message
When you receive a message related to the robots.txt, it does not mean it is related to this file. It can also mean that you have an error in the image URL or the image size is too large, which might cause your server not to respond quickly enough.
Single or multiple issues can trigger this warning.
Checklist
Let’s create a simple checklist for you to confirm that all settings on your website meet Google’s requirements.
- Make sure Google can crawl your images by allowing access in robots.txt. To know what to do, please follow Googles instructions.
- If you have additional images, as Google does not use them much at all at this current stage, is to remove them from your data feed
- Make sure your product covers at least 75% of the image size
- Optimize your images. Large images can cause delays in Google crawling or your server timing out. I recommend using compression tools to make your images lighter and also make sure the pixel size is not too large
- Check you have no violations on the images such as watermarks, borders, text, promotions, etc.
- You can check when Google last crawled your product by logging in to Google Merchant > Products > List and click on your product. Under technical details, you can see when it was last crawled.
If you have checked everything and there are no issues, wait for a few re-crawls, and Google will automatically resolve the issue.
Optional
You can force a recrawl quicker by changing the filename or appending a random query string.
hi buddy thanks for this. Do you know if its possbile to use Clickfunnels with google shopping ads? I have read that it is not possible to use Clickfunnels and advertise on shopping ads because google cant crawl the site for images. Any tips will help. Cheers
As the click funnel with shopping ads is to ensure users buy your product is so short, the key ingredient is ensuring you have a high quality data feed, which allows users to see in the ad the correct information. Then when they land on the product landing page, make sure the Title, image, description, price, availability and add to cart is above the fold. Which then allows the click funnel to be clear and obvious.
However if you mean the app clickfunnel, best to contact the support team of the app itself for advice. I don’t use this for any of my clients. And have never used it.