How to check what resources are blocked
Google Webmaster Tools can help you find the problem that may be blocked GoogleBot accessing your site. From Search Console Dashboard, navigate to Google Index > Blocked Resources.
Handling resources blocked by
robots.txt: Allowing the bots to index anything and everything in your site, can actually harm your sites rankings. If you using WordPress, I unblocked
wp-includesfolder where has default scripts & files which used by many themes and plugins. The next position is
wp-content with the same reason. Here is my new
User-agent: * Disallow: /wp-admin/ Disallow: /trackback/ Disallow: /xmlrpc.php Disallow: /go/
Google Webmaster Tools shows screenshots rendered both as Googlebot
and as a typical user, if the problem has fixed, you will got the result similar the image below:
After solving the mobile usability issues, you can Submit to index, because the changes you made can influence your rankings in search results. Submit to index will make Google crawl your new changes and index your page faster. This button only appeared beside the Status result, after the render processed.
- You have two choices:
- Crawl only this URL: You are allowed to submit only 500 URLs per month.
- This URL and its direct links: You are allowed 10 submission per month for this option.
- Be sure your website pass the Mobile-Friendly Test. In Mobilegeddon, it’s very important because Google announced that mobile-friendliness will become a ranking factor in the mobile search results on April 21, 2015. With this update, GoogleBot now has a smartphone, it’s using to visit and indexing your website, haha.
- Guarantee of technical quality for running your website flawlessly, some common problems causes GoogleBot is indexing your website hardly like DNS, Firewall, Over-Optimize for Server Configuration…
TIPS: Don’t include your website’s XML sitemap in
robot.txtbecause Google doesn’t interested with spider’s arguments that available in its.