Author Topic: Google doesn't crawl my images product  (Read 1119 times)

mjandia

  • Beginner
  • *
  • Posts: 13
Google doesn't crawl my images product
« on: June 10, 2012, 05:41:24 AM »
Hello,

I have a problem with google. it show in webmaster tools the following message :

URL blocked by robots.txt files.
example :http://www.lacaverneauxmerveilles.com/images/stories/virtuemart/product/006celestine2.jpg

I use jcrawler to create the sitemap

Find the robots.txt attachment

Thanks for youor help

Alan_Smith

  • Beginner
  • *
  • Posts: 40
Re: Google doesn't crawl my images product
« Reply #1 on: June 12, 2012, 04:20:52 AM »
http://www.lacaverneauxmerveilles.com/robots.txt is your robots.txt file and here given file name or directory has not been blocked.

Do one thing, create a sitemap for images and submit it to Google so crawler can find it as fast as possible.
SPINX INC.: A Creative Website Design Firm in Los Angeles www.spinxwebdesign.com

mjandia

  • Beginner
  • *
  • Posts: 13
Re: Google doesn't crawl my images product
« Reply #2 on: June 13, 2012, 01:27:48 AM »
Thanks. How to do that, please ?

coppo82

  • Beginner
  • *
  • Posts: 32
Re: Google doesn't crawl my images product
« Reply #3 on: December 19, 2012, 02:16:24 AM »
i have the same problem , how did you solve it ????
the best solution for the joomla virtuemart CSV
http://joomdonation.com/components/csv-advanced.html

P2 Peter

  • Jr. Member
  • **
  • Posts: 140
Re: Google doesn't crawl my images product
« Reply #4 on: December 19, 2012, 03:35:19 AM »
open the robot.txt in your site root folder

delete the line that says : Disallow: /images/

This line tells search bots not to crawl this folder with your images. But I would recommend that you set new rules for the bots so they only crawl the folder with virtuemart images. So you need to disallow the folders that you don't wanna have crawled

for example

Disallow: /images/slideshows/
Disallow: /images/banners/
Disallow: /images/sampledata/
Insanity: doing the same thing over and over again and expecting different results.  - Einstein

coppo82

  • Beginner
  • *
  • Posts: 32
Re: Google doesn't crawl my images product
« Reply #5 on: December 20, 2012, 04:56:21 AM »
this is my robots.txt and i already delete the line /images
but the problem still  :-(

what it sound very strange is that over 1500 are 1450 are blocked .....  94 % of image are blocked from robots.txt  and what about the other 6 % ????  i don't understand where the problem could be .

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
the best solution for the joomla virtuemart CSV
http://joomdonation.com/components/csv-advanced.html

P2 Peter

  • Jr. Member
  • **
  • Posts: 140
Re: Google doesn't crawl my images product
« Reply #6 on: December 20, 2012, 06:15:28 AM »
personally, I don't see any advantage why Google should be allowed to index your images. I prefer that Google indexes the pages wherein the images are shown, instead of only the images without further text.

Just remember that Google is not going to report this problem immediatly as solved because you changed robot.txt a halfour ago. Sometimes it takes a month before the google crawler revisits your pages. Only then will Google say.... "Owwww...  coppo gave me an new folder to crawl... yummie" and then the crawler will analyse the new folder and add it to your indexed pages.

But again... maybe it's better to let Google only crawl your pages wherein the images reside.
Insanity: doing the same thing over and over again and expecting different results.  - Einstein

coppo82

  • Beginner
  • *
  • Posts: 32
Re: Google doesn't crawl my images product
« Reply #7 on: December 20, 2012, 07:02:01 AM »
ok i rewrite a new robots.txt and then i will wait  :-(   

thanks a lot

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Disallow: /images/banners/
Disallow: /images/sampledata/
Disallow: /images/jsn_is_thumbs/
the best solution for the joomla virtuemart CSV
http://joomdonation.com/components/csv-advanced.html

VirtueMart Forum

Re: Google doesn't crawl my images product
« Reply #7 on: December 20, 2012, 07:02:01 AM »