VirtueMart Forum

VirtueMart 2 + 3 + 4 => Security (https) / Performance / SEO, SEF, URLs => Topic started by: parfumylacno on September 05, 2016, 08:45:55 AM

Title: Google stops indexed/crawled my web
Post by: parfumylacno on September 05, 2016, 08:45:55 AM
Hi all,

I m trying to find solution of this problem on another forums too, but unlucky. Hopefully, i will find solution here :)

There are my problems with my web site.

I was very surprised when I saw some results in ahref s tool. In section of Crawled Pages (i think that it means Google indexed pages) i have 0 crawled pages - and i have no idea why. You can see it in picture.

Then I go to google webmasters tools and there i can see another problem - Google has no access to CSS and JavaScript files - you can see print screen. Then I make changes in robots.txt -

And in sitemap section, there i can see another problem - google stops indexed/crawled my web. There are indexed only 78 from 1737 sites. And there is only warning (no error) about one url from one article - i have disabled that article but without help.

And there are another problems too :D when you search site:parfumylacno.sk in google most of results have Https:// but it have to be only http://! why is there https even it doesnt work and I have never setting https :)


User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/sampledata
Disallow: /images/sampledata
Disallow: /images/sampledata
Disallow: /images/sampledata
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

User-Agent: Googlebot (I have ADD that)
Allow: /*.js*
Allow: /*.css*

but it seem that it doesnt help at all.

Please, can anybody help me? On other forums i cant find help yet :)

i have Joomla! 2.5.25 and Virtuemart  2.6.10 and web with that issues is http://www.parfumylacno.sk

Thank you very much,
Have a nice day.
Title: Re: Google stops indexed/crawled my web
Post by: jjk on September 05, 2016, 20:40:02 PM
Did you check your 'Crawl Stats' and tried the 'robots.txt' tester in the Google webmaster tools 'Search Console' > 'Crawl' section.
Perhaps the Arefs website simply shows it wrong.

I don't see your addition to your robots.txt. Still seems to be the standard Joomla robots.txt

In mine I had added this:

User-Agent: Googlebot
Allow: .js
Allow: .css




Concerning your https issue: It looks like your website is accessible via https using a self signed certificate from your webhosting service. You may consider to ask your hosting company if they would install one of the free ssl certificates by 'StartSSL' or 'Let's Encrypt'. You can also set http as the prefered url in Google webmaster tools.
Title: Re: Google stops indexed/crawled my web
Post by: parfumylacno on September 06, 2016, 09:51:11 AM
thank you, yes i checked it in webmaters tool - can you see image above that show that google has indexed only 78 from 1737 sites.
Title: Re: Google stops indexed/crawled my web
Post by: VMTemplates.net on September 06, 2016, 12:07:17 PM
Hi,

did you tried to ask for the help here: https://productforums.google.com/forum/#!forum/webmasters

Thanks,
J.
Title: Re: Google stops indexed/crawled my web
Post by: jjk on September 06, 2016, 12:39:04 PM
Does webmaster tools show lots of crawling errors?
Title: Re: Google stops indexed/crawled my web
Post by: Adwans on September 06, 2016, 16:10:53 PM
Hej,
Different tools give different results. MOZ, Arefs, Majestic. From my location,  using SEO quake, you got:
Page .. of about 1,810 results (0.26 seconds)  - from Polish Google SERP.
Why not to use: "site:www.parfumylacno.sk" in Google?

Title: Re: Google stops indexed/crawled my web
Post by: parfumylacno on September 06, 2016, 21:23:01 PM
thank you for advice google forum, I wrote to them, we will see if anyone helps https://productforums.google.com/forum/#!topic/webmasters/C7G4xR-eBYM;context-place=mydiscussions :)

to another questions: I had use a site:parfumylacno.sk (you can see in attached picture) but there is another problem too: google insert here https but i dont know why. I have not ssl certificate or https setings, there have to be only http. little funny mystery for me (and other on others forums where we are looking for solution)

and there are 5 errors - see in picture

thanks a lot to all of you
Title: Re: Google stops indexed/crawled my web
Post by: jjk on September 07, 2016, 23:35:16 PM
Concerning your https issue I wrote above already that your webhosting company has installed a self signed ssl certificate. At many webhosters this is a default configuration.
Just look at your site using https://www.parfumylacno.sk/ (https://www.parfumylacno.sk/) and you will see. If you use an up to date browser, it will come up with a warning and ask if you want to add an exception. If you click yes, the unsigned certificate will be stored in your browser and the warning will disappear. Googlebots nowadays always try if a website is accessible via https. If you don't want https pages indexed, you will have to set http as your preferred prefix in webmaster tools.

Concerning the 'component' in the url, it looks to me like you have included VM products into a Joomla article. Example: http://www.parfumylacno.sk/index.php?option=com_content&view=article&id=132&Itemid=313 (http://www.parfumylacno.sk/index.php?option=com_content&view=article&id=132&Itemid=313). Maybe some of those have been indexed by Google in the past and now it can't find them anymore.
Title: Re: Google stops indexed/crawled my web
Post by: parfumylacno on September 08, 2016, 08:38:46 AM
Thank you very much.

Yesterday i wrote with my webhosting company about SSL certificate to solve this problem. But now i m not sure if use it - have to study pros and cons, whichone use (free or paid and why..) to which page sites (all, or only login, formulars, payment..)
And i go to webmaster tool to look how to set http as preferred prefix, thanks.

with your example of product in joomla article you make me another wrinkles :) whyyy is that link NON SEF url http://www.parfumylacno.sk/index.php?option=com_content&view=article&id=132&Itemid=313 :D it have to be that: http://www.parfumylacno.sk/ako-funguju-feromonove-parfumy  why SEF url does not work, or work when it wants? :D

thank you very much for your help
Title: Re: Google stops indexed/crawled my web
Post by: GJC Web Design on September 08, 2016, 10:34:47 AM
Quotewhy SEF url does not work, or work when it wants?

it is not a case of works when it wants

if Google finds a url like http://www.parfumylacno.sk/index.php?option=com_content&view=article&id=132&Itemid=313 it will try it and index it.. it works
there is to my knowledge no way to then reroute it to the sef equivalent

but if u use a link like index.php?option=com_content&view=article&id=132&Itemid=313 in content it should be rewritten as sef when rendered
Title: Re: Google stops indexed/crawled my web
Post by: parfumylacno on September 08, 2016, 11:22:04 AM
thank you, but that make duplicity now when works both links/urls  http://www.parfumylacno.sk/index.php?option=com_content&view=article&id=132&Itemid=313  and http://www.parfumylacno.sk/ako-funguju-feromonove-parfumy because it is same site, same content with difrent url.

how can i fix it?

if i understand you right = it is because i had used somewhere in article link to http://www.parfumylacno.sk/index.php?option=com_content&view=article&id=132&Itemid=313 instead of index.php?option=com_content&view=article&id=132&Itemid=313? So i have to find all links with all url and change it to index.php?option=com_content&view=article&id=132&Itemid=313 ?

thank you for your advice and patience :)
Title: Re: Google stops indexed/crawled my web
Post by: GJC Web Design on September 08, 2016, 11:33:23 AM
make sure all your links are sef everywhere

assume if Google has already indexed then 301 the non-sef to sef in your htaccess
Title: Re: Google stops indexed/crawled my web
Post by: parfumylacno on October 05, 2016, 09:44:22 AM
thank you very much, sory about late reply..

i made redirections of 404 pages in joomla redirect component and it helps - i think it helps, because in webmasters tools is now slowly growing a number of indexed sites (from cca100sites now it is 300 = still it is not enought (there are about 1700 sites, but it should be better, it seems like)
can i made some redirection direct in htaccess - can i ask how? because i m not very familiar with htaccess, i have there only redirection to solve duplicity = but i found that it is not good too = as you can see in http://www.ragepank.com/redirect-check/ tool there are some results:

http://www.parfumylacno.sk returns a 200 (OK) response. PR N/A
http://parfumylacno.sk returns a 200 (OK) response. PR N/A
http://www.parfumylacno.sk/index.php returns a 200 (OK) response. PR N/A
http://parfumylacno.sk/index.php returns a 200 (OK) response. PR N/A
http://www.parfumylacno.sk/index.htm returns a 404 not-found response
http://parfumylacno.sk/index.htm returns a 404 not-found response
http://www.parfumylacno.sk/index.html returns a 200 (OK) response. PR N/A
http://parfumylacno.sk/index.html returns a 200 (OK) response. PR N/A
http://www.parfumylacno.sk/index.shtml returns a 404 not-found response
http://parfumylacno.sk/index.shtml returns a 404 not-found response
http://www.parfumylacno.sk/index.asp returns a 404 not-found response
http://parfumylacno.sk/index.asp returns a 404 not-found response
http://www.parfumylacno.sk/default.asp returns a 404 not-found response
http://parfumylacno.sk/default.asp returns a 404 not-found response
http://www.parfumylacno.sk/default.aspx returns a 404 not-found response
http://parfumylacno.sk/default.aspx returns a 404 not-found response
http://www.parfumylacno.sk/index.aspx returns a 404 not-found respons

and there are a lot of 200 (ok) codes, there should be only one
6 pages returned a 200 response. This indicates potential for duplicate content problems. Ideally, only http://www.parfumylacno.sk OR http://parfumylacno.sk should return a 200 response.

but i dont know how to fix it - i have set htaccess to optimal setings for joomla to reduce duplicity but i dont know why it does not work properly.

it is maybe not a problem, just to be sure i attached a htaccess file in attachments.

thank you very much

Title: Re: Google stops indexed/crawled my web
Post by: Studio 42 on October 05, 2016, 23:02:35 PM
Hi,
If you watn have always SEF links, then migrate to Joomla 3.
The problem is that the Joomla 2.5 router do not SEF all links, but Joomla 3.6 Do ti perfectly include inline links.
Title: Re: Google stops indexed/crawled my web
Post by: parfumylacno on October 10, 2016, 15:07:50 PM
thans, really Joomla 3 solves all this problems with duplicity? but now i cant migrate to Joomla 3, by the now i have to found any solution with Joomla 2.5 :) but thanks again, i will notice it.
by the now another little issue: how can remove from url ID of article :) to example http://www.parfumylacno.sk/clanky/33-parfumerie-a-eshopy/837-parfums-sk    837 id of article and 33 id of category.. seems nonsense to mee to be here IDs..
Title: Re: Google stops indexed/crawled my web
Post by: Studio 42 on October 10, 2016, 18:54:08 PM
You have only one way(or use a SEF plugin)
http://forum.joomla.org/viewtopic.php?t=826165
To remove ID from categories(this can be hidden links), simple add menu links shoud remove the ID too, in the category.
Of course you can add menu links for each article too, but it's a pain
Title: Re: Google stops indexed/crawled my web
Post by: parfumylacno on October 11, 2016, 14:12:54 PM
thank you! :)