VirtueMart Forum

VirtueMart 2 + 3 + 4 => General Questions => Topic started by: marscom on December 03, 2018, 14:23:40 PM

Title: Again: massive server overload with J 3.9.1
Post by: marscom on December 03, 2018, 14:23:40 PM
Hi all,

as stated in this thread: http://forum.virtuemart.net/index.php?topic=141342.new;topicseen#new there was a massive server overload with J 3.8.13. As GJC Web Design proposed, I enabled the 404 error handling in VM - and this helped. Before I start clicking and enabling/disabling around: Now we have again a really massive server overload since the update to J 3.9.1. Is there another fix for that problem? Or do I simply have to ... disable the error handling again?

Thanks in advance for your precious hints,

Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: jenkinhill on December 03, 2018, 15:14:45 PM
You don't say which VM version, but the 404 issue was fixed for com_virtuemart.3.4.3.9989  - http://dev.virtuemart.net/projects/virtuemart/files
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 03, 2018, 15:23:22 PM
Hi jenkinhill,

thanks for your quick reply! Actually, I use 3.4.2 and I cannot see any update hint. Is it a release candidate or do I simply have to download the files you linked to and upload them - without destroying the current shop?

Sorry for my hesitating with updating ... but I'm rather careful as this shop is quite big...

Kind regards
Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: jenkinhill on December 03, 2018, 15:29:28 PM
As with all updates, of Joomla as well as extensions, you should test on a copy of the live site first before updating the live site itself.  You won't lose any data etc when you update. 3.4.3.9989 is officially a beta, but has a lot of bug fixes, and is the version I use.
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 03, 2018, 15:34:35 PM
I'll give it a try... Thanks a lot! I really appreciate your suggestions.

Let's cross fingers  ;)

Kind regards

Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 04, 2018, 15:40:51 PM
Hi jenkinhill,

after having updated according to your beta version, the server is even more overloaded (see attachments). We really are at the end of our knowledge. The server is nearly breaking down...

Every hint appreciated.

Kind regards

Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: Studio 42 on December 04, 2018, 15:47:33 PM
When i check your graphics, you have more traffic as before.
Have you check if this is not some hack attempt and have nothing todo with Virtuemart ?
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 04, 2018, 16:01:25 PM
We actually are subscribed to SiteGuarding to protect the site. They usually do their job very good.

The thing is, that the first series of overload had to do with the 404-error-handling which we activated according to a hint on this forum. The new overload started when updating Joomla to 3.9.1. And now, after having updated VM, it seems even more loaded...

What information or screenshots can I provide you?
Title: Re: Again: massive server overload with J 3.9.1
Post by: jenkinhill on December 04, 2018, 16:07:06 PM
I'd look at the raw access logs to see if there is a clue there. It is a good way to check for hacking attempts.
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 04, 2018, 16:15:41 PM
Doesn't look like. Just the usual ones, like this:

2018-12-04T14:14:33+00:00   INFO 91.210.147.71   joomlafailure   Benutzername und Passwort falsch oder das Benutzerkonto existiert noch nicht!

But that's about a hundred or so per day.

But what makes me a little wondering is the mass of Google requests according to the stat (attached). There are some 1314186 hits in december just by Google.
Title: Re: Again: massive server overload with J 3.9.1
Post by: AH on December 04, 2018, 16:29:05 PM
You may want to turn Virtuemart 404 handling off

Dodgy urls are rewarded by a VM shop page which drains resources - some people want this - I prefer a true redirect or nothing!

Another thing you can do is to return a very unhelpful and small 403 message using .htaccess



ErrorDocument 403 "Forbidden"

If you add this at the end of .htaccess - then you only give over a few bites to people using crap urls instead of rewarding them with a 404 page and bandwidth.


Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 04, 2018, 16:32:39 PM
I just added the htaccess code. Let's see...

Thanks so much for your caring!!!

Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: AH on December 04, 2018, 16:44:35 PM
Did you turn off VM error handling? - if you leave it on, some bad shop requests will get redirected to a VM shop page
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 04, 2018, 16:51:50 PM
Yes, it's turned off:
Title: Re: Again: massive server overload with J 3.9.1
Post by: AH on December 04, 2018, 17:01:31 PM
Good
Title: Re: Again: massive server overload with J 3.9.1
Post by: Studio 42 on December 04, 2018, 17:23:42 PM
Some .htaccess rule can bad redirect too. So check your rewriting rules.
Another problem is when you hardcode the http links and have rewrite rules to https, Joomla can use and rewrite relative links not absolute links. If you have image links that are not relative, the problem is same.
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 04, 2018, 17:35:16 PM
Hi Studio42,

we'll check that. Thanks for this precious hint! Although, I'm still wondering why we have the issue after updating to J3.9.1 and even more after updating VM to the beta version.

Cheers

Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 05, 2018, 10:36:06 AM
So we tried everything that was proposed so far. We could track down the problem a little:
Google indexes more than 1.4 URLs of the site www.kauer.ch - every second, a few thousands more. The URLs in Google look something like:

https://www.kauer.ch/index.php/de/produktsuche-ergebnisse/bett__or__gartenstuhl__or__gastrolounge__or__kommoden__or__buerotisch__or__regale2013-03-07-14-06-13__or__beistelltische__or__badmoebel__or__nachttische__or__duvet-kissen__or__test__or__schaffner-gartenm%c3%b6bel/

Now the weird thing: when you modify one of the words or numbers, there is no error page, but simply a new listing. But when one modifies the URL of non-shop-related addresses, then we have a correct 404 page.

Help... the server consumes actually the capacity of around 300 webspaces...

Thanks for every hint!

Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: AH on December 05, 2018, 10:59:46 AM
Even very badly worded "product url" with a valid category urls give a result

example:-
https://www.kauer.ch/index.php/de/produktsuche-ergebnisse/bad-url-from-me (https://www.kauer.ch/index.php/de/produktsuche-ergebnisse/bad-url-from-me)

Network tools:
Shows your site gives a 200 response to this url
However - there is also NO "canonical url" in the <Head> - so google will potentially index this url and keep visiting it



Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 05, 2018, 11:14:58 AM
Hi AH,

thanks so far. So what do you propose to put in the head of index.php?
Title: Re: Again: massive server overload with J 3.9.1
Post by: GJC Web Design on December 05, 2018, 11:19:47 AM
Have u disabled the VM 404 error handling?  this is in the VM config

IMHO non existing urls should give a 404 response otherwise there is nothing to tell Google not to index them
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 05, 2018, 11:23:13 AM
Yes, the 404 error handling is disabled (see previous post in this thread).
Title: Re: Again: massive server overload with J 3.9.1
Post by: GJC Web Design on December 05, 2018, 11:25:21 AM
try to track down then why no 404 is given .. try without sef and use the full VM url with a non existent product id etc .. what happens?

Is it VM, Joomla or at server level?
Title: Re: Again: massive server overload with J 3.9.1
Post by: jenkinhill on December 05, 2018, 11:42:45 AM
Canonicals are generated in view.html.php for products, categories etc - last year I fixed a site where the template developer had hacked the view.html.php files (not recommended!) and for some reason the canonical code was removed.
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 05, 2018, 12:47:21 PM
Hi jenkinhill,

Now THAT's a hint! Because I had to modify view.html.php (adding a module), I probably screwed up the code. Now I renamed it to NEWview.html.php in the html directory and we'll see what happens.

Thanks a lot!

Martin
Title: Re: Again: massive server overload with J 3.9.1
Post by: Milbo on December 06, 2018, 00:03:05 AM
Why did you change the view.html.php to add a module?
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 06:01:32 AM
Hi Milbo,

the client wanted a half-width slider for just one category at the end of the shop entry page. The other half should have been filled with another custom module. Therefore I added these in the view.html.php. This worked with no problem, but the slider module itself didn't do the job as expected. So we're back to normal with this page.

But the problem with the overload still persists. We're really, really at the end of our wisdom.
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 07:35:51 AM
Hi again,

since the 3.4.3 update, there is another problem, not related (obviously) to the overload: The product modules are going crazy in the backend (see attachment). They cannot be modified anymore. Hope there don't come out more such surprises with the beta version...
What can we do in the meantime?
Title: Re: Again: massive server overload with J 3.9.1
Post by: GJC Web Design on December 06, 2018, 10:18:57 AM
switch on Joomla debug and full reporting so u can see WHERE VMlang is missing

AIO updated?
no module over ride etc?
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 12:06:12 PM
Hi,

I had installed the complete beta package. But when I try to update the AIO database, I get an empty page in return. And: there are no modules overridden. The problem still persists: VM-modules cannot be modified in the BE (see previous screenshot)
Title: Re: Again: massive server overload with J 3.9.1
Post by: GJC Web Design on December 06, 2018, 12:15:56 PM
Quoteswitch on Joomla debug and full reporting so u can see WHERE VMlang is missing
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 12:22:42 PM
Ah - sorry!!!

0 Class 'vmLanguage' not found
/home/kauer.ch/ftp/www/administrator/components/com_virtuemart/fields/vmloadlang.php:39
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 12:27:04 PM
And another weird thing is: When going to the VM category in BE, I get the error message that the images couldn't be found here:
/home/kauer.ch/ftp/www/images/stories/virtuemart/product/images/stories/virtuemart/category/Tisch.jpg

but there is no images/stories...etc in product/.

The requestet image is in:
/home/kauer.ch/ftp/www/images/stories/virtuemart/category
Title: Re: Again: massive server overload with J 3.9.1
Post by: GJC Web Design on December 06, 2018, 12:50:43 PM
have you resaved your config page since updating?
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 12:53:14 PM
Now I just updated the 3.4.3 to yesterday's version... and at least the module part works now.
Title: Re: Again: massive server overload with J 3.9.1
Post by: Jörgen on December 06, 2018, 12:59:35 PM
This seems to be a category description image.
Have you checked the category Tisch ? Maybe there are wrong URLs.

Jörgen @ Kreativ Fotografi
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 13:16:21 PM
I checked them all; they were in the right directory. But what I did now: I deleted them all in VM, because they were not needed in the live shop. So maybe there was a context with the server overload due to not finding these images. Strange enough, that this came up after having updated J to 3.9.1 and VM to 3.4.3.
Title: Re: Again: massive server overload with J 3.9.1
Post by: Jörgen on December 06, 2018, 14:00:01 PM
I mean, have You checked the category ? Is there still a media item that points to this strange URL ?

regards

Jörgen @ Kreativ Fotografi
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 14:11:04 PM
No, I cleaned all categories from their images. They weren't needed anyway.
Title: Re: Again: massive server overload with J 3.9.1
Post by: Studio 42 on December 06, 2018, 15:54:21 PM
you category and product path is the same !
Perhaps the problem is from here.
Using same path mean that when you clear a Product media file, you delete same media for Category too.
At end you can have plenty errors, on removing media files
Title: Re: Again: massive server overload with J 3.9.1
Post by: marscom on December 06, 2018, 16:53:25 PM
I see and I'll change that. But this was actually never altered since the shop run on VM2... But anyway: I'll change it...
Title: Re: Again: massive server overload with J 3.9.1
Post by: Milbo on December 10, 2018, 10:26:17 AM
fixed in version 9993