Performance grinds to a near halt when add product categories!?

[xocigars]

Still using VM 3.0.9 (Joomla 3.4.1, PHP 5.4.45, DB: 5.5.51-38.2) basically, because have been debugging since I installed it (..lol we're all tryin')

So, I have a menu plug-in called juxmegamenu (default settings)...not sure if it is the culprit.

To-date, I have been using a smallish subset of virtuemart categories.
Now, when I populate all my categories, getting ready for "go-live", there are currently ~50,000 entries in vm_product_categories (which I expected, products may be in many categories).

Now, it takes approximatey 10-15 seconds to load any page! - including the home page, without products! (so, it might be that juxmegamenu thing?). Best menu I could find and too late to go back right now, as, I am trying to go live! Seems linear, in that, the more product-categories, the slower it gets.

The SHOP menu only has a max of 2 deep and very straight-forward. I have double-checked the db and it is fast. All indexes are working fine.

When I delete the four (4) category tables, the site is lightning fast.
  delete from xo_virtuemart_product_categories;
  delete from xo_virtuemart_category_categories;
  delete from xo_virtuemart_categories_en_gb;
  delete from xo_virtuemart_categories;

I did a chrome dev tools timing and there are what appear to be excessively high idle times like 4 seconds 6-8 total... I looked at all the sql running it is fine (I am a db/sql guy and 'C' programmer.. this Php stuff weirds me out . I have no idea how to find out what this "show-stopping bottle-neck" is.. I need a tool like: --> this is the slow script because... sure
Anyway, this seems like should be so easy to track, and maybe it is.. I hope. Could someone please shed some light on what possibly could be doing this?

regards,
Steve

[Milbo]

I think the problem is the menu. How does the menu work? It maybe even enough just to cache it. You can enable the vmdebug and set vmTime to measure the time. FPScherer meant sometimes ago, VM works with a lot categories, but not the joomla menu.

[xocigars]

Thank-you Milbo for your reply.

The category (shop) menu is a 'JUX megamenu' module. Is there a way to cache the whole module? (have never cached anything manually before)

Yes, perhaps the menu is loading every page refresh, but, I wouldn't think the menu is bringing back much data.. a few k.
Not sure where the contention is..

[jenkinhill]

Still using VM 3.0.9 (Joomla 3.4.1, PHP 5.4.45, DB: 5.5.51-38.2) basically, because have been debugging since I installed it (..lol we're all tryin')

See http://forum.virtuemart.net/index.php?topic=118683.0

[xocigars]

As I inferred, I will upgrade versions as soon as I get my site running somewhere close to bug-free, but, thanks in the interim to point out my insecure versions preesh lol.

[Milbo]

"I will upgrade versions as soon as I get my site running somewhere close to bug-free,"

You need to updat to get it bugfree = = = >  < = = =

[Ghost]

Thank-you Milbo for your reply.

The category (shop) menu is a 'JUX megamenu' module. Is there a way to cache the whole module? (have never cached anything manually before)

Yes, perhaps the menu is loading every page refresh, but, I wouldn't think the menu is bringing back much data.. a few k.
Not sure where the contention is..

Have you checked Debug information already or not? If not, do it now. Enable Debug in global configuration and configure Debug plugin to show profiling and query information.

[xocigars]

Setting System - Debug on and going to the site causes a 500 Server error...

tried all different cache settings still 500..

Turned log everything on Looked at all the logs I could find ..and basically only found this:

#
#<?php die('Forbidden.'); ?>
#Date: 2017-03-22 21:04:15 UTC
#Software: Joomla Platform 13.1.0 Stable [ Curiosity ] 24-Apr-2013 00:00 GMT

#Fields: datetime   priority clientip   category   message
2017-03-22T21:04:15+00:00   INFO 70.126.147.173   controller   Holding edit ID com_plugins.edit.plugin.425 Array (     

• => 425 )
  2017-03-22T21:04:16+00:00   INFO 70.126.147.173   controller   Checking edit ID com_plugins.edit.plugin.425: 1 Array (     
• => 425 )
  2017-03-22T21:04:51+00:00   INFO 70.126.147.173   controller   Releasing edit ID com_plugins.edit.plugin.425 Array ( )
  2017-03-22T21:07:40+00:00   INFO 70.126.147.173   controller   Holding edit ID com_plugins.edit.plugin.425 Array (     
• => 425 )
  2017-03-22T21:07:41+00:00   INFO 70.126.147.173   controller   Checking edit ID com_plugins.edit.plugin.425: 1 Array (     
• => 425 )
  2017-03-22T21:08:30+00:00   INFO 70.126.147.173   controller   Holding edit ID com_plugins.edit.plugin.425 Array (     
• => 425 )
  2017-03-22T21:08:30+00:00   INFO 70.126.147.173   controller   Checking edit ID com_plugins.edit.plugin.425: 1 Array (     
• => 425 )
  

[xocigars]

Here is the console before the crash (attached .jpg)...db looks fine..

thx - Steve

[xocigars]

This appears to be the last query that ran before the 500 error: (the timing was okay but sure is ugly)
UPDATE `xo_session`

  SET `data` = '__default|a:9:{s:15:\"session.counter\";i:22;s:19:\"session.timer.start\";i:1490292282;s:18:\"session.timer.last\";i:1490293810;s:17:\"session.timer.now\";i:1490293810;s:24:\"session.client.forwarded\";s:14:\"70.126.147.173\";s:22:\"session.client.browser\";s:68:\"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko\";s:8:\"registry\";O:24:\"Joomla\\Registry\\Registry\":2:{s:7:\"\\0\\0\\0data\";O:8:\"stdClass\":6:{s:11:\"application\";O:8:\"stdClass\":1:{s:4:\"lang\";s:5:\"en-GB\";}s:13:\"com_installer\";O:8:\"stdClass\":2:{s:7:\"message\";s:0:\"\";s:17:\"extension_message\";s:0:\"\";}s:10:\"com_config\";O:8:\"stdClass\":1:{s:6:\"config\";O:8:\"stdClass\":1:{s:6:\"global\";O:8:\"stdClass\":1:{s:4:\"data\";a:92:{s:10:\"MetaAuthor\";s:1:\"1\";s:8:\"MetaDesc\";s:0:\"\";s:8:\"MetaKeys\";s:0:\"\";s:10:\"MetaRights\";s:0:\"\";s:9:\"MetaTitle\";s:1:\"1\";s:11:\"MetaVersion\";s:1:\"0\";s:6:\"access\";s:1:\"1\";s:13:\"cache_handler\";s:4:\"file\";s:9:\"cachetime\";s:2:\"15\";s:7:\"caching\";s:1:\"0\";s:7:\"captcha\";s:1:\"0\";s:13:\"cookie_domain\";s:0:\"\";s:11:\"cookie_path\";s:0:\"\";s:2:\"db\";s:15:\"xocigars_joomla\";s:8:\"dbprefix\";s:3:\"xo_\";s:6:\"dbtype\";s:6:\"mysqli\";s:5:\"debug\";s:1:\"1\";s:10:\"debug_lang\";s:1:\"0\";s:23:\"display_offline_message\";s:1:\"1\";s:6:\"editor\";s:7:\"tinymce\";s:15:\"error_reporting\";s:4:\"none\";s:10:\"feed_email\";s:6:\"author\";s:10:\"feed_limit\";s:2:\"10\";s:9:\"force_ssl\";s:1:\"0\";s:8:\"fromname\";s:8:\"xoCIGARS\";s:10:\"ftp_enable\";s:1:\"0\";s:8:\"ftp_host\";s:0:\"\";s:8:\"ftp_pass\";s:0:\"\";s:8:\"ftp_port\";s:2:\"21\";s:8:\"ftp_root\";s:0:\"\";s:8:\"ftp_user\";s:0:\"\";s:4:\"gzip\";s:1:\"0\";s:7:\"helpurl\";s:90:\"https://help.joomla.org/proxy/index.php?option=com_help&keyref=Help{major}{minor}:{keyref}\";s:4:\"host\";s:9:\"localhost\";s:8:\"lifetime\";s:2:\"15\";s:10:\"list_limit\";s:2:\"20\";s:9:\"live_site\";s:24:\"http://www.xocigars.com/\";s:8:\"log_path\";s:30:\"/home/xocigars/public_html/log\";s:6:\"mailer\";s:4:\"smtp\";s:8:\"mailfrom\";s:23:\"aficionado@xocigars.com\";s:17:\"memcache_compress\";s:1:\"0\";s:16:\"memcache_persist\";s:1:\"1\";s:20:\"memcache_server_host\";s:9:\"localhost\";s:20:\"memcache_server_port\";s:5:\"11211\";s:7:\"offline\";s:1:\"1\";s:13:\"offline_image\";s:34:\"images/xo_logo_700x625-600x535.jpg\";s:15:\"offline_message\";s:81:\"Welcome to xoCIGARS!\r\nThis site will be live soon.<br /> Please check back again.\";s:6:\"offset\";s:3:\"UTC\";s:11:\"offset_user\";s:3:\"UTC\";s:8:\"password\";s:9:\"Delete321\";s:6:\"robots\";s:0:\"\";s:6:\"secret\";s:32:\"9rlMCVNprqrjTsLGq8hqXU3uHPgPyiWz\";s:3:\"sef\";s:1:\"1\";s:11:\"sef_rewrite\";s:1:\"0\";s:10:\"sef_suffix\";s:1:\"0\";s:8:\"sendmail\";s:18:\"/usr/sbin/sendmail\";s:15:\"session_handler\";s:8:\"database\";s:8:\"sitename\";s:8:\"xoCIGARS\";s:19:\"sitename_pagetitles\";s:1:\"1\";s:8:\"smtpauth\";s:1:\"1\";s:8:\"smtphost\";s:20:\"smtp.mandrillapp.com\";s:8:\"smtppass\";s:22:\"z9vHmzx0BGLo2eXBbMGUSA\";s:8:\"smtpport\";s:3:\"587\";s:10:\"smtpsecure\";s:4:\"none\";s:8:\"smtpuser\";s:18:\"admin@xocigars.com\";s:8:\"tmp_path\";s:30:\"/home/xocigars/public_html/tmp\";s:12:\"unicodeslugs\";s:1:\"0\";s:4:\"user\";s:14:\"xocigars_admin\";s:10:\"mailonline\";s:1:\"1\";s:17:\"memcached_persist\";s:1:\"1\";s:18:\"memcached_compress\";s:1:\"0\";s:21:\"memcached_server_host\";s:9:\"localhost\";s:21:\"memcached_server_port\";s:5:\"11211\";s:13:\"redis_persist\";s:1:\"1\";s:17:\"redis_server_host\";s:9:\"localhost\";s:17:\"redis_server_port\";s:4:\"6379\";s:17:\"redis_server_auth\";s:0:\"\";s:15:\"redis_server_db\";s:1:\"0\";s:12:\"proxy_enable\";s:1:\"0\";s:10:\"proxy_host\";s:0:\"\";s:10:\"proxy_port\";s:0:\"\";s:10:\"proxy_user\";s:0:\"\";s:10:\"proxy_pass\";s:0:\"\";s:11:\"massmailoff\";s:1:\"0\";s:28:\"session_memcache_server_host\";s:9:\"localhost\";s:28:\"session_memcache_server_port\";s:5:\"11211\";s:29:\"session_memcached_server_host\";s:9:\"localhost\";s:29:\"session_memcached_server_port\";s:5:\"11211\";s:12:\"frontediting\";s:1:\"1\";s:8:\"asset_id\";i:1;s:7:\"filters\";a:11:{i:1;a:3:{s:11:\"filter_type\";s:2:\"NH\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:13;a:3:{s:11:\"filter_type\";s:2:\"BL\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:6;a:3:{s:11:\"filter_type\";s:2:\"BL\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:7;a:3:{s:11:\"filter_type\";s:4:\"NONE\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:2;a:3:{s:11:\"filter_type\";s:2:\"NH\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:3;a:3:{s:11:\"filter_type\";s:2:\"BL\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:4;a:3:{s:11:\"filter_type\";s:2:\"BL\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:5;a:3:{s:11:\"filter_type\";s:2:\"BL\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:10;a:3:{s:11:\"filter_type\";s:2:\"BL\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:12;a:3:{s:11:\"filter_type\";s:2:\"BL\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}i:8;a:3:{s:11:\"filter_type\";s:4:\"NONE\";s:11:\"filter_tags\";s:0:\"\";s:17:\"filter_attributes\";s:0:\"\";}}s:5:\"rules\";a:10:{s:15:\"core.login.site\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"1\";i:7;s:0:\"\";i:2;s:1:\"1\";i:3;s:0:\"\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:16:\"core.login.admin\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"0\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:0:\"\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:18:\"core.login.offline\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"1\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:0:\"\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:10:\"core.admin\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:0:\"\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:0:\"\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:1:\"1\";}s:11:\"core.manage\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:0:\"\";i:7;s:1:\"1\";i:2;s:0:\"\";i:3;s:0:\"\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:11:\"core.create\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"1\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:1:\"1\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:11:\"core.delete\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"1\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:0:\"\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:9:\"core.edit\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"1\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:0:\"\";i:4;s:1:\"1\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:15:\"core.edit.state\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"1\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:0:\"\";i:4;s:0:\"\";i:5;s:1:\"1\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}s:13:\"core.edit.own\";a:11:{i:1;s:0:\"\";i:13;s:0:\"\";i:6;s:1:\"1\";i:7;s:0:\"\";i:2;s:0:\"\";i:3;s:1:\"1\";i:4;s:0:\"\";i:5;s:0:\"\";i:10;s:0:\"\";i:12;s:0:\"\";i:8;s:0:\"\";}}}}}}s:9:\"com_cache\";O:8:\"stdClass\":1:{s:5:\"cache\";O:8:\"stdClass\":4:{s:8:\"ordercol\";s:5:\"
  group\";s:6:\"filter\";O:8:\"stdClass\":1:{s:9:\"client_id\";i:0;}s:9:\"orderdirn\";s:3:\"asc\";s:10:\"limitstart\";s:1:\"0\";}}s:11:\"com_plugins\";O:8:\"stdClass\":2:{s:7:\"plugins\";O:8:\"stdClass\":4:{s:6:\"filter\";O:8:\"stdClass\":4:{s:6:\"search\";s:5:\"debug\";s:6:\"access\";i:0;s:7:\"enabled\";s:0:\"\";s:6:\"folder\";s:0:\"\";}s:8:\"ordercol\";s:6:\"folder\";s:9:\"orderdirn\";s:3:\"asc\";s:10:\"limitstart\";i:0;}s:4:\"edit\";O:8:\"stdClass\":1:{s:6:\"plugin\";O:8:\"stdClass\":2:{s:2:\"id\";a:1:{i:0;i:425;}s:4:\"data\";N;}}}s:6:\"global\";O:8:\"stdClass\":1:{s:4:\"list\";O:8:\"stdClass\":1:{s:5:\"
  limit\";i:20;}}}s:9:\"separator\";s:1:\".\";}s:4:\"user\";O:5:\"JUser\":28:{s:9:\"\\0\\0\\0isRoot\";b:1;s:2:\"id\";s:3:\"764\";s:4:\"name\";s:10:\"Aficio\";s:8:\"username\";s:5:\"admin\";s:5:\"email\";s:23:\"aficio\";s:8:\"password\";s:60:\"$2y$10$XKkcTNjL8rxgXr16YVPOr6BnADQpv96dW3tDUAjycpjJ1ycrmi2\";s:14:\"password_clear\";s:0:\"\";s:5:\"block\";s:1:\"0\";s:9:\"sendEmail\";s:1:\"1\";s:12:\"registerDate\";s:19:\"2015-02-11 17:41:51\";s:13:\"lastvisitDate\";s:19:\"2017-03-23 13:31:18\";s:10:\"activation\";s:1:\"0\";s:6:\"params\";s:92:\"{\"admin_style\":\"\",\"admin_language\":\"\",\"language\":\"\",\"editor\":\"\",\"helpsite\":\"\",\"timezone\":\"\"}\";s:6:\"groups\";a:1:{i:8;s:1:\"8\";}s:5:\"guest\";i:0;s:13:\"lastResetTime\";s:19:\"0000-00-00 00:00:00\";s:10:\"resetCount\";s:1:\"0\";s:12:\"requireReset\";s:1:\"0\";s:10:\"\\0\\0\\0_params\";O:24:\"Joomla\\Registry\\Registry\":2:{s:7:\"\\0\\0\\0data\";O:8:\"stdClass\":6:{s:11:\"admin_style\";s:0:\"\";s:14:\"admin_language\";s:0:\"\";s:8:\"language\";s:0:\"\";s:6:\"editor\";s:0:\"\";s:8:\"helpsite\";s:0:\"\";s:8:\"timezone\";s:0:\"\";}s:9:\"separator\";s:1:\".\";}s:14:\"\\0\\0\\0_authGroups\";a:2:{i:0;i:1;i:1;i:8;}s:14:\"\\0\\0\\0_authLevels\";a:5:{i:0;i:1;i:1;i:1;i:2;i:2;i:3;i:3;i:4;i:6;}s:15:\"\\0\\0\\0_authActions\";N;s:12:\"\\0\\0\\0_errorMsg\";N;s:13:\"\\0\\0\\0userHelper\";O:18:\"JUserWrapperHelper\":0:{}s:10:\"\\0\\0\\0_errors\";a:0:{}s:3:\"aid\";i:0;s:6:\"otpKey\";s:0:\"\";s:4:\"otep\";s:0:\"\";}s:13:\"session.token\";s:32:\"f7b4676b387b7a058187df5b5fa255d\";}'
   , `time` = '1490293810'

  WHERE `session_id` = 'a43de78552b1f51c9f4325821d6b5c'


So, now I'm debugging the debugger lol?

[xocigars]

..just a response to upgrading (I do though very much need your help to find out why 15 seconds to load home page or category-product search..any page.. when number of product-catefories increase.. ugh.. please).

I would love to upgrade but have made many customizations, and although, I have kept track of them all, I still need to get the customizations/extensions working first. Then, I can upgrade, apply the mods, successfully, and release the awesome customizations to fellow Joomla and Virtuemart enthusiasts..

regards,
Steve

[xocigars]

fyi.. GoDaddy looked at memory and I/O and said they are just fine..

[xocigars]

Have kept trying.. Just found this at the end of a debug log if this might mean anything to anybody?

WARNING - deprecated
JHtml::getJSObject is deprecated. Use json_encode instead.


WARNING - deprecated
JHtml::getJSObject is deprecated. Use json_encode instead.


DEBUG - databasequery
SHOW FULL COLUMNS FROM `xo_assets`


DEBUG - databasequery
SELECT id FROM xo_assets WHERE parent_id = 0


DEBUG - databasequery
SELECT b.rules FROM xo_assets AS a LEFT JOIN xo_assets AS b ON b.lft <= a.lft AND b.rgt >= a.rgt WHERE (a.id = 1) GROUP BY b.id, b.rules, b.lft ORDER BY b.lft


WARNING - deprecated
JApplication is deprecated.


DEBUG - databasequery
SELECT * FROM xo_acymailing_config


WARNING - deprecated
JRequest is deprecated.


WARNING - deprecated
JResponse is deprecated.

regards,
Steve

[xocigars]

I turned on depracated logging..this cant be good....(please see attached)

thanks in advance for any and all support.
Steve

[jenkinhill]

Well JHtml::getJSObject is deprecated. Use json_encode instead. was a reported bug in Joomla 3.4.1
JRequest is deprecated and JApplication is deprecated  were errors from Joomla 3 days.
not sure about the others, but they are consequences/legacies of old versions, deprecated logging is much better handled in Joomla 3.6.5