About adding more than 100.000 products

[gboul]

We developed a bookstore site with nearly 4000 digital books (J 2.5.4 - VM 2.0.6).
We are planning to add paper books as well.
In the next couple months we will add about 100.000 paper books.

Does anyone has the a site with a similar number of products?

I am looking for any issues that might arise with such number of products.

Any input will be highly appriciated.

[Apemb5]

Great efforts! you can search with sites like Amazon to add your targeted books!

[bitstomper]

I have a site with about 8500 items. Not as many as yours, but still.
Some tips:
Good you use VM 2, VM1 is a lot slower with large sites.
I used a caching plugin to make the site faster, but it stored cached versions of visited pages and there were so many that it took ages for every visit to run through all the cached version to see if it had a cached version. This makes these kind of plugins totally useless.

So I suggest you dissect the categories as much as possible. It is good to have choice. It is not good to have too many
Use Amazon as an example. They have many books on sale, but you do not get overwhelmed.
That way you can load less books on one page and minimize the load time.

Make sure you host on a local and fast server.
If most of your audience is in the states than host your site on a server there.
Believe me, it does make a big difference with large databases.

VM is excellent for use with large sites. Esp. VM2.

[gboul]

bitstomper  thank you for your responce. We already have problems with the cache. We found out the some views are cached, where others are not. On the cached views, the cart and the wishlist extension that we use, they do not work as expected.

Yesterday we started loading the first data. We loaded all categories (about 4700 categories), all the authors (around 68000) and all the editors (around 3500) and around 10000 books to start the development.
The immediate problems started with the size of the memory allocation in the PHP. We had to increase it to 512MB to be able to load the administrator page.
Then we noticed that the loading of the product page and the product details page in the administrator it was slow. I think the main issue is that in both pages we load all the categories and all the manufactures. We are planning to change the administrator pages to use AJAX search to filter by category and by manufactures instead of the dropdowns.

I will keep you posted on the developement.

[Milbo]

I just got an email by a guy with a bookstore and more than 170k products. Seems to run. And btw, of course the cache works only when you call the page the second time. That is the way a cache works. Use the different caching strategies, when you get problmes (only in j2.5.x available)

[gboul]

Milbo,
It is nice to here that there is a already a shop out there with a big number of products.

Concerning the cache. I do know how cache is working. We had and we have issues with the cache, others are virtuemart related and others are joomla in general (I have posted one or two regarding these in the past).
The majors ones that we have at the moment are: a) the virtumart frontpage (view) is not cached and b) for some modules that they do not use cache, their javascript injections are been replicated (I posted this on the joomla forum, but still no luck).
This is a critical issue for us since we are looking up to 400 current users.
This is something I would be looking into it at the next days. If anyone has any ideas that he can kindly share, it would be highly appreciated.

Also, we are looking for a consultant to help us with the caching and the performance issues that we are currently dealing. Anyone interested, please feel free to contact me.

[amorino]

Hello,
I have a vm installation with 3500 products
My hoster contact me to tell me that  I should split/move the files within the /home/******/public_html/components/com_virtuemart/shop_image/product folder because this is a direct violation of our RUP.

"A directory can not contain more than 2,500 immediate child files. This includes subdirectories themselves, but does not include files contained within those directories."

Otherwise they will close my site any help please to split the directory?

VM 1.1.x and joomla 1.5.26
Best regards
Amorino