VirtueMart Forum

VirtueMart 2 + 3 + 4 => General Questions => Topic started by: gba on December 05, 2013, 10:54:14 AM

Title: [solved] Import 100000 products with images
Post by: gba on December 05, 2013, 10:54:14 AM
Dear all,

I am building a web shop that shall contain product data originating from an external source.
The amount of products to be imported once a day will be about 100000.
The images to be imported are stored on an external server and the import file (CSV) contains the URL of the image, only.
The import shall start at about 5:30am and should be finished within one hour.

What is the most effective and secure way to get so much product data into VM2?

Thank you very much in advance for any help!

Best regards,
Gerald
Title: Re: Import 100000 products with images
Post by: 911websiterepair on December 05, 2013, 16:29:43 PM
we could help you with this as it is a little more complicated as you may think

you use http://www.csvimproved.com/overview

and you set the import to run as a cron

but you have to format the import file so it has corresponding fields in virtuemart so it knows what to populate. this is the only complication

as for time, there are a lot of factors that will go into this, but you must have a persistent connection in order for this work
Title: Re: Import 100000 products with images
Post by: Milbo on December 05, 2013, 18:21:43 PM
The best way is to write a real script for that. Franzpeter is running his stores that way.
Title: Re: Import 100000 products with images
Post by: 911websiterepair on December 06, 2013, 15:42:11 PM
that is correct          it is not as simple as just doing an import

we specialize in this
Title: Re: Import 100000 products with images
Post by: gba on December 17, 2013, 16:56:06 PM
Hello!

Thank you for your replies!
I am already using CSVI.
But it is way too slow, although image processing is not necessary anymore - I display images using the URL of the originals.

Best regards,
Gerald
Title: Re: Import 100000 products with images
Post by: gba on February 02, 2015, 13:43:01 PM
Hi all!

I sticked to CSVI and could contribute to big improvements concerning processing speed import a large amount of product data.  ;)
Meanwhile CSVI 6 is available and way more performant than ever before.
By the way: It already supports VM3!

Best regards,
Gerald