All I have been asking for a few months now I have a client with 12k products. I have copied the old tables to my new extension and ran the migration. My categories appear to be created by my products are not migrating at one point I got 1000 products migrated but couldn't get past that number no matter how many times I keep running the migration.
I have my server settings set to
execution 300
memory 124 mb
I really need help to get this client moved I have exhausted all my options.
Thanks
Joe
what it is saying?
Doesn't say anything just never goes any further it originally stopped at 1007 products and no matter how many times I keep running migrate it never goes any higher.
I just blew everything away not I have ran the migration 5 times and only 24 products have migrated.
So it appears my categories all migrated which seems to go fine
Looks everything worked correct, migrated 0 categories
Looks everything worked correct, migrated 117 categories xref
Looks everything worked correct, migrated 0 manufacturer categories
Looks everything worked correct, migrated 0 manufacturers
But when I migrate products it is only migrating 5 at a time or nothing at all.
Found prior migration process, resume migration maxScriptTime 119 maxMemoryLimit 110
Worked on general migration for 233.932654142 seconds
1
Info: port Products take 5 vm1 entries for migration
Info: Migration: 0 products processed
The other messages I get Info: port Products take 5 vm1 entries for migration
Info: Coulndt find category for product, maybe just not in a category
Info: Coulndt find category for product, maybe just not in a category
Info: Coulndt find category for product, maybe just not in a category
Info: Coulndt find category for product, maybe just not in a category
Info: Coulndt find category for product, maybe just not in a category
Info: Migration: 5 products processed
Info: port Products take 5 vm1 entries for migration
Info: Coulndt find category for product, maybe just not in a category
Info: Migration: 6 products processed
are you migrating everything at once?
or doing different parts at a time?
Tried it every way. But have been trying 1 thing at a time. So only migrating products.
Joe
Quote from: Joe Bourque on March 12, 2013, 14:42:43 PM
Tried it every way. But have been trying 1 thing at a time. So only migrating products.
Joe
if you do it that way, you need the cats before the products
I did the catagories they have all been migrated but the products have not.
Joe
and you keep trying to migrate?
It tooks me many times, and i only have 400 products
Yes it only migrates 5 products but not everytime, only if I delete the migration data from the database. As I stated I have over 12,000 products and I can't sit here for days refreshing the data 5 at a time :)
Joe
In my tests it did 1000 products in one row, so there is obviously something wrong.
what is your exact vmart version you are trying to migrate to?
I'm using 2.18.
Guys I'm more than willing to let you guys take a look, I have been struggling with this for months and my client is really running out of patience.
Just an FYI the day before yesterday I actually uninstalled VM and deleted all the tables and tried it again with the same exact outcome. I have rebuilt the site 4 times with Jupgrade and all the same.
Thanks for your time.
Joe
Quote from: Joe Bourque on March 13, 2013, 14:45:07 PM
Guys I'm more than willing to let you guys take a look, I have been struggling with this for months and my client is really running out of patience.
Just an FYI the day before yesterday I actually uninstalled VM and deleted all the tables and tried it again with the same exact outcome. I have rebuilt the site 4 times with Jupgrade and all the same.
Thanks for your time.
Joe
test this on a clean Joomla install and Vmart and see what happens
I have installed a clean joomla install with Virtuemart and still no luck. I have about 2,000 images and 12,000 products. If anyone can give any advice or assistance with this issue please let me know.
Thanks
Joe
Try comment out line 1057 of the file administrator/components/com_virtuemart/helpers/migrator.php. See if this works better.
Anyway, 12000 products may be too heavy for the migrator script. You probably will encounter problems down the road unless you can increase the memories and script time.
I ran it the 1st time and 970 1001 products imported then I tried again a few times and it's only migrating 1 at a time :(
Info: port Products take 1000 vm1 entries for migration
Info: Coulndt find category for product, maybe just not in a category
Info: Migration: 1 products processed
Apparently, the migrator runs out of memory.
Ah... I forgot to mention you have to choose migrate products only. That will leave more memories for product migration. Did you try that?
Yes I'm migrating the products only. I have allocated 124mb of memory I can adjust it to 512M
The other strange part is I have 117 categories and the some of the products are not showing the category saying category not found although I see all the categories.
Joe
So I bumped the memory up to 512 still no luck
Start with a new migration process and setup log maxScriptTime 159 maxMemoryLimit 498
Worked on general migration for 229.350474119 seconds
Info: port Products take 1000 vm1 entries for migration
Info: Coulndt find category for product, maybe just not in a category
Info: Migration: 1 products processed
I looks like the migration does not continue I went into my sql db and deleted the data in virtuemart_migration_oldtonew_ids then rerun the migration script it imports 1000 more products which are all duplicates.
Please advise if anyone knows how to fix this
Joe
Did you have debug ON in VM configuration? There may be more error messages coming up if debug is ON.
Yes I did nothing else I can see
Joe Bourque,
what info do you need to be migrated?
the whole thing?
users? orders? products , cats?
Hellowz ! ^_^ !
Hi PRO, Hi Milbo, Hai Joe & Mr Kwan (ty for your work on 1.2b)
Having a similar problem here - product migration just stopped @2411 or something - would not migrate any further products.
Thanks Kwan, your comment out //$maxItems = 5; seemed to get things moving, but quite slowly - before commenting out maxitems the product migration was absolutely stuck migrating 0 products on each migration, sql process was using CPU though.
Now, with //$maxItems = 5; its processing products again, I'm going to up the max exec and mem limit again and hoping this will help process more products.
P.s 22, 000 products
Best Regards
RJ
**EDIT** 22,130 products migrated
Hi all,
I also was stuck on importing +26000 products. Kwan's solution to comment out $maxItems solved partially the problem. I had increased memory_size to 512MB and timeout to 600 but even then mysqld also was running wild.
Finally traced down the excessive cpu use to the LEFT JOIN SQL just below $maxItems in migrator.php which was deadslow.
The problem was the datatype of #__vm_product_mf_xref.product_id which had varchar(32) type. Changed this to INT(11) and from then on the LEFT JOIN was running smoothly.
Looking back it seems VARCHAR(32) was used in v1.1.2 and never altered to INT(11) since.
Hello,
i have the same problems in migrating... i use J 2.5.14 and VM 2.0.22a...
...first i tried to do this directly on server but because of low execution_time and memory_limit i couldn't migrate too many products and images (i have 12 000 products and 30 000 images)
... now im trying to migrate localy (nstaled xamp) and increased memory limit to 2000mb and execution_time to 12000s (2 hrs : )... i managed to migrate all images ok... but products again stoped at cca 7 000... and when i run migration start again... it migrates 0
... i gues i will just have to increase more execution_time (maybe to 5 hrs) in php.ini of server...and just let it process for hours (it seems if data migration is interupted, and it is not migrated in 1 try... later doesnt want to continue right...)
I will keep you informed about this...
Greetings,
Sasa
What migration method/manual do you use?
Hello...
I found interesting situation and cant figure it out... ( i have 12 000 products to migrate, but to debug it, first i used 16 to see what is wrong)
1. I enabled debug function in VM... and imported 16 products to my old VM 1.9 table vm_product
2. i migrated products and from 16 only 9 migrated ok... but in table _virtuemart_products, there are all 16 products migrated... but in my _virtuemart_products_hr_hr table there are only 9 (and only these show in administration)
....After migration I got this debug message (this is message just for one... rest are the same)
vmdebug Check returned false VmTableData #__virtuemart_products
vmdebug $product["virtuemart_product_id"] or $product["product_id"] is EMPTY? Var1:
Array
(
[product_id] => 13
[vendor_id] => 1
[product_parent_id] => 0
[product_sku] => CERNIT 200
[product_s_desc] => Blue
[product_desc] =>
[product_thumb_image] => resized/Cernit_Number_On_4aa8b2d661609_150x150.jpg
[product_full_image] => Cernit_Number_On_4aa8b2d68c16c.jpg
[product_publish] => Y
[product_weight] => 62.0000
[product_weight_uom] => G
[product_length] => 0.0000
[product_width] => 0.0000
[product_height] => 0.0000
[product_lwh_uom] => CM
[product_url] =>
[product_in_stock] => 24
[product_available_date] => 2009-08-30 00:00:00
[product_availability] =>
[product_special] => N
[product_discount_id] => 0
[ship_code_id] =>
[cdate] => 1251699144
[mdate] => 1353275956
[product_name] => Cernit Number One
[product_sales] => 1
[attribute] =>
[custom_attribute] =>
[product_tax_id] => 4
[product_unit] => komad
[product_packaging] => 0
[child_options] => N,N,N,N,N,Y,20%,10%,
[quantity_options] => none,0,0,1
[child_option_ids] =>
[product_order_levels] => 0,0
[manufacturer_id] => 1
[virtuemart_vendor_id] => 1
[categories] => Array
(
)
[published] => 1
[mprices] => Array
(
[product_price_id] => Array
(
[0] => 0
)
[product_id] => Array
(
[0] => 13
)
[product_price] => Array
(
[0] => 10.56098
)
[product_currency] => Array
(
[0] => 160
)
[price_quantity_start] => Array
(
[0] => 0
)
[price_quantity_end] => Array
(
[0] => 0
)
[product_price_publish_up] => Array
(
[0] => 0
)
[product_price_publish_down] => Array
(
[0] => 0
)
[created_on] => Array
(
[0] => 2009-08-31 06:12:24
)
[modified_on] => Array
(
[0] => 2012-11-18 21:59:16
)
)
[created_on] => 2009-08-31 06:12:24
[modified_on] => 2012-11-18 21:59:16
[virtuemart_media_id] => 2030
[virtuemart_product_id] =>
)
...here at the bottom [virtuemart_product_id] => there is no product id number, but i checked in database and in _virtuemart_products... there are virtemart_product_id exist for all products...
ANOTHER FUNNY THING:
At second attempt to test it... i used only those 7 products that havent't migrated at first try.... and here 6 products migrated ok, and one with the debug error mentioned above...
VM debug error that is apearing is connected with ...administrator/components/com_virtuemart/helpers/migrator.php file.... lines around 1236
if($this->_keepOldProductIds){
$product['virtuemart_product_id'] = $product['product_id'];
}
$product['virtuemart_product_id'] = $productModel->store($product);
if(!empty($product['product_id']) and !empty($product['virtuemart_product_id'])){
$alreadyKnownIds[$product['product_id']] = $product['virtuemart_product_id'];
} else {
vmdebug('$product["virtuemart_product_id"] or $product["product_id"] is EMPTY?',$product);
}
Im not a programmer, so I have no clue why this is happening...
If someone could help me with this, i would be very happy.
Thanks,
Sasa
Oh, come on. Just export everything with CSVI Improved and import this stuff then.
Hello,
I would do this migration with CSVI improved if you show me how :)
fields that i export with CSVI and VM 1.9, are completely different that CSVI for VM 2 uses for import... and i dont know how to manage that : (
Any help appreciated.
Thanks,
Sasa
Hey svarmuza,
Why don't you try something like cart2cart (http://www.shopping-crat-migration.com/supported-carts/127-virtuemart) ?
It allows to migrate data from one version to another easily.
Hello,
I finally migrated my products...and here is a procedure...maybe it helps somebody...
1. I migrated Categories, Media, Users, Orders... using built in migration tool in VM 2.XX
2. For products i managed to do it with CSVI IMPROVED component... this way
- 1. exported all products from VM 1.1.9 with built in EXPORT PRODUCTS TEMPLATE
- 2. in J 2.5 and VM 2.xx IMPORT PRODUCTS TEMPLATE...under fields i added all the same fields as they are in VM 1.9 CSV exported template (take notice that some fields in VM 2 are called differently)...drag and drop fields to be in order like in 1.9 CSV export template...
- 3 for images... mark YES to process images (all other stays default)
...and click process
I did this on localhost, so I increased memory limit, and execution times...and i had no trouble now to import 13.000 products (and it connected categories, prices, photos...)
...so happy now after ten days of nightmares...
Thanks,
Sasa
P.S. and thanks @dlassar for mentioning cart2cart ... i was about to use their service, but i hope now that i wont need it.