VM1 to VM2: Not all Products are Migrated

[mbarry]

VM 2.0.26d
Joomla 2.5.17

Scenario:
      Run the VM2 Migration tool with "everything" selected".

      A typical output of the Product Migration may look something like this for a VM1 database of less than 2000 products (In my case I have 1367 products)
     
        port Products take 1000 vm1 entries for migration  <- reports there are 1000 products to migrate on first cycle
        Migration: 0 products processed                    <- reports 0 migrated; this is a bug
        port Products take 367 vm1 entries for migration   <- reports there are 367 left to process, so we know 1000 products were processed
        Migration: 0 products processed                    <- again reports 0 migrated
       
Issue:
      I had noticed on previous migration attempts that some child products were missing. The problem seemed to be random as to which items were missing.
           
      An inspection of the migrated #__virtuemart_products Table it would only report 1303 products.
      The VM1 #__vm_product table reports 1367 products.     
   
      On inspection of the VM2 table #__virtuemart_migration_oldtonew_ids which stores the migration progress results, I noticed that `products_start` had a value of 2303.
      It should have been reporting 1303 in order for the remaining products to be processed.
     
      The observed results of the product migration will appear to be very random and is dependent on the amount of processing time allocated.
      Should the Migration fail to complete all product migration in a single run, then this issue will most definitely prevent any further migration of products or worse it will skip
      products by some variable amount each time the migration is run.
 
     
Solution:

      It turns out to be a very simple yet significant fix in migrator.php

      On each cycle of the while loop, $doneStart is assigned the current $startLimit. On the first cycle this is 0, second 1000, third 2000 and so on as it batch processes Products in groups of 1000.
       For correct operation, the index $j must be reset on each cycle of the while loop such that the $limitStartToStore = $doneStart + $j gives the total products processed.
     
      Currently, index $j is never reset and hence $limitStartToStore will be assigned an incorrect value on exit from the while loop.


      administrator/components/com_virtuemart/helpers/migrator.php           
     

Code Select Expand


$startLimit = $this->_getStartLimit('products_start');;
//$j=0;  // <- Comment out this line
     
     
      and move it to just be before the foreach loop
     

Code Select Expand


$j=0; // <- move it here
foreach($oldProducts as $product){

      if(!empty($product['product_id']) and !array_key_exists($product['product_id'],$alreadyKnownIds)){
     
     
     
      Fix up the error reporting by changing the $i index to $j     
     

Code Select Expand


$errors = $productModel->getErrors();
if(!empty($errors)){
     foreach($errors as $error){
           vmError('Migration: '.$j.' ' . $error);  // <- Change to $j
    }
    vmdebug('Product add error',$product);

         
      Fix up the process reporting by changing the $i index to $j   
     

Code Select Expand


            $limitStartToStore = ', products_start = "'.($doneStart+$j).'" ';
            $this->storeMigrationProgress('products',$alreadyKnownIds,$limitStartToStore);
            vmInfo('Migration: '.$j.' products processed ');  // <- change to $j