VM1 to VM2: Missing Category Images after Migration

[mbarry]

VM 2.0.26d
Joomla 2.5.17

Scenario:
      Run the VM2 Migration tool with "everything" option selected".
      Each time it reports "Migration was interrupted by max_execution time, please restart"
      Rerun the migration until it reports " Migration finished"

Issue:
      View site product category pages; under some circumstances some of the images may appear to be randomly missing.
      An inspection of the old vm tables and image folders (images/stories/virtuemart/...) will show that the category images do infact exist.
      An inspection of VM2 tables #__virtuemart_medias and #__virtuemart_category_medias will find that these particular media items do not exist.

     
      The problem is when $this->portMedia() is run as part of function migrateAllInOne() and fails to complete the migration of all the media in a single cycle.
      When it fails to complete the following migration activities may continue to be processed:
          $result = $this->portShoppergroups();
            $result = $this->portUsers();
            $result = $this->portVendor();
            $result = $this->portCategories();
            $result = $this->portManufacturerCategories();
            $result = $this->portManufacturers();
            $result = $this->portProducts();
            $result = $this->portOrders();

      How is this possible ....
     
      For each media type (Product, category, vendor, manufacturer and ForSale), portMedia calls _portMediaByType() which is currently allocated 40% of the total processing time
      available in a single migration run. If this limit is reached part way through the product or category image migration then portMedia will exit with only a fraction of the media migrated.
     
      There are three lines similar to the one below that will break _portMediaByType() out of its media collecting loops. After which it processes what it has collected and adds them to the database (#__virtuemart_medias).
     

Code Select Expand


            if((microtime(true)-$this->starttime) >= ($this->maxScriptTime*0.4))
     
     
      The problem occurs where there is still script execution time remaining noting that portMedia will exit when approximately 40% of the time has been exceeded.
      All of the subsequent migration processes identified above test for either:     
     

Code Select Expand


      if($this->_stop || (microtime(true)-$this->starttime) >= ($this->maxScriptTime)){
       return;
      }
     
           
      Or
     

Code Select Expand


      if((microtime(true)-$this->starttime) >= ($this->maxScriptTime)){
   return;
  }
     

      Where this is a problem is that none of the above tests will stop further processing and in particular, portVendor, portCategories, portProducts, etc will be attempting to migrate image references using 'virtuemart_media_id'. If portMedia has not finished, then chances are the other processes will fail to find all the associated media. Subsequent runs will not resolve this since the categories, and products have now been processed and will be identified as having already been migrated.
     
      This problem only started occurring for me when I upgraded XAMPP to 1.8.3 (php 5.5.6, sql 5.6.14, windows 7). It runs slower than earlier versions and hence, I started seeing random missing images on the site. Prior to my upgrade i could complete the whole migration run in a single 360 sec window. I have been looking at the localhost issues but still doesn't seem to solve all the performance problems.
           
Solution
    My strategy for fixing this problem was to cease any further processing whenever a script timeout was reached. To achieve this:
    1. always set $this->_stop = true on a script timeout
   
    eg.
   

Code Select Expand


          if((microtime(true)-$this->starttime) >= ($this->maxScriptTime*0.4)){
  $this->_stop = true; //once the allocation of time has been exhausted no point in processing any further
  break;
  }
   
   
    or more generically
   

Code Select Expand


          if((microtime(true)-$this->starttime) >= ($this->maxScriptTime)){
  $this->_stop = true; //once the allocation of time has been exhausted no point in processing any further
  break;
  }
   
   
    Also anywhere there is an error that is expected to prevent further processing I would set $this->_stop = true; to avoid any side-effects.
   
   
    2. on entry into a port process, always check for $this->_stop or script timeout.
      eg.
   

Code Select Expand


         if($this->_stop || (microtime(true)-$this->starttime) >= ($this->maxScriptTime)){
     return false;
    }
   
   
    Putting these into the migration process will ensure reliable and consistent migration results

   
Affected files:
      administrator/components/com_virtuemart/helpers/migrator.php