Not all new medias can be synchronized in VM admin panel

[borro]

Hello!

Our eshop contains 10011 photos of products in the /images/stories/virtuemart/product folder. Among 85 of them are new one which I'm trying to synchronize by pressing the "Synchronize media to Virtuemart" button in the Shop->Media Files section of admin panel. But after synchronization is over no one of these new uploaded by ftp files can be seen in the Media Files section of admin panel.

Everything worked fine a few days ago. It looks like there is some problem with image quantity

Why is it so?

Here is the list of messages that appears after the end of synchronization:

vmdebug PHP 5.4

vmdebug Show All Errors

vmdebug $siteLang: ru-RU self::$_jpConfig->lang ru_ru

vmdebug vmTime: time to load config: 0.00257992744445801

vmdebug Start used Ram 4.25M

Found prior migration process, resume migration maxScriptTime 47 maxMemoryLimit 114

Synchronized media for product in directory images/stories/virtuemart/product/

Synchronized media for category in directory images/stories/virtuemart/category/

Synchronized media for manufacturer in directory images/stories/virtuemart/manufacturer/

Synchronized media for vendor in directory images/stories/virtuemart/vendor/

Synchronized media for forSale in directory /var/www/antilopa/data/www/vmfiles/

vmdebug vmTime: VirtueMartControllerMedia Finished task media: 22.9149630069733

vmdebug End used Ram 73.5M

vmdebug Peak memory peak 74M

Synchronized alltogether files

vmdebug PHP 5.4

vmdebug Show All Errors

vmdebug $siteLang: ru-RU self::$_jpConfig->lang ru_ru

vmdebug vmTime: time to load config: 0.0024561882019043

vmdebug Start used Ram 4.25M

vmdebug vmTime: VirtueMartControllerMedia Finished task media: 0.0641698837280273

vmdebug End used Ram 6M

vmdebug Peak memory peak 6M

VM 3.0.9

[Jörgen]

Hello

9999 files in one directory seems to be a very common limit for many hosts. I have been forced to split my images into subdirectories products/lenshoods, /products/lensadapters and so on. The files are stored but when You look for them you will only see the first 9999 files. Check if You can see all Your files with a ftp manager. If You can´t You have to split the files in separate subdirectories. Managing files in subdirectories is better in my view anyhow. You host could probably raise this number to something more than 9999.

regards

Jörgen @ Kreativ Fotografi

[borro]

Hello

9999 files in one directory seems to be a very common limit for many hosts. I have been forced to split my images into subdirectories products/lenshoods, /products/lensadapters and so on. The files are stored but when You look for them you will only see the first 9999 files. Check if You can see all Your files with a ftp manager. If You can´t You have to split the files in separate subdirectories. Managing files in subdirectories is better in my view anyhow. You host could probably raise this number to something more than 9999.

regards

Jörgen @ Kreativ Fotografi

Thank you, Jörgen
I see all of 10011 files via FileZilla but it looks like sometimes there is some problems with displaying them and i have to press F5 to reload the folder content.
So you suggest to split files in 2 smaller subfolders and rerun Synchronization?

[borro]

Hello

Can you tell what the rights should be assigned to photos in /images/stories/virtuemart/product folder? I see that 5700 files have the 644 code and the others(about 4300) have the 775. Is it bad?

[Jörgen]

Hello again

Yes, I would break the images into subdirectories. If this isn´t something that your host could increase. But subdirectories is a good way to go.
I can´t explain why You have 755 and 644 in the same directory. For Joomla usually directories are more permissive, 755 and files less, 644. The files should really all have just 644.

First figure is for owner, second group and third other.
4 is read
5 is read/execute (execute is always needed for directories)
6 is read/write
7 is read, write, and execute

regards

Jörgen @ Kreativ Fotografi

[borro]

Hello again
Yes, I would break the images into subdirectories. If this isn´t something that your host could increase. But subdirectories is a good way to go.

Hello and thank you, Jörgen.
Do I have to change VM settings after creating subdirectories to let VM know where it should check a new files to synchronize medias? Do I have to point VM to a subfolder with only new images?

[Jörgen]

Hello borro

No I don´t Think so. But I have bad experiencies with the syncronize feature. It think it works without changing the paths. If I remember correctly changing the path ended up with multiple copies of the same image.

I do it this way, not saying You should do the same

I

• I upload the images to the subdirectories with ftp
• Create a pattern media item with the Product management and fill in all the details
• Use phpadmin to search for this media item and make a copy
• Take the copy and change the file URI and other data if neccessary

I get all the details copied correctly and simply change file name, URI and clear thumbnail field. When all media items have been copied with PhpAadmin I simply assign them to the product in the Product management view.
I repeat this for each Product that uses multiple images. This takes much less time than uploading the images with Product management.
regards.

Uploading images (medias) to VM is a pain. But I haven´t had any time to plan how it should be done. Even uploading a few images to a Product is hard work.

The simple task of assigning a image to a Product is cumbersome. If a write in a search pattern i quickly get the image I want. But when I select this image my search pattern is exchanged with the full path to the selected Picture. This means that I have to edit the search pattern every time I select a Picture. The one who designed this probably only selects one image per Product. In that case it would be ok. I could go on but this would not be constructive right now. But I don´t understand why the search pattern would be updated with the path of the last selected image. Not changing the search pattern after selecting an image would be great.

Jörgen @ Kreativ Fotografi

[borro]

• Use phpadmin to search for this media item and make a copy
• Take the copy and change the file URI and other data if neccessary

When all media items have been copied with PhpAadmin I simply assign them to the product in the Product management view.

Thank you.

I'm trying to understand what is the phpadmin. Is it something similar to PhpMyAdmin?

[Jörgen]

Sorry, Yes it is PhpMyAdmin I mean

Choosing appropriate filenames, for example jjc-lma-eos-nx-a.jpg, jjc-lma-eos-nx-b.jpg, jjc-lma-eos-nx-c.jpg, makes it easy to change the filename and URL (Products/lensdapter/jjc-lma-eos-nx-a,jpg) changing the a, to b and so on in the media item that I have copied.

regards

Jörgen @ Kreativ Fotografi

[borro]

I found out one solution but be careful you may be blocked by your hoster due to exceeding the limits of consumed resources. Use it at your own risk
To make VM 3 be able synchronize a huge massive of photos open the /administrator/components/com_virtuemart/helpers/migrator.php file
and comment these lines(around line 371 in a private function _portMediaByType code):

Code Select Expand

if((microtime(true)-$this->starttime) >= ($this->maxScriptTime*0.4)){
    break;
}


[borro]

I think it's better to run such sql script under the PhpMyAdmin:

Code Select Expand


INSERT INTO ytgb1_virtuemart_medias(file_title, file_meta, file_mimetype, file_type, file_url, file_url_thumb, published)
SELECT
i.file_url as file_title
, i.file_url as file_meta
, 'image/jpeg' as file_mimetype
, 'product' as file_type
, CONCAT('images/stories/virtuemart/product/',i.file_url) as file_url
, '' as file_url_thumb
, 1 as published
FROM bng347_urls_for_import i
WHERE
NOT EXISTS(
SELECT *
FROM ytgb1_virtuemart_medias
WHERE file_url LIKE CONCAT('%',i.file_url))
to create instances of new medias and then assign them by any way you can do it or like it to do
Here ytgb1 is the prefix of tables in your DB, and bng347_urls_for_import is auxiliary table with imported names of all uploaded medias.