You can modify the size in the front layout files
Search for displayMediaThumb
For eg. In file /components/com_virtuemart/views/manufacturer/tmpl/default.php
$manufacturer->images[0]->displayMediaThumb("",false);
replace with
displayMediaThumb("",false,"",rue, false,false,100,100);
to restrict the size to 100X100px
Of course, it's better to copy the file in your template overrides