I think , it's not VIrtuemart related.
Problem is that cart need to be displayed by user and always refresh on each change. All other Joomla page can use cache without issue.
So if you use any type of cache and this cache the cart then you get problems. This can be browser cache, joomla cache or any other caching system.
Check that the expire time for cart is always set to an old date eg in your document :
Cache-Control: no-store, no-cache, must-revalidate, max-age=0
But perhaps this should be done by Virtuemart to be sure it's not cached in the server ?
A PHP sample is
header("Expires: Tue, 03 Jul 2001 06:00:00 GMT");
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-store, no-cache, must-revalidate, max-age=0");
header("Cache-Control: post-check=0, pre-check=0", false);
But need to be set Joomla way is better.
If some code developper read this post, i hink this should be in the VM core code, to be sure cart get not cached by browser.