On my test installation I don't have those problems. If I delete a shipment address, it doesn't show up anymore. But I have a different issue. After saving the ShipTo address, VM doesn't return to the cart view without selecting the 'View Cart' menu item again.