Jorgen, i think it's possible to filter comming from Google using HTTP_REFERER, so user comming from email have note same HTTP_REFERER.
I only gave this sugestion, of course if you don't filter corretly you stop user acces.
Anotehr possible filter, is to check if the order have a Joomla user account associate and force user login.
I think using the 2 system should stop most possible access and google.
You can use another system using an existing value as customer name for eg, so external cannot access to order if they don't know the customer name.
This can be done using a system plugin for eg.