I had this same problem, and I fixed it by going into the Tax & Calculation Rules and editing the Tax rule by selecting 'default' for the Shopper Group. This way users who are not logged in get assigned the 'anonymous' Shopper Group and no tax is calculated for them. When a user is logged in, they get assigned the 'default' Shopper Group and the tax rule is appropriately applied if the user is from that state/province.
I agree the system should check if the user is not logged in and just skip the tax rules for users not logged in, but until that gets changed someday, for now you can simply assign the 'default' user to the Shopper Group in the tax rule, and it will work properly.