Different Tax on Virtual Product Variants

I understand that it is possible to individually assign Products to different Tax Categories, but is this possible with Variants?

We sell audio disc series which as a physical product should be taxed if purchased within the state. The main Product is assigned the Default Tax Category which has been set up for this. If the customer chooses the physical variant, this tax should apply. If a virtual product is selected (no weight / file(s) attached) I do not want tax calculated. (Or maybe a different tax rate would apply; still researching that.)

Is there a way to achieve this without dividing the audio series into two separate Products, physical and virtual?

Thank you.

Thank you for the prompt response.