Hi,
In each variant of your product, you have an "access level" setting that you can use to restrict a variant based on the user groups of the user.
The problem with that is that it works the opposite to what you need. In your case, you want to list the user groups not allowed to access the variant, and not the user groups allowed to access it.
So what you could do, is to create a custom product field to store the id of the user group you don't want to see the variant. And then, you could add a bit of custom code in product / show.php via Display>Views to skip the display of the add to cart area for the variants (and replace it with a message telling the user that he already bought it) depending on the value of that custom field and the user groups of the user.