Hide Characteristics that have no quantity

I have a show store where we list sizes as Characteristics. We want to have a complete list of sizes that we can use for each shoe. When applied to a shoe (product) we set the quantity we have to the size (characteristic). The problem is that there is no way to hide sizes that we don't have in stock. It's very tedious to go through each size checking to see if it's in stock.

Am I missing a setting somewhere or is there a bit of customizing that can be done to only display characteristics that have a non 0 quantity?