That's indeed the correct approach.
The fields are displayed in the file custom_fields of the view user that you can edit via the menu Display->Views.
There, you need first to parse the products in the cart.
$product_there = false;
foreach($this->rows as $i => $row){
if($row->product_code=="my_product") $product_there = true;
}
then, in the foreach of the fields, if you don't want to display the field "my_field" when the product "my_product" is in the cart, you need to add at the beginning something like that:
if($product_there && $fieldName=="my_field") continue;