I didn't totally understand this, so I solved it in a different way and I am putting it here so others can also learn this.
I use Advanced Module Manager and selected the module to display in certain categories
Within the module I select to NOT display the filter module on the product itself (but this leads to an empty column)
So what I did (within the module) is go to the Advanced Module Manager properties ' url' and exclude.
Here I manually filled the machines where I don't want the filter to display. Since this is around 20 it's not that much work
I am pretty sure it can be done with a regular expression OR CSS or Template changes, but this was a quick fix for now
Hope somebody else learns something from this while having the same issue.