Well, I don't see what you want to do actually.
A product has only one brand. So on the product page, displaying a listing of brand which would display only the brand of that product will only display one brand. Is that what you want ?
On the other end, categories do not have brands. The brands are only associated to products. So "Displaying the brands of the category" has no sense as far as I understand.
So I can't say if we could or couldn't implement what you want but in any case, I can say for sure that it's not possible to display only some brands in a brand listing module for now.