Hi,
Yes. You need to use coupons instead of a discounts.
When you configure your coupons, turn on the auto load setting so that it will be loaded automatically on the checkout if the restrictions apply. Then, restrict it to the category you want, enter its minimum quantity (5, 10, 15, etc) and its corresponding discount amount (fixed or a percentage). If the amount is a percentage, you want to turn on the "Coupon percentage applies to product only" setting of the coupon.
The system will then load automatically the coupon with the biggest amount based on its restrictions and the products in the cart. So when you add more products, or remove the products, the coupon in the checkout will update automatically.