Cancelled orders not put back onto stock quantity

Just noticed after a test that I had only 1 item left of a product. When I ordered it correctly set the quantity to zero and showed this on the front-end saying Out of Stock. But when I went in the backend and cancelled the order, the quantity is still showing as zero for that product when it should now be set to one.