Hi,
Well, the thing is that when the payment plugin you use for your payment method redirects to your payment gateway, it gives the "return URL" where to go back after the payment. So it's not HikaShop directing the customer to that page, it's the payment gateway (or sometimes the payment plugin which is called by the payment gateway and then redirects to that page).
So HikaShop doesn't have the control of directing there even though the payment is failed.
In some cases the payment gateway provides two parameters: one for when the payment is successful, and one for when the payment is failed. And if the payment plugin is correctly developed, then it can redirect back to the checkout if the payment failed.
But this depends on the payment gateway and the payment plugin and how they are done.
So to the question "Is there anything that needed to configure on this part?", I can only say "maybe, but we would first need to know what payment plugin you're using and how you configured the payment method settings) so say if something can be done or not.
Also, knowing your version of HikaShop would be great. You noted that you have the Business 4.3.0 in the topic creation form, but I can see that your Business edition expired 2 years ago so you shouldn't be able to have it. Can you comment on this ?