Hi,
Before you do anything else, I would recommend to turn off the "checkout legacy" setting of the HikaShop configuration in order to use the new checkout system. You're now using the old one from HikaShop 1.x and 2.x.
With the new checkout, you'll see that you don't have the system going to the next page when you change the payment method selection.
Also, if you check the "checkout workflow" setting, you'll now see a whole new interface, with many more settings.
And in the "address" view, you'll see the setting "Allow multiple addresses" that you can disable to remove the "new" button.
Then, if you want to customize the view files used on the checkout, you can edit them via the menu Display>Views. To know which view file corresponds to what, you can temporarily activate the "Display view files" setting of the HikaShop configuration.