Bare metal server sounds like optimal solution for you and set up a hypervisor on top of it, so it’s pretty trivial to migrate VMs to your own hardware when needed. But then for your ‘long term’ environment VPS would most likely be better and migrating a full VM from your hypervisor to VPS is a bit more work, but can be done.

I don’t know about providers in Australia, but Hetzner has both and combined billing and my personal experience with them is pretty good. But I’m in Europe, so bandwidth nor latency is not a problem.

My experiences are few years old, so I don’t remember excact models anymore, but some back-ups models (es series rings a bell, but as I said, it’s been a while) had batteries with soldered connectors and form-factor which (at least at the time) wasn’t available from anyone else than APC.

I have older 1500VA FSP UPS, I don’t think that exact model is available anymore, but it’s been solid for several years. It currently has 3rd or 4th set of batteries and they are standard bulk batteries, so replacements are easy to find from anywhere. Only problem I’ve had with that is that on display it doesn’t give out clear warnings when batteries degrade and it has crashed my system few times in a power outage, but I’ve been lazy and didn’t bother to properly monitor it nor have scheduled battery replacements, so that’s mostly on me.

Eaton seems to be pretty solid too, but I don’t have a ton of experience on any of their models. Local suppliers had dirt cheap PowerWalker UPS’s a few years ago, but one of them didn’t survive when battery died, so maybe I got what I paid for. Those worked fine too, but apparently they cooked the carging circuit when battery degraded.

This is of course just my own experience over a few models, but personally I wouldn’t spend my money on APC. Propietary batteries and multiple failures after battery replacement at work few years back were enough to choose something else.