I'd scour the offers out there for cheap dedicated servers (if your budget allows the luxury). There are a number of companies offering ARM platform which will work just fine for most uses. Some of those can be had for $10-20 a month. Downside is you aren't going to do lots of nested virtualization on those. Maybe Docker.

I'd find cheapest ARM offerings and use them for single task sort of chores.

There is older generation gear piled up all over and it can be cheap rental. Straight Intel and AMD gear. Can slice and dice your server up real good and do everything.

Like here (no clue about their general reliability or private customer tolerance): https://billing.dacentec.com/hostbill/index.php?/cart/dedicated-servers/

There dedicated servers start at $20 a month.

Other companies out there like OVH which host a lot of sketch on their big fat network, but may be a sign up issue.