it is extremely funny to me that a literal trillion dollars funnelled into a cloud provider results in more sev0 / sev1 outages per year than you'd get with a hosting provider run by a polycule out of an apartment block in rural Germany.

@gsuberland I mean *sure* but also most tiny providers rather balk at requests like "I would like to replicate 3pb of data to you tomorrow would that be ok kthx". Nobody running tiny easily movable things is in us-east-1 anymore.

@jonty @gsuberland I have bad news for you about all the tech startups whose entire business could fit in a moderately sized postgres database

@gsuberland ... until the polycule has a messy breakup and the service just evaporates in the ensuing mess.

@joepie91 @jonty @gsuberland i’m sorry what the fuck is an index

@joepie91 @gsuberland @jonty i think there’s also a bunch of exceptions to behavior that exclusively apply to us-east-1

at which point i wonder if those exceptions may be why some people might choose to go there even though the exceptions almost certainly just are due to some very well-paying customers doing it wrong™

@charlotte @gsuberland @jonty @joepie91 “SQL? We just vibe coded some calls to an ORM.”

@gsuberland That's why you just host at the big german hosters, where the catgirls are also in charge of operations and things work :P

@gsuberland @cstross Rural Germany has maybe DSL 16000, if you’re lucky 😭

@schrotthaufen @cstross the catgirl polycule will find that one place in Meck-Pom that happens to have a fiber trunk running past it

@gsuberland my personal mailserver has more uptime than both gmail and us-east-1 combined. and i have the graphs to prove it.

turns out a one-person mailsever is so underutilized that it will live forever, and with only one person using it, the complextiy is a micron thin by comparison to any cloud provider.

less compelxity = less problems = less maintanence = happier nerds

I guess at that point the major points of failure would be the ISP.
@Viss @gsuberland

@JessTheUnstill @gsuberland at one point in my career i heard a senior networker type say something like:

"you can never completely remove a network bottleneck. the best you can do is push it so far out that its not on your network, not on equipment you can control anymore"

and today, in 2025, i wager if youve done that and accounted for it, dual isp is the fix :D

At least until all of the ISPs for your data center share the same fiber conduit. And then that one asshole backhoe...
@Viss @gsuberland

@JessTheUnstill @gsuberland yeah - but the same sorta premise exists for them too.

though, the advantage is that if all of your shit is in your home office, then your isp or uplink going down means 'you just dont get new mail until the outage is resolved', not: oh fuck my entire business is down because all my core equipment is on the other side of that link".

you are still up. your shit is still up. you are on the same side of 'the outage' as your shit

@jonty I know, I know, apples to oranges :P

@gsuberland More like grapes to moons

@jonty also yes

although on this note I recently priced up a 100PiB rack (complete with redundant head nodes) and it wasn't as ridiculous as I was expecting. within "handful of friends deciding to set up a backup provider" distance. drive density per £ has been skyrocketing.

@jonty although I suspect the real wallet-stinger would be peering somewhere that can meaningfully support such large transfers

@gsuberland @jonty truck full of disks has the highest bandwidth. Always has, always will. At that amount of data, you'd likely drive over at least the initial sync

@gsuberland @jonty transit has gotten cheaper, depends upon bandwidth requirements. Also peering cost is heavily location dependent and who you need to receive data from

@gsuberland @jonty (which is to say: your customers use Telekom or Orange? You're fucked and the only thing saving you from being uberfucked is the fact that your DTAG customers are stuck on VDSL)