Charlotte 
/Cinny 
charlotte
giving people that are complaining that their 20+ year old computer can’t look at “html documentation” due to anubis a dime so that they can pick a better computer up at the scrap yard
3
0
1
Klaus Frank
agowa338@chaos.social@charlotte Just set your user agent to curl or wget snd you can se most Anubis sites without having to wait.
We're now in the period where humans disguise as bots and bots as humans...
0
0
0
[GRLC] 
novenary@akko.wtf
1
0
0
@novenary but yeah i’d reckon blaming xe for shit deployments of it is genuinely just shooting the messenger
there is a reason why the only site i got that is behind anubis is git.chir.rs and it’s because git sucks ass
1
0
1
Charlotte /Cinny
charlotte
@novenary imo the better solution long term is to build websites that can handle more than a few requests a second without significant resources. hell you can probably implement most of a code forge like that too if, say, you could O(1) index objects in the repo or if certain things were rate-limited or precomputed…
2
0
1
@novenary (the other piece of the puzzle is to finally stop using wordpress because holy fuck does wordpress suck ass performance wise, and most other things)
1
0
1
[GRLC]
novenary@akko.wtf
or in some cases not understanding what they're doing and cranking the difficulty to fuck with the bots (they're only fucking with their users, the bots have already given up by the time the challenge is served)
0
0
0
[GRLC]
novenary@akko.wtf
"let's build backends for high traffic websites in a language that doesn't support multithreading by design" 🤡
0
0
1
Inga stands with 🇺🇦 🇵🇸
IngaLovinde@embracing.space@charlotte @novenary but it's not "more than a few requests per second". It's up to hundreds or thousands of requests per second; and if your website can handle that, then they'll come back with tens of thousands of requests per second.
I've heard of people with well configured webservers where just TLS termination of scrapers' connections was consuming the entire CPU.
1
0
1
Charlotte /Cinny
charlotte
@IngaLovinde @novenary did they happen to have a very large amount of distinct URLs on their site? i mentioned that factor also
as someone who has been defending against this myself, an individual scraper will ime not continue spamming the site with requests once they are done. and so the primary thing i have done to defend it is to ensure that the pages that they hit the most often were extremely cheap to return (i took my mirror of qmk private)
I think i would first run into bandwidth limits for my site which I have not hit. I sit at about 1-2Mbps consistent, but I imagine most of that is just due to federated services I run
1
0
1
Inga stands with 🇺🇦 🇵🇸
IngaLovinde@embracing.space@charlotte @novenary for my gitea it was iirc around 30mbps at peak, but that's because they were requesting complex pages requiring a lot of time to render
0
0
1