Conversation

Charlotte lotteheartplural/Cinny cinny_heart_plural thetadelta ursaminor treblesand

TIL about Content-Encoding: dcb, dcz, which effectively allows you to send only the differences from the previous version over the wire, if the client has it

2
0
3
testing it with a sample page change and it compressed my homepage down to 59 bytes given that the original was available on the client
0
0
0

@charlotte

(from the RFC 9842)

It is not unusual for devices to be on the network path that intercept, inspect, and process HTTP requests (web proxies, firewalls, intrusion detection systems, etc.). To minimize the risk of these devices incorrectly processing dictionary-compressed responses, compression dictionary transport MUST only be used in secure contexts (HTTPS).

i think we should kill the MITMs

(but also "no new features for insecure origins" is a reasonable approach too. but that's not the justification given.)

1
0
0
@5225225
yeah given that chir.rs should get put on the hsts preload list it's not a problem
2
0
0
@5225225
thinking about not just having one version of history however, since not everyone has the previous version cached
1
0
0

@charlotte yeah, i'm """waiting for my new domain to be on the hsts preload list in browsers before i do anything with it""" (absolutely not just being lazy and not wanting to make a new site just yet)

1
0
0

@charlotte practically speaking imo you'd be better off using dictionaries as intended (train one on all your html, use that for a while until it gets too outdated), so the odds of them having a useful dictionary is higher (and it helps fresh page loads, since i assume 2 different pages probably have pretty similar html)

1
0
0
@5225225
i took a dictionary that is the entire dump of my website (about 2MB) and the delta was only 3 bytes bigger
1
0
0
@5225225
i could do *both* too tbh
1
0
0

Charlotte lotteheartplural/Cinny cinny_heart_plural thetadelta ursaminor treblesand

Edited 19 days ago

@5225225 i don’t think it helps actually

the maximum zstd lvl supported by browsers compressed index.html to 4.35kB, a dictionary generated from all the html files on my website brings that to 2.28kB PLUS a 14.5kB (transfer size, also zstd max level) dictionary file that would need to be downloaded

1
0
0

Charlotte lotteheartplural/Cinny cinny_heart_plural thetadelta ursaminor treblesand

Edited 19 days ago

@5225225 what seems to work better is to take an empty page which Just consists of the site layout (that would be the https://lotte.chir.rs/sneakerweb.html page). dictionary compressed index.html is 2.60kB and the sneakerweb.html page itself is 1.94kB compressed

1
0
0

@5225225

!  Warning : data size of samples too small for target dictionary size
!  Samples should be about 100x larger than target dictionary size
Trying 5 different sets of parameters
WARNING: The maximum dictionary size 112640 is too large compared to the source size 544030! size(source)/size(dictionary) = 4.829812, but it should be >= 10! This may lead to a subpar dictionary! We recommend training on sources at least 10x, and preferably 100x the size of the dictionary!
k=50
d=8
f=20
steps=4
split=75
accel=1
Save dictionary of size 39419 into file out.dic

i think what it is telling me is that zstd –train really wants me to have at least 2x the amount of pages, ideally 20x. this will eventually happen but yeah

0
0
0