Conversation

nixpkgs cross compilation has an annoying property that it has entirely unique hashes from the native versions, because the inputs are “different compilers”.

content-addressed derivations solve this to an extent, but there’s still the need to compute that a non-cross derivation had the same input. so when “rebuilding the world” offline you’ll have the same issue; but it DOES mean that caches can claim “yup the cross and non-cross derivations built the same content”. so it saves a lot of rebuilds if you cross-deployed a system that you then rebuilt locally on, as long as most of the packages are in a cross-compilation-aware cache.

but what if cross compilation wasn’t a unique derivation? what if the nix builder could execute wasm-wasi binaries and all compilers were built to wasm? and that even the “seed” packages at the root of the build graph were wasm-based?

then, cross-compilation would be the “always” case and it would always be wasm->target.

and it would make a cross/native build use the same derivation. because the derivations are architecture-agnostic.

(this all is completely ignoring the idea of supporting multiple operating systems. darwin/linux would need special care. but i’m specifically thinking of the linux->linux case where they have different CPU architectures)

it doesn’t have to be wasm. but like, any architecture-independent intermediate representation. what if nativeBuildInputs had to be architecture-independent?

1
0
0
@sodiboo
ca derivations do not solve this to any extent, it is unclear what problem they solve to begin with
1
0
1

@charlotte sure, they don’t solve this. but they do generally cause the property that “the cross compiled derivation has the same output path as the non-cross compiled version”.

and if we have a binary cache doing some verification for us ahead of time, it does mean that for the packages it covers, we don’t need to redownload the outputs.

and in practice, in nixos, most derivations are cached by cache.nixos.org. and if that’s your entire system, it’s “basically solved” with ca derivations. the non-cross rebuild will take essentially no time.

but in the general case, yeah there’s absolutely still a problem. which is why i said “to an extent”. i stand by that.?

1
0
0

@sodiboo i guess it does work for cross compilation from say x86_64-linux to riscv64-linux vs aarch64-linux to riscv64-linux, but native builds specifically have a different path since their names are different

1
0
1

@charlotte what, like there’s a -cross name added? sure, whatever, that’s besides the point. i’m not talking about nixpkgs in practice today . i’m talking about what it could do in theory. just like, abstractly, what’s possible, with the nix store concept. nixpkgs is my baseline for an implementation with flaws.

1
0
0

@sodiboo -${target} suffix but yeah

at that point you could also kind of flip nix’s cross compilation approach on its head

“this is the derivation. here is a list of ways you could get here” and treat the variants as logical equivalents (fetch whichever one is present or buildable)

1
0
1

Charlotte lotteheartplural/Cinny cinny_heart_plural thetadelta ursaminor treblesand

Edited 18 days ago

@sodiboo i think most of the benefits of CA derivations can’t be realizes while nix is dynamically linked or nix doesn’t actually replace the build systems

0
0
1