the idea of a posix user as a permission boundary is inherently flawed. it made sense in the infancy of computing when you knew all the software you ran (maybe even wrote it yourself), and the primary threat was university students messing with each others’ shit on the expensive mainframe. but with modern computers being largely personal, it basically means there are no security boundaries whatsoever.
i don’t think that is a controversial statement. clearly people agree. see all sandboxing efforts ever, dynamic users for systemd services, android applications being basically entirely isolated things communicating via messages, that concept being borrowed to Flatpak and portals, or when we give up and run everything inside a virtual machine.

and yet, i still have to think about POSIX users when i use my computer. this concept is ossified into every operating system because that’s how every filesystem ever handles permissions basically. even if everything is isolated into a sandbox with its own restricted access, i still have to think about users! if i’m doing that on Linux, i must think about mapping a range of user IDs and that’s not always possible and then certain access boundaries are just impossible to construct because we ran out of users . what the fuck.
it’s all just workarounds and they all suck because they’re leaky abstractions over the same horrible concept. and we only have those abstractions because all our disks rely on that concept a million times over, once per file they store. so it’s really difficult to NOT have this concept be prevalent in a desktop operating system.

i perceive that the filesystem being the reason this concept didn’t die out, is because “everything is a file”, so it’s the least friction way of providing a semblance of access control to anything, a socket, shared memory, physical devices, whatever. and because file paths are ubiquitous in IPC (see: the Command Line. your shell doesn’t sandbox shit. it gives every command full authority as your user, and just names the resources being passed as arguments. everything you invoke could, in theory, just make your shellrc have a malicious alias for sudo). the way this should be done is by passing capabilities instead of paths. capabilities are unforgeable, and must be explicitly shared, but by default dropped, as opposed to users which are implicitly inherited and must be explicitly sandboxed.

but all capability-based solutions I’m familiar with (especially anything related to Linux) are just bodges over the user-based filesystem. they feel horrible to use. it’s a pain to set up.

is there an operating system where it’s capabilities all the way down? where users are just Not A Concept? (obviously, the semantic concept of a “user” in the sense of login credentials is useful. but i mean as the fundamental method of access control)

how wasi does filesystems feels pretty intuitive. it feels obvious that every operation should be relative to a capability. there is no such thing as an absolute path here, because that concept is inherently harmful. the root directory being ambiently visible is harmful. it encourages the idea that we need to restrict permissions on files, hide directories, so that everyone can refer to global paths nicely. we want global paths because we put everything in /bin or in the /nix/store and those file paths are stored within environment variables and we pass them as command line arguments.
i dream of a system where file descriptors (aka capabilities) are the default way of passing arguments that refer to any resource (i.e. along with options and data that we already pass this way; something like “listen address” is neither. it is a resource). a directory has no parent (up-traversal considered harmful); it is strictly a collection of other capabilities/file descriptors (and usually isn’t even persisted to disk). environment variable like PATH should contain capabilities to read particular paths; not a stringly-typed list of paths.

something like the nix store should be an implementation detail, and not world readable actually. building a derivation should give you access to a particular directory that is otherwise inaccessible. rpaths in ELF executables should actually be capabilities (file descriptors; or when written to disk, this is also called a “hard link”), and the “closure” of a Nix store path shouldn’t need to be visible unless you explicitly go through the exact dependencies that lead you there. (i think this requirement systemically fixes “compressed files are opaque to the nix store garbage collector”; because capabilities that are forgotten can by definition not be reconstructed, unlike how garbage may decompress into a real store path. however, this solution requires that you put hard links in the middle of executable files. the obvious solution is to make executables directories, which is the way to reference a collection of capabilities, one of which might be “the executable machine code for this program” and another might be “a shared object library that this program depends on” [not a copy of it, but an unforgeable reference to it]; obviously this doesn’t work for NixOS but i’m thinking about how I’d do something like Nix in a Very Capability Computer)

instead of a root directory, such a system might have an intrinsic Persistent Directory. it is the single directory passed to the init process. all other directories are “ephemeral”/“indistinguishable” in that they have no intrinsic name and can be created and dropped at will. but the Persistent Directory is special in that. anything written to it persists. (and anything reachable from it persists)

posix-esque shells would not work at all on such a system. major consideration must be taken for how a command line could feel ergonomic when there is a strong intrinsic distinction between options and capabilities/resources. GUI workflows would be mostly fine: flatpak+Portals already has a capability-based file picker, for example. as does Android. (i.e. programs can access files because they have a capability to “present the user with a file picker”, not because they have a capability to access user files in general)

in particular for command line, anything which keeps state would be painful. something like cat or sed is relatively straightforward to port (by default, no access to anything; but you explicitly pass in the files they operate on, so passing them as capabilities would not be super painful), but how would you manage ssh and its keys? you don’t want to pass ~/.ssh as an environment variable (ambient capability), because then everyone can read it. so maybe we need to have something akin to .desktop files for command line applications too? to define any “implicit capabilities” they’re allowed to access. or maybe .desktop files are not necessary? executables are already directories. maybe they have a section for “ambient capabilities” of that executable? you could easily generate per-login-name versions of your executables with additional, user-specific capabilities, and just make both hard-link to the same executable code segment (minimal duplicated data on disk; compared to storing two sightly different ELF files). yeah i like this.

I’m thinking directories/files as capabilities can be downgraded to read-only versions of those capabilities. not only do users not exist in the model I’m describing, but neither does any intrinsic permission bits on a resource. to make a file read-only, simply forget the fact that it was ever mutable. if a read-only file descriptor is Persisted with no writable handles to it, then by definition that file is forever immutable until it is deleted. (could something interesting be done with copy-on-write to create a stronger sense of “immutable means immutable”? it’s probably undesirable by default but might be an interesting semantic to expose?).
“ambient capabilities that are only visible to one program” being embedded inside the executable (e.g. for state management) would be a neat way for something like sudo or pkexec to work as well. but it requires a third visibility than mutable/immutable. there should probably be an “opaque executable” capability that contains other capabilities, but cannot be inspected like most others (directories are always readable; because otherwise you couldn’t do anything with them).

managing multiple filesystem volumes/partitions (in particular, hotpluggable ones) under this model would be kinda pain. as i’ve described it, symlinks can’t really exist? but for hotpluggable drives, it’s perhaps good? this means that by default, any programs isolated to such a drive are “portable” (normally, symlinks break if a drive is removed). maybe even “safe” (if it has no capabilities to your main system, it can’t harm anything except shit on the removable drive). for them to interact with the system at all needs something in the ambient environment variables (which should mostly be read-only. though you might also stick a Wayland socket as an ambient environment variable capability, for instance). maybe for portable apps with per-machine metadata, you need a daemon to parse some app definition and generate executables for your system with the relevant state directories included. or maybe there’s an ambient capability for “portable state directory” or something.

similar concerns apply to packaging things that contain such hard links when installed to disk. especially with serializing cyclic references (silly thing to create; this entirely defeats the point of a “child directory” as a permission boundary, if you can just walk back up to its parent)

i should really use my regular size blog more and not make Sophisticated Microblog all the time

i noted that the command line shell would need serious rethinking under this model (how to make passing options, resources, and inputs, ergonomic). and while i have nothing more to say about the shell syntax, i do think there’s an obvious abi: every process invocation takes one capability in the place of argv, and it’s expected to be a directory containing a structure of other capabilities (such as stdin, stdout, stderr, and custom entries like listen_socket: STREAM (pseudo-ls notation for a TCP-like socket resource)). this also lends itself to nesting: your ephemeral argv directory-struct can contain “arbitrarily shaped” less trusted directory-structs, as “input” (which might in itself also recursively contain more capabilities and more trust boundaries).

this argvish directory-struct, and a capability representing “ambient” things (environment variables, intrinsic things only readable by this executable like a state directory), are the two arguments to main. all IO must somehow go through these. most of the time, you might have an ambient environment capability to make outgoing network requests, as well as an ambient capability to request more capabilities (i.e. sudo/polkit/UAC). but you might drop that ambient network capability. or, an application might decide that you can parameterize it and pass multiple different network capabilities for different parts of that program (e.g. a reverse proxy, which might have an upstream url for an internal network interface, but ACME requests should be done with the public interface. this kind of separation also means that most things that were previously single config files, kinda need to be directories on this kind of system. all configuration is directory now! because it’s the only way to compose capabilities).

@sodiboo This sounds a lot like what Apple tried to make with iOS. No user, only a passcode. Each app a sandbox. No way to up-traverse outside of your box. Most of the time, you don’t even have to think about specific directories, everything has a predefined “place” in your sandbox, to a point where even files used to not be a thing at all. Of course, the thing being based on Darwin meant POSIX compliance and thus you’re just a jailbreak away from getting a root user and seeing through the illusion

It also sounds a lot like what Microsoft tried to do with Windows Phone, and WinFS before that, but with a focus on personal data instead of applications. You had this centralized store of things. Apps could define their own new hub for those things if there wasn’t already one out of the box, like pictures, music, and contacts. You could request access to those hubs and integrate in them, and that wasn’t exposed as giving user access.

The real problem here is that industry standardized to POSIX and will probably never move away from that, instead preferring to slap patches like docker on top of the current solution so that existing stuff keeps running. I truly can’t picture what you’re proposing running without forking the Linux kernel. That’s probably why no one has truly done it yet, because everyone I know have been chasing that dream, either consciously or unconsciously since forever.

i’ve considered going through and doing some additional revision/cleanup on several of my Sophisticated Microblog and promote them to Real Blog. this one seems perfect for such a promotion.

@kawazoe something i don’t really like about Android and iOS in this regard is that they have a strong concept of “these are the Installed Apps”. and they all hook into very specific parts of The System . also that’s the Only thing you can install.
and crucially, the sandbox exists around an App. not a process. does an app want to spin off something else in its own sandbox? sure, call out to another app. want to spin off the same thing but in a different sandbox? haha, no, it’s the same sandbox. it’s the same app, after all :)

their capabilities don’t compose very well. because it’s not capabilities all the way down. it’s secretly just user-based access control, except the “users” are system users generated per application. (not literally system users, i believe? there’s something Capable happening in there. but like, it feels exactly like systemd dynamic user. this doesn’t make it easy to run that service from within another service, but it does “isolate” each service from one another)

this is the inevitable result of a sandboxy approach. it needs to be capabilities all the way down.

@sodiboo

@novenary oh yeah! nice! there’s prior art here! i came to the same conclusion, almost verbatim writing the same thing as the title of the page you linked:

up-traversal considered harmful

@sodiboo though one note is i think it doesn't need to work like a straight up directory, and "everything is a file" even for things that kinda aren't, is a bad move

(though idk how "directory" like you mean)

also, just rediscovered https://www.skyhunter.com/marcs/emily.pdf which goes into "how do you call a program as a user and be explicit about what you're giving it?"

where it has syntax like program {input }output !readwrite *time

i think "programs declare a "schema" for their input (akin to how you can often define a struct that your inputs are parsed into), and now your shell knows "this is a file that needs to be given read-write", "this is a file that is read-only", etc.

(and have the actual data in-memory handed to be the program probably be integer file descriptors)

"program argv is a structure that's reflectable" also makes it safer to call programs from other programs (filenames can't be confused as flags, for example)

@5225225

though one note is i think it doesn’t need to work like a straight up directory, and “everything is a file” even for things that kinda aren’t, is a bad move

yeah. to some extent i agree. however, I’m coming up with this design from a background of mostly Linux. “everything is a file [descriptor]” makes sense enough. the fact that i can name sockets by placing them next to regular files and such, feels fine and non-problematic. this perhaps could change; I’m not convinced it is the best way to do it. but i don’t think this needs to change; it’s not the part that bothers me.

(though idk how “directory” like you mean)

yes. i think the terminology is a bit off (for example, I’ve been extremely loose about “capability” vs “file descriptor” vs “handle” vs “resource”; they’re all synonyms as far as i’m concerned; as well as any subset of “ambient environment variable capability”). i’m very much focusing on using directories as a way to organize non-regular-files first and foremost. some other terminology could be useful. but because they also contain regular files with arbitrary bytes, i felt it was fine to keep the name “directory”. i wasn’t trying to come up with the final design with good user(developer?)-facing names. just sketching out the fundamental mode of access control

@sodiboo For the past 5 years i've been trying to work on tools needed to make the foundation needed to make the kernel needed for an OS that would be able to fix this but absolutely nobody seems interested in helping once they see the gigantic mountain of work required to pull it off.

so, no.

I can point you at lots of research papers that theorize about what such an operating system would look like, so we might gaze at the beauty of the concept of an idea that absolutely nobody seems willing to actually make into anything more substantial. You can also look at our numerous failed prototypes that didn't work because modern computers are shockingly hostile to good ideas in ways I never thought possible.

@sodiboo "is there an operating system where it's capabilities all the way down? where users are just Not A Concept?"

you might be interested in Genode: https://genode.org

actually no, i think environment needs to be explicitly passed: env is just another parameter inside of argv (which now has a misleading name because it’s not a vector; it’s effectively a dictionary). there is no reason it actually needs to be special, at the kernel level. there should simply be a user space convention of how arguments launched by a command line interactive shell are named, and how graphically launched applications are parameterized, etc. in both of these, some bottom of “global arguments” (environment variables) is useful.

but in general, “environment variables” needn’t be an intrinsic thing that is implicitly inherited. it can just be a conventional name of “a bunch of arguments you PROBABLY don’t care about but child processes might, pass them if those children are trustworthy”.

and, for shell syntax, there is prior art in nushell and powershell for structured stuff. in particular, they both represent environment variables as “just a shell variable” which can be non-stringy values (in nutshell, tables are similar to my “directory-structs”, and manipulable from the REPL, and in PowerShell, you can have .NET objects, which very directly represents a “reference” to a specific object/capability). in particular, both have some notion of inherently structural key-value-mapping terms.

i’m thinking some more commonly passed “implicitly inherited” arguments should not even be “environment variables”, but even stronger conventions. like the default network capability should be passed as net, similar to stdin/stdout. maybe a shell should implicitly understand some of these and have syntax for passing them by default / explicitly not passing them. maybe some special variable for “which shell variables are arguments by default”. although at this point i’m maybe just reinventing environment variables, immediately after deciding that environment variables might be considered harmful.
dropping environment variables is gonna be a hard sell if we’re porting literally any software to such a system. but at least they can remain just a convention (and yes, they have the network capability, right alongside a wayland_display). i really like the ABI-level of “main accepts exactly one root capability and the kernel enforces no further structure”

something in particular about how much directories are used to bundle capabilities: they are inherently ephemeral by default. persistence is the abnormal case.
but they’re also expected to be used so much that i don’t think they should fit in ram by default. it’s reasonable that they (or, entries within them) will be written to disk (this is literally just swap; just with slightly different heuristics).

a consequence of this is that /tmp doesn’t need to exist. /tmp is considered harmful. want a temporary directory? just create a directory and don’t store it anywhere. want to share that with another process? pass it, through the arguments when invoking, or share it over a socket.
want a temporary synchronization primitive so that two unrelated invocations [perhaps if the same executable] don’t step on each other’s toes? inherit a directory explicitly. if it’s difficult to synchronize both because there’s no directory they share, then, theoretically, they should by design not be able to interfere with each other at all. (e.g. if they both have write access to the same database, they can also create a lock next to it / inside of it. but if there’s no place to share a lock, there is no resource to contend)
want something like X11 display “ports” so you can just DISPLAY=:0 and DISPLAY=:1 at will? then you must have an inherited X11_DISPLAYS capability which is a temporary/ephemeral directory. this namespace is inherently scoped. you can just create another one in a separate part of the process hierarchy, and :0 would mean a different thing in that part of the system.

@LunaDragofelis

Capabilities are objects with methods (my pro-OOP bias shows)

yeah. but i think that makes sense. it’s not just “i like OOP”, it’s that capabilities are objects and thinking about them any other way is just wrong. there are different types of objects and objects have different types of interactions. every object has an interface.

objective-C (for apple system frameworks), Microsoft’s COM, D-Bus, and Wayland, all to some extent, use this to represent “capabilities in a foreign function interface”. in Wayland for example, you have a sense of “ambient capabilities” in the form of manager objects (emitted by a single intrinsic wl_display zeroth root capability), which contain top-level functionality, such as the ability to create a screen capture session. but if you never received that screen copy capture capability, you can by construction not even form a request that means “record the user’s screen”. so, to “sandbox” a flatpak application, you just don’t give them “dangerous” capabilities, then they can interact with exactly the same codepaths as “fully trusted applications”.

this is also roughly how wasi works: the “system interface” is defined in terms of the webassembly Component Model (whose name is a subsequence of the windows equivalent). no rules ever dictate that any of them have to be backed by “intrinsic” operating system primitives. it’s just a strongly typed way to compose “processes” (isolated parts of an execution environment). but when you start having strongly typed communication like this, the exact boundaries between a “child process” and a “library” starts becoming very blurry on purpose.

i think there’s a certain complexity to the strong typing of these systems. i kinda like the ducktypiness of “everything is a file”. it does make IPC and composing scripts a bit less painful. i’m thinking in the back of my mind that “a lot of these interfaces will go through an application layer of serde-equivalent, to parse the directory structure and create a language-specific object”. but idk. maybe there’s value in the interface actually being properly strongly structured/typed.

something else that happens in wasi is that the boundary between kernel and userspace can disappear too. because wasm is isolated from the ground up, any generated machine code is “well-formed”. you get some JIT overhead, but in return, syscall overhead disappears. there needn’t be context switching. sysvalls are reduced to virtual function calls, i think. (and because there’s no clear distinction between a process and a library, the call can just happen on the same thread). it doesn’t really work with native code. but it seems obvious for a wasm-based system that one should build it like this.

so that’s an interesting thing that kinda naturally falls out of “all I/O is capability/object-oriented”.

also i think Fuchsia does this kind of thing? they have strongly typed resources, i think, and files are just a userspace abstraction. i need to look into how exactly they do IPC. maybe that’s a good starting point for a system I’d build.

cwd is another good one that should just be an optional argument.

and, perhaps, for cases where directories are used for organization and not as a security boundary (e.g. a git repo), it would make sense to have .. entries everywhere within that workspace.

@sodiboo why not pass in the repo root plus some sort of object reference within the repo?

@charlotte hm. i was thinking in the case of e.g. invoking a bare git command from somewhere within a repo.
though, maybe cwd like this is just straight up an anti pattern. i was thinking it should be implicitly passed by a shell but lol no because then if you do run anything in ~ it has access to ≈everything. so actually cwd considered harmful.

so i think you’re right. commands that are source control aware or otherwise have a notion of a workspace should have that workspace explicitly passed, not through cwd.

perhaps it would make sense to have a “development shell” pattern like we do on Nix, which on this system, would define aliases like git = GIT_ROOT=${.} git (pseudo-shell syntax, ${} is a capability, not string interpolation. and it’s not an “environment variable” passed to everything; only git invocations get it) and cargo = CARGO_ROOT=${./backend} cargo (let’s say, for the sake of argument, it’s a web service and the frontend is JavaScript; cargo doesn’t need to see it)

unclear how those tools should know how to convert a referenced file into a path to backtrack (e.g. to resolve include! if it is like, a cargo script and we didn’t start from a Cargo.toml). again, need to revisit syntax before any of this is usable or implementable. but i’m pretty okay with this kind of way to handle the capabilities that git / cargo / whatever needs to work (i.e. a workspace root reference)

i’m also thinking a shell is gonna keep track of a “stack” of directories it followed. so ~/Documents/Code/project1 could be displayed as such because it remembers that we entered subdirectories in that order. and it keeps the capability of ~/Documents around for when we want to step up in interactive mode.