In April, I worked on lots of different projects. Thankfully, things have been quiet on the ppxlib front (in part due to Nathan taking on some more work and being between releases of the OCaml compiler). I managed to improve some of our migration error logic and do a little spring-cleaning.
More Landlock Thoughts
Elsewhere, I spent some time exploring Landlock and how it might be used with Eio. I think it is useful to down tools once in a while and try out some newfangled API or OS feature. Landlock is quite different to Capsicum (I had assumed they would be more similar). Whilst I enjoy the fine-grained control over access and capabilities, the manual process of constructing rules is a little cumbersome (and perhaps easy to make mistakes with). Combining that could have some Eio code like:
let
=
Eio_linux
.
run
@@
fun
env
->
let
data
=
Eio
.
Path
.
env
#
fs
/
"
/data
"
in
Eio
.
Path
.
with_open_dir
sw
data
@@
fun
data_dir
->
match
Eio_linux
.
Landlock
.
enter
with
|
Error
`Msg
m
->
failwith
msg
|
Ok
->
analysis
data_dir
Opening the directory (in the openat-sense) adds a Landlock rule to the current context (some kind of process-local state). This rule would be removed whenever the data directory is closed (by hand or when the function returns, i.e., when the internal Eio.Switch releases its resources). Doing the bookkeeping internally means mistakes are less likely to be made whilst also improving, in my opinion, the API.
This breaks down for networking, unfortunately. Whilst the filesystem hierarchy provides a clean way to produce capabilities (a handle on a directory + openat + RESOLVE_BENEATH), networking does not. The Landlock API, at the time of writing, only allows rules based on port numbers. The system API is completely ambient (socket + connect). Eio's API provides access to the network through a single network `capability. All of these mechanisms are somewhat at odds. What does a network look like if it adheres to the ideas we have for combining Landlock and Eio (like in the filesystem example)?
The API becomes port-oriented. We're already at odds with Unix domain sockets, but we will carry on regardless. A capability on a port means you can listen on that port and accept connections, or you could connect to it and send data. At the time of writing, Landlock is restricted to TCP ports. In Eio, there is no API to with_open_port, i.e., to register your intent to do something with that port -- you either listen or connect.
All of this starts feeling a lot like Plan 9; something which I hope to explore further soon.
Weekly logs
The full details are in the weeklies transcluded below:
Moving away from Github
A short week for me as I was hiking with family in and around Porthmadog, Wales.
Reliance on Github
I joined Github in 2016, but only began using it in earnest (according to my Github activity) in July 2018: right after Microsoft announced its intent to acquire Github; and just before they closed the $7.5bn deal. I originally joined to boost my résumé and I hoped to receive the odd open-source contribution to projects I was working on (like ppx_deriving_yaml). Since then, I have used Github extensively both at work (Tarides) and for more personal projects.
There have been many reasons to jump ship from Github ever since the Microsoft acquisition (e.g. their planned (and quickly unplanned) announcement to charge self-hosted Github runners). Ultimately, I was convinced when I noticed the new AI tool on the homepage with options to select your "versatile and highly intelligent" model. There is no need for me to stay on the platform for my personal projects. AI integration into Github has nullified those earlier motivations for hosting my projects on the site. The number of contributions has increased, but many of them fix superficial elements of an underlying bug. In my (maybe too uncharitable) imagination, it is all too easy for contributors to prompt their built-in LLM to "fix this broken unit test" and for the fix to layer a single-use workaround on top, instead of identifying and correcting the relevant code. This is, at least, what it looks like to me when I receive a new review request on my open-source projects.
At the end of last week I set up my own Git infrastructure following Ryan's nixos setup. In the future, the source of truth for my projects will be git.sirref.org (I will likely continue to mirror to tangled.org or Github).
Reduce, Reuse, Recycle
Part of self-hosting your own source code is managing some form of automatic testing. I hesitate to describe it as "continuous integration" as the term is confusing and overloaded. Luckily, parts of the OCaml community have been building test infrastructure for a while.
I have a proclivity for rebuilding things from scratch be it to learn or to build something the way I want. It is a proclivity I am trying to reign in as I find less and less time to hack on projects. I think this is a good thing. Reusing and recycling existing technologies, in earnest, feels like a pushback on the not-invented-here syndrome that seems to be snowballing thanks to LLMs and code generation tools.
To that end, I dusted off my OCurrent knowledge, and repurposed the "local" mode of OCaml-CI to work with my git repositories. After battling through nested layers of git submodules, I managed to Nix-ify this and get it running at tests.sirref.org. I needed to add multi-repository support as well as Current.Monitors for the local git refs. By no means perfect, but it is nice to own the testing infrastructure (and the cost of downloading OCaml images and the compute to run the tests).
Landlocked Eio
I spent the start of last week wrapping up the "local" CI I mentioned the previous week. Unfortunately it was mostly a packaging nightmare, followed by missing features in ocaml-ci-local. It would be great to have a richer CLI tool for running tests on a directory of git repositories, but unfortunately most of the existing tooling has prioritised the Github/Gitlab etc. APIs.
After coming across minimal during the week, I took it as an opportunity to learn a little more about Linux's Landlock mechanism.
Landlocked Eio
Landlock is a sandboxing mechanism that has existed in the Linux kernel since 5.13. It allows unprivileged processes a means to restrict their ambient rights (e.g. creating a file or opening a network connection). These rights are ambient in the ambient authority sense, i.e., any process can attempt to perform an action without needing access to any particular capability to do so.
Ambient systems are often contrasted with capability-based ones. In terms of operating systems, this is most commonly illustrated with access-control-lists-as-columns and capabilities-as-rows in an access matrix.
Eio is a capability-based library in OCaml -- resources are provided at the start of the application and can be passed around wherever they are needed. The design is covered in Thomas Leonard's lambda capabilities blog post. Unfortunately, any application (willingly or via a third-party library) may ambiently access OS resources. In OCaml, this may be by using the standard library functions (e.g. Sys.readdir), things from the Unix module, a C FFI function or by spawning a subprocess.
I had a go at "landlocking" Eio applications last week after stumbling across minimal's hakoniwa library for process isolation (which is very reminiscent of void processes).
module
Landlock
=
Eio_linux
.
Landlock
let
/
=
Eio
.
Path
.
/
let
run_eio
file
=
Eio
.
traceln
"
Writing to file
"
;
Eio
.
Path
.
save
~
create
:
`Exclusive
0o644
file
"
hello, world!
"
;
Eio
.
traceln
"
Reading file:
"
Eio
.
Path
.
load
file
;
Eio
.
Path
.
unlink
file
let
bad_program
=
In_channel
.
with_open_bin
"
/etc/passwd
"
In_channel
.
input_all
|>
Eio
.
traceln
"
/etc/passwd:
"
let
=
Eio_linux
.
run
@@
fun
env
->
Eio
.
Path
.
with_open_dir
Eio
.
Stdenv
.
cwd
env
/
"
.
"
@@
fun
dir
->
let
parent_fd
=
Eio_unix
.
Resource
.
fd_opt
fst
dir
|>
Option
.
get
in
let
file
=
dir
/
"
hello.txt
"
in
let
rules
=
Landlock
.
Rule
.
path_beneath
~
parent_fd
~
flags
:
Flags
.
Fs
.
write_file
+
read_file
+
make_reg
+
remove_file
;
in
match
Landlock
.
enter
rules
with
|
Ok
->
run_eio
file
;
bad_program
|
Error
`Not_supported
->
Eio
.
traceln
"
Landlock not supported!
"
Running this program (with the necessary privileges) prints the following.
Eio already has a similar function for the BSDs using capsicum. In capsicum, processes enter a capability mode:
cap_enter()places the current process into capability mode, a mode of execution in which processes may only issue system calls operating on file descriptors or reading limited global system state.
This plays nicely with Eio's capabilities. Within the scope of Eio.Path.with_open_dir, the directory will remain usable after entering "cap" mode. This is not the case with landlock unless you add a specific path_beneath rule to the ruleset (like in the example above). However, you could imagine plumbing the necessary information throughout the Eio_linux backend for it to build a set of rules that can be applied at any given moment in order to mimic the capsicum semantics.
Self-hosting Music
After my friend, Greta, sent me some articles about music streaming services and their exploitation of artists, I decided enough was enough and setup navidrome. It is compatible with Subsonic and so far I'm very pleased to be streaming my own music to my devices. I am using substreamer on iOS. On my laptop, I tried out Andreas Rossberg's Camp
What you got here is an old-school music player heavily inspired by good old
I took Anil's OCaml Installer tool for a spin with great success... after a few mishaps mostly related to system dependencies.
The fixes in my fork of Camp are just to install the assets using dune and also statically provide the -lpulse flag (which should probably be dynamically picked up). Otherwise, raylib or miniaudio selects the Null playback device. Roll-on cross-ecosystem package management.
Shell forks revisited
Whilst I did get up to some other work last week, I decided to dedicate this weekly to fork-ing in the shell.
Must we fork?
I have spent some time trying to figure out: can you implement a shell that never has to fork and run its own code (as opposed to fork+exec-ing, which is necessary)? There are, at least, two bits of POSIX shell semantics that make this hard: concurrency and redirects, particularly in the presence of shell built-ins.
Individual commands in a pipeline, according to the POSIX specification, will have their standard input and output connected up before other redirections.
The standard input, standard output, or both of a command shall be considered to be assigned by the pipeline before any redirection specified by redirection operators that are part of the command.
Additionally:
...each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment.
Where a subshell is defined as:
Subshell: A shell execution environment, distinguished from the main or current shell execution environment.
We will note that subshell \neq child process. In practice, however, the two are used fairly interchangeably given the semantics of spawning a child process. Perhaps the two most crucial aspects of a given command in a subshell are that:
-
It is run in parallel with the other commands.
-
It should be reentrant.
In what follows we will assume that we are discussing a multi-command pipeline p.
\texttt {p} = \texttt {cmd}_1 | ... | \texttt {cmd}_n \text { where } n \geq 2
Pipeline parallelism
The POSIX specification does not mandate parallelism explicitly for commands in a pipeline. However, in practice the semantics of a pipeline necessitate concurrency at the very least.
Suppose we have \texttt {cmd}_1 | \texttt {cmd}_2 where \texttt {cmd}_1 produces an infinite amount of output (e.g., cat /dev/urandom) and \texttt {cmd}_2 consumes a finite amount of input (e.g., hxd-xxd -l100). What is the result of running this pipeline with the default POSIX shell options?
$ dash -c
"
cat /dev/urandom | hxd.xxd -l100
"
00000000: 5033 7981 ae44 d475 074c 5361 0a9e 3828 P3y..D.u.LSa..8
;
There are three factors that influence the pipeline's execution behaviour (i.e. what is done about the commands that are run and the exit status we return).
-
pipefail: Withpipefailset tofalse(the default), the exit status of the pipeline is entirely decided upon by the exit status of the rightmost command (above\texttt {cmd}_2). Withpipefail, the pipe will exit with 0 if all commands exit with 0; otherwise the non-zero exit status of the rightmost command is taken. -
!: If the pipeline begins with!then this inverts the exit code in the way you might expect. -
All commands must terminate.
Whilst in theory \texttt {cmd}_1 does not terminate, the termination of \texttt {cmd}_2 closes the reading end of the pipe resulting in the Broken pipe system error.
Let's replace \texttt {cmd}_1 with the following compound command that also produces, left to its own devices, infinite output on stdout.
while
echo
"
some bytes
"
;
do
:
done
|
hxd.xxd -l100
In this example, we no longer have the luxury of being able to fork and exec. The compound command (the while loop) forces our hand -- we must rely on the internal execution logic of the shell to keep it going. How then do we prevent \texttt {cmd}_1 from blocking our shell from spawning \texttt {cmd}_2?
For most shells, this is where they might fork themselves and allow the while loop to execute in the child. The parent continuing on. Alternatively, the work could continue on in a separate thread (in OCaml parlance, domain). Fibers (green threads, coroutines etc.) are only possible if there are a sufficient number of co-operative yields (which can be hard to track: any control flow or shell built-in would need to ensure they periodically yield to the other commands in the pipeline).
With that being said, it would seem we have workarounds to ensure the parallel aspects of pipelines are possible to build without forking the shell state itself.
Command Reentrancy
Command reentrancy deals with the fact that commands in a pipeline run in subshells. In practice, this means any \texttt {cmd}_i should not impact the execution environment of any other \texttt {cmd}_j. Commands may certainly interfere with one another: a rogue rm might remove a file that some other command was about to read or was in the middle of reading!
\S 2.13 of the POSIX shell specification describes the execution environment. In Merry we lift this into the ctx type making it completely immutable; the only way to update the context is to destructively declare new fields, e.g., . All shell built-ins work directly from the context as opposed to any state that might also be kept by the process itself (e.g., we do not call getcwd(3) to expand PWD, but instead read it from the context).
Most state is handled nicely by this functional implementation of the shell; except open (or not) file descriptors. The exec built-in is particularly thorny in this regard:
If exec is specified with no operands, any redirections associated with the exec command shall be made in the current shell execution environment.
So, exec 3>&1 says "in the current execution environment map file descriptor 3 to stdout" and exec > /dev/null says "in the current execution environment map stdout to /dev/null. This is often used to silence anything writing to stdout.
info
echo
"
INFO:
$
1
"
>&3
exec
3>&1
>
/dev/null
info
"
Starting up server...
"
echo
"
Some noisy program spamming STDOUT
"
info
"
All is quiet...
"
If we are not running shell built-ins inside new child processes, then we must lift the file descriptor table into the context as well and disallow any two built-ins to run in parallel. Consider this example:
exec
<
./hello.txt
while
read
;
do
echo
"
Got
$
line
"
>&2
done
|
<
;
If we allow exec to change the set of open file descriptors of the shell process directly, then individual commands in the pipeline are no longer re-entrant. The pipeline here is not being used to pipe bytes from one command to another, but instead to expose concurrency and the subshell behaviour.
The second exec < ./world.txt has no impact on the currently opened file on stdin, the while loop continues to read ./hello.txt. This is straightforward to implement with fork as you can dup2 any files you need. The bookkeeping and isolation is all provided by the POSIX process semantics. Without fork, the file descriptors would have to be saved and restored on every suspension and resumption to provide an illusion of isolation.
We could relax our rule to "disallow any two built-ins to run in parallel" by taking a lock on the file descriptor table, but considering the lock is held from the point of resumption until the next suspension it might not achieve very much in practice.
Some of the intricacies of scheduling are also covered in Michael Greenberg's An executable and formal semantics of the POSIX shell.
To motivate the question, consider the following two pipelines:
(1)
while true; do echo 5; done | true(2)
Both pipelines spawn two processes, both of which use shell builtins exclusively: neither of these pipelines needs to make an
execvesystem call (though some systems may implement true or echo as executables, Smoosh and most shells build them in).
Scheduling is an opaque part of the POSIX specification. Indeed, as we have argued here, there is no semantics for scheduling. Here is a similar pipeline run through a script that straces for clone and fork calls on various shells:
It might make an interesting use case for effects in OCaml -- whilst we can certainly piggyback on Eio's existing suspension mechanisms, this is slightly different in that we need to ensure some preconditions are true before resuming certain computations. Fiber-local state is insufficient as programs implicitly depend on global state of the process (in particular the FD table). In this sense, we need some form of suspension-resumption hooks.
