ICFP 2025 [icfp-2025]

October 24, 2025
Patrick Ferris

Two weeks ago I was fortunate enough to attend the International Conference on Functional Programming in Singapore. My first time in Asia and my second time at the conference, what follows are some thoughts and presentations I enjoyed whilst I was there.

I must thank my office mate (and friend!) Ryan Gibb for his camaraderie and sage advice: "the best track is hallway track". Not to mention all of his photos too. Thanks Ryan!

From right to left, top to bottom: Anil Madhavapeddy, Sadiq Jaffer, Patrick Ferris, Ryan Gibb and Roy Ang.

Random Hacking

October 24, 2025
Patrick Ferris

Below are just some of the pieces of random hacking I got up to whilst in Singapore.

OxCaml and Uring [oxcaml-and-uring]

Patrick Ferris

Jane Street were a big presence at ICFP 2025, carting along with them a shiny new OCaml compiler: OxCaml. If have been playing around with OxCaml recently but nothing outside toplevels in Javascript. Until now!

After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change:

  module Heap = struct
    type ptr = int32#
    (* ... *)
  end

  type cqe = {
    user_data_id : Heap.ptr;
    res : int32#;
  }

The idea being that a completion queue entry (a notification that some operation has completed) could be fully represented using 64 bits (two 32-bit, unboxed values). You can see how this impacted the library! I'm not certain about this change (and I'm sure I did it wrong) but it was nice to realise OxCaml gives you this kind of control. However, I am worried about the ergonomics of manipulating values like int32# and the temptation to case it into an int (presumably losing a good portion of the value of having an unboxed value in the first place).

Bibtex Parser

October 24, 2025
Patrick Ferris

On the flight over to Singapore, I started to write a post for this website. Some readers may remember that I use forester after a preprocessing step via graft to build this site. Part of that preprocessing consumes bibtex files and outputs forester trees. For the initial graft release I used the excellent Talaria Bibtex library. Unfortunately, it presented a pretty confusing API that, whilst nicely typed, felt a little cumbersome to use. Additionally, it was too specific in what it would accept as valid Bibtex. I wanted something that was a little more straight-forward to use (if a little fast-and-loose when it comes to types...). And so I built bib.

This was my first real attempt at using the Bytesrw library. I was happy with the results, although I did have to copy a substantial amount of code from other bytesrw-compatible libraries. I'd be happy to shift to this bytes-first world if it wasn't for the ever-present-in-my-mind issue of relocatable values in the OCaml heap!

Geocaml Libraries

October 24, 2025
Patrick Ferris

There has been some excellent outreachy work happening on the geocaml/ocaml-tiff library which is a potential Outreachy internship project for December 2025. More on this in a future post.

On the way back (and a little during the conference), I spent some time building an OCaml counterpart to geotessera, a library for working with Tessera. The library is not ready for people to use directly, but I was working on various constituent parts including:

Modern OCaml bindings to PROJ (a coordinate reference system projection library)
Further work on a WKT library for OCaml
Coming soon... ocaml-geotessera

Talks

October 24, 2025
Patrick Ferris

Hazel of OCaml at TyDe

October 24, 2025
Patrick Ferris

The type-driven development (TyDe) workshop was held on the first day of ICFP at the NUS School of Computing. I find the workshop name to be a little misleading, but perhaps this comes from my engineering background. The talks focused a lot on type theory, presenting the formal mathematics of interesting type systems with less focus on why it might be useful to have such an exotic type system... but then again, that was the nature of the entire conference.

I presented a talk on hazel_of_ocaml. The slides are here, along with the extended abstract. My talk ended up focusing a lot on the pedagogical benefits of having a type system that helped explain the most important thing a type system does... catch type errors! This work represents a small piece of work inside Max Carroll's excellent part II project, which he presented at the HATRA workshop: Decomposable Type Highlighting for Bidirectional Type and Cast Systems.

Scientific Programming at PROPL

October 24, 2025
Patrick Ferris

I think it is important to be honest whenever you get a talk accepted that you otherwise feel should not have been. My talk "what we talk about when we talk about scientific programming" is one such talk. The slides are here. The talk itself was nothing to write home about, but I think there were a few nice ideas that came out of it and the conversations I had with fellow PROPL attenders.

TMF Data Discrepancies Redux

October 24, 2025
Patrick Ferris

On the eve of my talk, I decided to re-run some analysis I did a few years ago on the tropical moist forest dataset, knowing that since then a few more years had become available. After fighting with some bit-rot in the Google Earth Engine script I had used, the new results were not very uplifting.

Land Use Class	Percent Change
Undisturbed	-2.9
Degraded	-5.5
Deforested	8.6
Regrowth	0.5
Water	0.0
Other	-0.6

The data in the table corresponds to the land use class values in Indonesia in the year 2008. The analysis computed the difference across each of the classes between the dataset released in 2021 and the dataset released in 2024. Most notably, the amount of deforested land has gone up by 8.6 percent!

As a somewhat tangential, though related, aside, I had the pleasure of talking to Sam Westrick, Seong-Heon Jung and Sundara Vishnu Satish of the ParCour research group. They were telling me about MaPLe, a programming language from the MLs built for parallelism. We discussed some ideas of using real-world, geospatial algorithms as benchmarks for MaPLe. Michael, if you got this far remind me to chat to you about this!

Shelter Reactions

October 24, 2025
Patrick Ferris

At the end of my talk, I briefly mentioned shelter. By far, this was the topic I got the most engagement on post-talk. During the conference, this was also true when people asked what I was working on. The tag line "a shell with undo" seemed to capture the attention of most people.

I'm feeling a renewed sense of interest in my own work thanks to that, even if the "research" contributions are not very apparent. However, I also had a good discussion with KC about the underlying MRDT in Shelter and how we might think about collaborating in the future on MRDT-related work.

Related to that, I managed to spend the evening in Chinatown with Roly Perera. Roly's work has focused on Fluid, a transparency-based programming language that allows authors to reveal their computation to their user. Excitingly, Roly presented a new text-based part of the language at PROPL. Our work is in the same universe, loosely orbiting some notion of reproducibility and explainability, but quite different in engineering work and how users might actually interact with our systems. I thoroughly enjoyed talking about solutions or research ideas that preserve existing workflows. Roly was great at pulling that out of the current work I have been doing.

Other Talks and Highlights

October 24, 2025
Patrick Ferris

I was a part of, often in a very small way, some other talks that were presented at ICFP including

Functional Networking for Docker, I encourage you to read Anil's notes on the topic. It is also worth mentioning Anil's post-posix talk (not unrelated to my Uring musings above).
Michael gave an excellent, very live, talk about Yirgacheffe at PROPL which I thoroughly enjoyed. I'm also excited about his recent developments which I'm sure you can clue into over on his blog.

Below is a non-exhaustive list of talks I enjoyed whilst at the conference. Some relevant, most of them not!

By far, one of the best talks was Polynomial-time Program Equivalence for Machine Knitting. I think the talk really nailed the "here's the idea, the details are in the paper"-presenting style as they described an "...algorithm that canonicalizes the algebraic representations of the topological semantics of machine knitting programs".
I attended most of the OxCaml tutorial presented by the incredibly friendly Gavin Gray.
Ryan's talk Spatial Programming for Environmental Monitoring at PROPL was excellent.

There's plenty more to talk about, and I'm sure my future posts will reference other parts of my experience at ICFP 2025. Thanks for reading, and if anybody is interested in any aspects of this do reach out!

References

Reference. TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis [tessera2025]

2025
Zhengpeng Feng, Clement Atzberger, Sadiq Jaffer, Jovana Knezevic, Silja Sormunen, Robin Young, Madeline C. Lisaius, Markus Immitzer, David A. Coomes, Anil Madhavapeddy, Andrew Blake, Srinivasan Keshav
10.48550/arXiv.2506.20380
http://arxiv.org/abs/2506.20380

Abstract

2025
Zhengpeng Feng, Clement Atzberger, Sadiq Jaffer, Jovana Knezevic, Silja Sormunen, Robin Young, Madeline C. Lisaius, Markus Immitzer, David A. Coomes, Anil Madhavapeddy, Andrew Blake, Srinivasan Keshav

Petabytes of satellite Earth Observation (EO) data are freely available and can address critical global challenges. However, EO data quality is poor due to clouds and variable lighting conditions. To address this, practitioners typically use compositing, but this critically removes the temporal phenological signal. Moreover, supervised machine learning to map composited pixels to task-specific classes requires accurately labelled data that are rarely available. We present TESSERA, a pixel-oriented foundation model for EO time series that creates 128-dimensional latent embeddings requiring only a few labels for task-specific training to achieve state-of-the-art performance across diverse complex tasks. TESSERA uses two encoders that combine optical data with synthetic aperture radar backscatter coefficients at 10m resolution, creating embeddings fused with a multilayer perceptron to generate annual global embedding maps. TESSERA closely matches or outperforms state-of-the-art task-specific models and other foundation models across five diverse downstream tasks. It is unprecedented in ease of use, scale, and accuracy: no other open foundation model provides precomputed outputs with global, annual coverage at 10m resolution.

Context

Posts [posts]

Occasionally I write posts, mostly about OCaml or something related.

An RSS feed is available.

Merging pure shell histories [merging-pure-shell-histories]

September 3, 2026
Patrick Ferris

We define a pure shell history as an execution trace from a shell that only changes the shell's execution environment, that is, there are no external side-effects. In this article, we will reason about merging independent pure shell histories.

We can make a simplifying assumption that our shell executes commands on a readonly filesystem without any network connectivity. So for example, we could have the following script:

export DOWNLOAD_DIR=/home/bactrian/downloads
export PATH=/bin

exit () {
  echo "Goodbye!"
}

trap exit EXIT

analyse $DOWNLOAD_DIR

The shell execution environment is state managed by the shell. It may be materialised concretely in the shell's runtime (e.g. a map of function names to function definitions) or managed by the kernel in the process state of the shell (e.g. open file descriptors).

Each pure shell command has a dynamic dependency cone. For simplicity, we will restrict this dependency cone to just the shell execution environment excluding resources like files or the network. When a command is run, we can evaluate that dependency cone using a custom eBPF program in addition to information provided by the shell (see Merry).

Only shell built-ins can modify the shell execution environment; therefore these operations are known by their constructor. For example, export FOO modifies the variable FOO to be marked as exported — in general it is not known statically what precise modifications are made due to, for example, word expansion or the dynamic lookup of functions. An executable cannot perform such a modification on the shell execution environment.

We can think of pure shell scripts in terms of a mergeable replicated data type.

Definition. Mergeable Replicated Data Type Implementation [mrdts]

Patrick Ferris

A mergeable replicated data type (MRDT) implementation for a data type $\tau$ is a tuple $D_{\tau } = (\Sigma , \sigma _{0}, do, merge)$ where:

$\Sigma$ is the set of all possible states at a branch,
$\sigma _{0} \in \Sigma$ is the initial state,
$do : Op_{\tau } \times \Sigma \times Timestamp \rightarrow \Sigma \times Val_{\tau }$ implements every data type operation,
$merge : \Sigma \times \Sigma \times \Sigma \rightarrow \Sigma$ implements the three-way merge strategy.

Definition 2.1 from "Certified Mergeable Replicated Data Types".

Treating $\tau$ as the shell execution environment and $Op_{\tau }$ as pure shell commands we can reason about our shell using MRDTs.

There are some nice properties that fall out of algebraic properties of the operations that are allowed on your data type. For instance, commutative operations.

Notes

September 3, 2026
Patrick Ferris

Orthogonal MRDT operations — where it is possible to compute a single operation that lead to a particular state.

A Shell is Born [a-shell-is-born]

June 17, 2026
Patrick Ferris

Merry passed a milestone this week after installing the following packages vim fakeroot fakechroot debootstrap and successfully performing a fakechroot debootstrap.

To me, this signals that Merry is closer than ever to being an actual POSIX-ish shell. This is exciting because we now have a completely customisable, OCaml implementation of a shell that can run a significant number of existing shell scripts in the world. Let the hacking commence!

Installation

June 17, 2026
Patrick Ferris

You can install the binary, msh, today:

opam pin git+https://git.sirref.org/merry
eval $(opam env)
msh

What's next?

June 17, 2026
Patrick Ferris

There are a few avenues to try next. In the short-term, more testing of the shell is probably worthwhile. In particular, trying to boot an Alpine Linux image with msh as the default shell would be good.

Integrating the various components of Shelter is also on the list. A time-travelling POSIX shell is within reach; the actual semantics still need to be ironed out.

I know for certain that some shell options and built-ins will be the first issues to arise going forward.

As a fun experiment, Anil's vibecoded OCaml busybox translation could be plumbed into msh. This would allow a very versatile, but zero-forking shell! Throw in an in-memory Eio filesystem into the mix and the whole thing becomes in-memory... mirageSH.

Weeknotes

June 17, 2026
Patrick Ferris

Below are a collection of weeknotes for Merry.

Shell Benchmarks, Back to Writing [weekly-2026-w26]

July 4, 2026
Patrick Ferris

A little late due to travel, but I did a little benchmarking of Merry.

Signal Handling in Merry

July 4, 2026
Patrick Ferris

In order to even run shellbench, I first had to fix some bugs in the shell implementation. Shellbench aims to warm up the shell before starting any benchmarks; it includes this prelude before getting started:

__ready=
trap __ready=1 HUP
kill -HUP "$MAIN_PID"
until ["$__ready" ]; do __dummy=; done

Whilst writing Merry, I hadn't really considered that the return context from a trap needed to be taken into consideration! I had assumed they would be silently ignored, how else would we know where to thread that context back into the shell's execution.

Even after reading through the specification for trap I am still none the wiser. I've opted to place some sensible signal polling points during the execution of the commands to run any signals that have come in. The single handlers in Merry simply push thunks onto a run queue. Looking at the Dash source code, it seems to be something that happens just before evaluation.

Benchmarking Merry [merry-benchmarks]

July 4, 2026
Patrick Ferris

Medium-to-small potatoes

July 4, 2026
Patrick Ferris

I used shellbench to understand where some of the performance pitfalls were coming from.

$ ./shellbench -s bash,zsh,msh sample/func.sh
-----------------------------------------------------------------------------------
name                                                     bash        zsh        msh
-----------------------------------------------------------------------------------
func.sh: no func                                      775,539  1,124,485      5,229
func.sh: func                                         398,861    197,169     93,424
-----------------------------------------------------------------------------------
* count: number of executions per second

This particular benchmark first runs nothing (i.e. :) and then a function lookup and execution. Surprisingly msh is slower at doing nothing! I perf-ed a run of msh test.sh with just a single : in the file:

Test built-in

July 4, 2026
Patrick Ferris

After making various improvements to Merry that almost halved the memory usage (e.g., removing extraneous string manipulations, adding a fast-path to the PATH variable for resolving programs). I was still pretty far off the performance of these other shells. I decided to have a look at dash which has a simple implementation.

This flame graph is for the following shell script:

f () { :; }
while __count=$(($__count+1)) && ["$__count" -le 10000 ]; do
	f
done
echo "count is $__count"

And here are some rudimentary benchmarks:

$ time -p dash test.sh
count is 10001
real 0.01
user 0.01
sys 0.00
$ time -p msh test.sh
count is 10001
real 15.53
user 4.28
sys 12.28

It turns out the massive slowdown is very simple to explain. In the dash flamegraph there is a oexpr stack; which comes from the bltin/test.c file. dash, along with the other shells, does not spawn a process for the [-test command! Forcing dash to do use the binary [ gives:

$ time -p dash test.sh
count is 10001
real 7.55
user 3.95
sys 3.44

Which is much more respectable in terms of performance compared to msh. Plenty of work to do though!

Parsing arithmetic expressions

July 4, 2026
Patrick Ferris

There was some obvious low-hanging fruit to help improve some of the performance differences. The first was pre-parsing arithmetic expressions. Morbig, the static parser, does not implement arithmetic expression parsing; so Merry does it for us. Before, it was on-demand, as we encountered an expression of the form $((<e>)). Now, it happens up front. This should improve tight-loops.

Writing

July 4, 2026
Patrick Ferris

More on this soon, but I have started writing two separate papers bringing all of this work together.

Three weeks in the Shell [weekly-2026-w25]

June 16, 2026
Patrick Ferris

The last three weeks have been focused work on building Merry. In practice, I wanted to run the following two commands through Merry's shell: msh:

apt-get install debootstrap -y
debootstrap stable /stable-chroot http://deb.debian.org/debian/

Seemingly innocuous. But both commands run thousands of lines of shell script. Whether that's Debian package maintainer scripts or state-machines for processing package release files.

What follows is a handful of the bugs and the shell scripts that caused them.

Field Splitting

June 16, 2026
Patrick Ferris

Edge-cases abound in the POSIX shell specification; and field splitting is one of the most serious perpetrators. Field splitting happens after word expansion. It is a known footgun amongst shell script authors.

$ count () { echo "Count: $#"; }; FOO="hello world"; count $FOO
Count: 2

Quoting variables that need expanding will prevent field splitting from taking place, but it is an easy thing to forget.

Field splitting uses the IFS (internal field separator) environment variable to determine how to do the splitting, and it comes with some extra rules beyond "split on these characters".

$ IFS=" "; FOO=" hello    world"; echo $FOO
hello world
$ IFS=" "; FOO=" hello    world"; echo "$FOO"
 hello    world
$ IFS=":"; FOO="a:b:c:"; echo $FOO
a b c
$ IFS=":"; FOO="a:b::c:d"; echo $FOO
a b  c d
$ IFS=":"; FOO=":a:b:c"; echo $FOO
 a b c
$ IFS=; FOO="unset   IFS"; echo $FOO
unset   IFS

Variable assignment

June 16, 2026
Patrick Ferris

How a shell stores variables also impacts field splitting. Naively, Merry stored variables as strings, but really they should be stored as their expanded form.

Consider this example of saving a function output in a variable called base.

iter () {
 echo "a"
 echo "b"
 echo "c"
}

base=$(iter)

for n in $base; do
  echo "Got $n"
done

for n in "$base"; do
  echo "Got $n"
done

Later, when we come to iterate across the fields of $base we wouldn't be able to reconstruct the intial fields that $(iter) produced if we simply saved the result as a string. Too much information would have been lost.

Redirections

June 16, 2026
Patrick Ferris

Redirections control what file descriptors point to. Each redirection has a lifetime associated with it. For example: cat < README.md really says "open README.md and execute the cat program mapping the process's FD 0 to the FD of README.md". When cat returns that redirection is no longer valid.

Other redirections are not so simple. Consider:

$ echo "!dlrow olleh" > input.txt
$ (read line; echo $line) < input.txt | rev
hello world!

There are lots of redirections here too. Firstly the standard input for everything inside the braced command (read line2; echo $line) should be the open file descriptor for input.txt. The standard output of the braced command should be the writing end of a pipe whose reading end is passed to rev as that process's standard input. The final standard output for rev should be unchanged from what it was originally (the shell's standard output).

Getting redirections right is hard, consider this snippet from the debootstrap scripts (all in the context of stdout and stderr pointing to a debootstrap.log file and file descriptor 4 pointing at the true stdout).

for m in $MIRRORS; do
    pkgdest="$TARGET/$($DLDEST pkg "$s" "$c" "$a" "$m" "$path")"
    if [! -e "$pkgdest" ]; then continue; fi
    pkgs_to_get="$(download_debs "$m" "$pkgdest" $pkgs_to_get 5>&1 1>&6)"
    if [ -z"$pkgs_to_get" ]; then break; fi
done 6>&1

More niche built-ins

June 16, 2026
Patrick Ferris

As the dust begins to settle on the nuts and bolts of implementing a POSIX-ish shell, new challenges are allowed to emerge from deep within scripts.

One such challenge is the seemingly endless (I know, they're finite) shell built-ins! Whilst trying to run debootstrap using its fakechroot variant, I bumped into getopts... a whole command-line parsing shell built-in.

An In-memory Filesystem for Eio [eio_mem]

June 16, 2026
Patrick Ferris

Towards the end of last week and the beginning of this week, I threw together a prototype for an in-memory filesystem for Eio: Eio_mem. Concretely, this provides a means for constructing an Eio.Fs.dir_ty Eio.Path.t value that is completely in-memory.

Thanks to Eio's capability-first approach, any existing code implemented against the _ Eio.Path.t API should just work with the in-memory filesystem.

There's a convenient Eio_mem.Fs.load function that can take a path from one filesystem (most likely a normal one from Eio_linux or Eio_posix) and replicate it into the in-memory filesystem. For example, you might have a static web-server pointing to a directory like "/var/www". The following might be the only change you need to make it completely in-memory (example from ocaml-cohttp):

 let () =   let port = ref 8080 in
   Arg.parse
     [ ("-p", Arg.Set_int port, " Listening port number(8080 by default)") ]     ignore "An HTTP/1.1 server";
   Eio_main.run @@ fun env ->
+  Eio_mem.run @@ fun menv ->
   Eio.Switch.run @@ fun sw ->
-  (* Restrict to current directory: *)
-  let htdocs = Eio.Stdenv.cwd env in
+  (* Restrict to in-memory current directory: *)
+  Eio_mem.Fs.load ~filter (Eio.Stdenv.cwd env);
+  let htdocs = Eio.Stdenv.cwd menv in
   let socket =
     Eio.Net.listen env#net ~sw ~backlog:128 ~reuse_addr:true
       (`Tcp (Eio.Net.Ipaddr.V4.loopback, !port))

The keen-eyed reader might have noticed the filter parameter. This can be used to ignore certain paths (here I ignored .git and _build which were adding substantial time to loading the filesystem).

Using meio to debug Merry [weekly-2026-w23]

June 4, 2026
Patrick Ferris

During a large rewrite of Merry, I introduced a regression causing certain shell programs to hang indefinitely. I thought it might be a good opportunity to try out Meio.

First, let's look at a trace of Meio from some real-world programs. For example: meio -- msh -c "sleep 100".

One problem that comes up immediately from this example is that the tree hierarchy of tasks and cancellation contexts makes it tricky to reason about the logging. Here, we can see the pipeline's cancellation context clearly labelled pipeline-id1. This is an Eio.Switch.t for the pipeline sleep 100. The actual execution log for sleep 100 bubbles back up to the main fiber (that has no name, so it is called <task>). The daemon fiber, for ensuring the spawned process is reaped, is attached to the switch and is displayed below it.

Meio has a long way to go. Most programs are too fast to be monitored in this way. Using something like tokio-console's model might be a better idea here focusing less on the tree strucuture of the program, and more on the performance and monitoring of individual tasks. Without this, it turns out Meio was not all that useful for debugging this specific issue (which ended up being some bad pipe management on my part).

Shell forks revisited [weekly-2026-w17]

April 28, 2026
Patrick Ferris

Whilst I did get up to some other work last week, I decided to dedicate this weekly to fork-ing in the shell.

Must we fork?

April 28, 2026
Patrick Ferris

I have spent some time trying to figure out: can you implement a shell that never has to fork and run its own code (as opposed to fork+exec-ing, which is necessary)? There are, at least, two bits of POSIX shell semantics that make this hard: concurrency and redirects, particularly in the presence of shell built-ins.

Individual commands in a pipeline, according to the POSIX specification, will have their standard input and output connected up before other redirections.

The standard input, standard output, or both of a command shall be considered to be assigned by the pipeline before any redirection specified by redirection operators that are part of the command.

Additionally:

...each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment.

Where a subshell is defined as:

Subshell: A shell execution environment, distinguished from the main or current shell execution environment.

We will note that subshell $\neq$ child process. In practice, however, the two are used fairly interchangeably given the semantics of spawning a child process. Perhaps the two most crucial aspects of a given command in a subshell are that:

It is run in parallel with the other commands.
It should be reentrant.

In what follows we will assume that we are discussing a multi-command pipeline $p$ .

$\texttt {p} = \texttt {cmd}_1 | ... | \texttt {cmd}_n \text { where } n \geq 2$

Pipeline parallelism

April 28, 2026
Patrick Ferris

The POSIX specification does not mandate parallelism explicitly for commands in a pipeline. However, in practice the semantics of a pipeline necessitate concurrency at the very least.

Suppose we have $\texttt {cmd}_1 | \texttt {cmd}_2$ where $\texttt {cmd}_1$ produces an infinite amount of output (e.g., cat /dev/urandom) and $\texttt {cmd}_2$ consumes a finite amount of input (e.g., hxd-xxd -l100). What is the result of running this pipeline with the default POSIX shell options?

$ dash -c "cat /dev/urandom | hxd.xxd -l100"
00000000: 5033 7981 ae44 d475 074c 5361 0a9e 3828  P3y..D.u.LSa..8(
00000010: f88c 1f9e 3b0e bcde 528a f78c b1e4 7195  ....;...R.....q.
00000020: 16f2 8325 c90b 9310 539c af04 8289 c3a0  ...%....S.......
00000030: 57c5 680a 2c81 a999 71b6 708e 95df e720  W.h.,...q.p....
00000040: 004c 407a 8365 9448 65c6 b435 bed2 4084  .L@z.e.He..5..@.
00000050: 62ea 96ab 2b23 0385 f9d2 fc30 d090 1719  b...+#.....0....
00000060: 68d1 c514                                h...
cat: write error: Broken pipe

There are three factors that influence the pipeline's execution behaviour (i.e. what is done about the commands that are run and the exit status we return).

pipefail: With pipefail set to false (the default), the exit status of the pipeline is entirely decided upon by the exit status of the rightmost command (above $\texttt {cmd}_2$ ). With pipefail, the pipe will exit with 0 if all commands exit with 0; otherwise the non-zero exit status of the rightmost command is taken.
!: If the pipeline begins with ! then this inverts the exit code in the way you might expect.
All commands must terminate.

Whilst in theory $\texttt {cmd}_1$ does not terminate, the termination of $\texttt {cmd}_2$ closes the reading end of the pipe resulting in the Broken pipe system error.

Let's replace $\texttt {cmd}_1$ with the following compound command that also produces, left to its own devices, infinite output on stdout.

while echo "some bytes"; do
  :
done | hxd.xxd -l100

In this example, we no longer have the luxury of being able to fork and exec. The compound command (the while loop) forces our hand -- we must rely on the internal execution logic of the shell to keep it going. How then do we prevent $\texttt {cmd}_1$ from blocking our shell from spawning $\texttt {cmd}_2$ ?

For most shells, this is where they might fork themselves and allow the while loop to execute in the child. The parent continuing on. Alternatively, the work could continue on in a separate thread (in OCaml parlance, domain). Fibers (green threads, coroutines etc.) are only possible if there are a sufficient number of co-operative yields (which can be hard to track: any control flow or shell built-in would need to ensure they periodically yield to the other commands in the pipeline).

With that being said, it would seem we have workarounds to ensure the parallel aspects of pipelines are possible to build without forking the shell state itself.

Command Reentrancy

April 28, 2026
Patrick Ferris

Command reentrancy deals with the fact that commands in a pipeline run in subshells. In practice, this means any $\texttt {cmd}_i$ should not impact the execution environment of any other $\texttt {cmd}_j$ . Commands may certainly interfere with one another: a rogue rm might remove a file that some other command was about to read or was in the middle of reading!

$\S 2.13$ of the POSIX shell specification describes the execution environment. In Merry we lift this into the ctx type making it completely immutable; the only way to update the context is to destructively declare new fields, e.g., { ctx with cwd = "/home/bactrian" }. All shell built-ins work directly from the context as opposed to any state that might also be kept by the process itself (e.g., we do not call getcwd(3) to expand PWD, but instead read it from the context).

Most state is handled nicely by this functional implementation of the shell; except open (or not) file descriptors. The exec built-in is particularly thorny in this regard:

If exec is specified with no operands, any redirections associated with the exec command shall be made in the current shell execution environment.

So, exec 3>&1 says "in the current execution environment map file descriptor 3 to stdout" and exec > /dev/null says "in the current execution environment map stdout to /dev/null. This is often used to silence anything writing to stdout.

info () {
  echo "INFO: $1" >&3
}

exec 3>&1 > /dev/null
info "Starting up server..."
echo "Some noisy program spamming STDOUT"
info "All is quiet..."

If we are not running shell built-ins inside new child processes, then we must lift the file descriptor table into the context as well and disallow any two built-ins to run in parallel. Consider this example:

exec < ./hello.txt

while read line; do
  echo "Got $line" >&2
done | (exec < ./world.txt; cat)

If we allow exec to change the set of open file descriptors of the shell process directly, then individual commands in the pipeline are no longer re-entrant. The pipeline here is not being used to pipe bytes from one command to another, but instead to expose concurrency and the subshell behaviour.

The second exec < ./world.txt has no impact on the currently opened file on stdin, the while loop continues to read ./hello.txt. This is straightforward to implement with fork as you can dup2 any files you need. The bookkeeping and isolation is all provided by the POSIX process semantics. Without fork, the file descriptors would have to be saved and restored on every suspension and resumption to provide an illusion of isolation.

We could relax our rule to "disallow any two built-ins to run in parallel" by taking a lock on the file descriptor table, but considering the lock is held from the point of resumption until the next suspension it might not achieve very much in practice.

Some of the intricacies of scheduling are also covered in Michael Greenberg's An executable and formal semantics of the POSIX shell.

To motivate the question, consider the following two pipelines:
(1) while true; do echo 5; done | true
(2) while true; do echo 5; done | { read x; echo $((x+42)); }
Both pipelines spawn two processes, both of which use shell builtins exclusively: neither of these pipelines needs to make an execve system call (though some systems may implement true or echo as executables, Smoosh and most shells build them in).

Scheduling is an opaque part of the POSIX specification. Indeed, as we have argued here, there is no semantics for scheduling. Here is a similar pipeline run through a script that straces for clone and fork calls on various shells:

% ./test/check_fork.sh "while echo 5; do : ; done | true"
bash  3
dash  3
yash  1 
zsh   1
ksh   1
ash   3

It might make an interesting use case for effects in OCaml -- whilst we can certainly piggyback on Eio's existing suspension mechanisms, this is slightly different in that we need to ensure some preconditions are true before resuming certain computations. Fiber-local state is insufficient as programs implicitly depend on global state of the process (in particular the FD table). In this sense, we need some form of suspension-resumption hooks.

Forking in Shells & Library Maintenance [weekly-2026-w14]

April 3, 2026
Patrick Ferris

This week, Merry got close to executing apt-get install ca-certificates and running Debian's debootstrap scripts correctly. Nathan and I released Ppxlib 0.38.0, I added support to ocaml-proj for compiling to the browser and I fixed a long-standing bug in hilite.

Forking in Merry

April 3, 2026
Patrick Ferris

So far, Merry has managed to support many features of the POSIX shell specification without needing to do a fork(2) without an exec. Or, to put it another way, all the forking that Merry does, will only be followed by running C code. This is actually a feature of Eio's Process API. It is required to make process execution safe in the context of multiple domains in OCaml (mostly due to ensuring consistency within the garbage collector). However, it is perfectly okay to fork and run OCaml code provided there is only a single domain.

In a shell, many features of the shell language require that commands (or built-ins, function applications etc.) execute within a subshell. The specification does not go into detail about how shells should implement this, but many choose to fork in order to preserve some state in the parent (for example, the file descriptor table). Some shells try to minimise the number of forks as an optimisation. The korn shell (ksh) is one such shell.

Using the notation $(command) will cause command to execute in a subshell of the current ksh. In many instances, ksh will not actually fork/exec a subshell when command is a built-in or a shell function.

By trying to adhere to Multicore OCaml's "thou shalt not fork" commandment, Merry is much more similar to ksh in this regard. However, whenever there is an interaction with a shell built-in and semantics that need a child process, things get tricky very quickly. Consider the following:

while echo "hello"; do
  :
done | head -n 3

We have a mix of shell built-ins, normal commands, a compound command (the while loop) and a pipeline. Each individual command in the pipeline requires you to run them in a subshell and set them up before executing them. Without a fork for that first command, you may end up looping forever which is not the intended behaviour here. I had a good conversation with Michael about the implications of this for reproducibility.

For now, I have started implementing a fork for some of the shell features whilst still trying to maintain the functional core of the implementation.

An OCaml 5.5 compatible Ppxlib

April 3, 2026
Patrick Ferris

Nathan and I released an OCaml 5.5 compatible version of ppxlib.

Ppxlib Releases › 0.38.0 [ppxlib-release-0-38-0]

March 30, 2026
Patrick Ferris

Nathan and I released ppxlib.0.38.0 last week. Its main feature is full migration support for the upcoming OCaml 5.5 compiler (currently in its alpha3 release). This means supporting the handful of new features landing in OCaml 5.5: modular explicits, external type declarations and arbitrary "local" structure items.

As Nathan and I continue to find a plausible maintenance story for ppxlib, Nathan has opened an issue on the OCaml compiler to discuss the idea of adding additional extension points to the language.

This comes from the desire to be able to store encoded versions of new OCaml features inside older abstract syntax trees.

There are also some nice bug fixes in there too:

A potential OOM loop has now been removed.
Locations have been restored to long identifiers!

Browser support for ocaml-proj [ocaml-proj-virt]

April 3, 2026
Patrick Ferris

A while ago I built some "modern" bindings to PROJ4 in OCaml. After reading Jon Ludlam's weeknotes (and speaking with him and Anil, I thought it might be nice to add a Javascript backend to those bindings). This was relatively straight-forward using Dune's virtual libraries and is available on Github.

With virtual libraries, your own data analysis could (if you wished) depend solely on the proj library and later choose to either link it with proj.c or proj.js depending on where the analysis is being deployed.

I briefly looked at WASM support, but quickly realised there was not much appetite for it and trying to compile around the C FFI was going to be hard.

Hilite updates

April 3, 2026
Patrick Ferris

Since I helped relaunch the ocaml.org website a few years ago, I have maintained hilite, a tool for doing build-time syntax highligting.

It is being used on the ocaml.org website, in yocaml_markdown and also in Michael's webplats (when it works...).

And it was Michael who led me to finally fixing and adding syntax highlighting support for Python and Go. Once I was on this roll, I finally took a look at fixing the long-standing bug of trying to highlight ocaml-mdx code, like the following:

# 50.123;;
# 1 "file.ml";;

In fact, this code now highlights fine, but was the source of the bug. The syntax highlighting grammer confuses the ocaml-mdx # as a toplevel directive and trouble ensues.

Ppxlib release and Merry updates [weekly-2026-w13]

March 30, 2026
Patrick Ferris

Ppxlib Releases › 0.38.0 [ppxlib-release-0-38-0]

March 30, 2026
Patrick Ferris

As Nathan and I continue to find a plausible maintenance story for ppxlib, Nathan has opened an issue on the OCaml compiler to discuss the idea of adding additional extension points to the language.

This comes from the desire to be able to store encoded versions of new OCaml features inside older abstract syntax trees.

There are also some nice bug fixes in there too:

A potential OOM loop has now been removed.
Locations have been restored to long identifiers!

Merry updates

March 30, 2026
Patrick Ferris

It turns out writing a POSIX(ish) shell is hard; at least, there is a vast number of slightly obscure features that one needs to support. I think this is surprising because most people writing shell scripts write simple shell scripts; scripts that use a much smaller subset of features. In the same breath, those same developers are likely to use something like apt-get install bash which runs a plethora of more advanced (and non-POSIX) shell scripts!

Exec redirects

March 30, 2026
Patrick Ferris

In a shell script, you can use the built-in command exec to replace the current process with a new one (e.g. exec vim). However, there is a different mode of operation for exec:

If exec is specified with no operands, any redirections associated with the exec command shall be made in the current shell execution environment.

One of my litmus tests for Merry is the Debian debootstrap scripts (h/t Anil). One thing that it does is the following:

err () {
  printf "err $1" >&4
}

exec 4>&1

A lot of detail has been elided for clarity. exec 4>&1 sets up a redirection for the shell's execution environment in which file descriptor 4 is now an alias for standard output. So, writing to 4 (by redirecting a command's standard output to 4 i.e. >&4) will output to where standard output is going (most likely the terminal).

For this to work, shell's must dup2 the relevant file descriptors which have the following condition:

If the file descriptor newfd was previously open, it is closed before being reused; the close is performed silently (i.e., any errors during the close are not reported by dup2()).

This is all very well, unless your program has an important file already open that happens to have file descriptor 4. The Eio Linux and POSIX backends suffer from this problem. Both make use of a file-based synchronisation mechanism for waking up the eventloop should another domain push a completion to the scheduler's run queue. On Linux this is via an eventfd and in POSIX, a pipe.

This dup2 will close the eventfd and will likely grind Eio to a halt (or an assert false). For now, I have resorted to vendoring Eio and moving the eventfd file descriptors to higher values, though I have opened a PR to make this more configurable. This bug is quite easy to write up... it was not so easy to find!

As a random example, consider the debconf/confmodule script which offers very little room for a buggy implementation!

Outreachy

March 30, 2026
Patrick Ferris

The presentations from the demonstrations for this round of Outreachy are now online!

Write support in OCaml TIFF library [outreachy-ocaml-tiff]

February 9, 2026
Patrick Ferris

I am mentoring Tambe Salome during the December 2025 Outreachy round to add support for writing TIFF files in the ocaml-tiff library.

You can now see the video of the demonstration day presentation:

A POSIX Shell in OCaml [weekly-2026-w12]

March 19, 2026
Patrick Ferris

Long time, no weekly. Since the start of this year I have been building a POSIX shell in OCaml called msh (with the underlying library called Merry). Merry is available online now.

A POSIX(ish) shell in OCaml

March 19, 2026
Patrick Ferris

Merry is a POSIX(ish) in OCaml. It uses Eio alongside Morbig (a static parser for POSIX shell).

Why another (POSIX) shell?

March 19, 2026
Patrick Ferris

Shells have been around for a long time. In my research on the notion of metashell I wrote about Louis Pouzin originally coining the term:

Metashells › Louis Pouzin's "SHELL" [pouzin-shell]

May 19, 2025
Patrick Ferris

I spent some time reading part of the multics design documentation this week. Louis Pouzin coined the term "SHELL" in this document, and I was reminded yet again just how important it is to be a good writer even as a "computer science researcher". For example, this excerpt from the requirements section of the document

The previous definitions imply that a command MUST be designed while keeping in mind the user, sitting at his console, wondering about what might be going on, mistyping or forgetting arguments, even if fully aware of the conventions, and possibly interfering with the command by hasty quits, carriage returns, and other temperamental reactions.

And then later, when defining the "SHELL".

We may envision a common procedure called automatically by the supervisor whenever a user types in some message at his console, at a time when he has no other process in active execution under console control (presently called command level). This procedure acts as an interface between console messages and subroutine. The purpose of such a procedure is to create a medium of exchange into which one could activate any procedure, if it were called inside of another program. Hereafter, for simplification, we shall refer to that procedure as the "SHELL".

It still surprises how little the undergraduate degree in computer science at Cambridge focuses on writing skills.

I built Shelter exploring the idea of a shell-like interface that allowed users to undo their shell actions (amongst other cool tricks). Unfortunately shell-like is not enough. Shelter cut many corners to masquerade as a shell (e.g. appending env to understand how a command may have altered the execution environment). I felt it was necessary to make Shelter a "SHELL"! To do that, I needed a solid foundation to build on.

msh, the POSIX shell that comes with Merry, is by no means POSIX-complete in terms of features. But it is my daily driver at this point. It includes a pure OCaml rewrite of linenoise (a small, self-contained alternative to the venerable readline) called bruit.

If you have docker installed on your machine, you can take it for a spin today:

docker run -it --rm patrickferris/msh

The patrickferris/msh docker image is just for trying it out. It is based on the OCaml 5.3 alpine image.

Alternatively, you can build msh from source and have it available in your opam switch.

opam pin git+https://tangled.org/patrick.sirref.org/merry

There are many small paper cuts left to patch over, but most of it is porcelain (e.g. ctrl+left-arrow for moving in bruit). Unfortunately, these are the kinds of things that you will immediately stumble upon.

What makes Merry different?

March 19, 2026
Patrick Ferris

Nothing.

Merry is supposed to be a solid, POSIX-ish base to build on. Unfortunaltely, as it turns out, the subset of features from the POSIX specification that people actually use... is pretty much all of it. Every possible redirection, variable expansion, shell built-in and compound command make some appearance. Not to mention the non-POSIX bits of shell we all take for granted (e.g. &>-redirection).

Merry and msh are useable. You will most likely find bugs if you use them. If, when using msh, you find something obscure happening you can enable debug mode either my setting the variable MSH_DEBUG or by invoking msh with -v -v.

What's next?

March 19, 2026
Patrick Ferris

It is soon time to combine Shelter and Merry into the time-travelling, POSIX-ish shell that I have been trying to build since I first started working on Shelter.

For this to be successful, I need Merry to be able to pass plenty of tests and right now that involves trying to install plenty of packages using tools like apk and apt.

Shell MRDT

March 19, 2026
Patrick Ferris

What I am particularly interested in reasoning about, is the execution context in Merry. This value, alongside the file-system, constitutes a fairly deep understanding of the state that changes in each step of a shell's evaluation loop.

This was, in terms of Shelter, the missing piece for truly building some kind of MRDT across shell sessions.

TIFF in OCaml

March 19, 2026
Patrick Ferris

I picked up from the excellent Outreachy work of Tambe Salome in getting write-support in ocaml-tiff. We are getting closer to the kind of API I envisaged in this latest round of refinement and review.

let ( / ) = Eio.Path.( / )

let checkerboard ~size =
  let v = Nx.zeros Nx.uint8 [| size; size |] in
  for row = 0 to size - 1 do
    for col = 0 to size - 1 do
      if (row + col) mod 2 = 0 then Nx.set_item [ row; col ] 254 v
    done
  done;
  Nx.to_bigarray v

let () =
  Eio_posix.run @@ fun env ->
  Tiff_eio.with_open_out (env#cwd / "example.tiff") @@ fun w ->
  let tif = Tiff.make (checkerboard ~size:256) in
  Tiff.to_file tif w

I have also been extremely pleased to see further external collaborators appearing:

Nicolas helping out with metadata maintainence.
Virgile adding support for reading multi-image TIFF files.

Merry Shell(e)y [weekly-2025-w50]

December 15, 2025
Patrick Ferris

Aside from a serious amount of admin, last week I decided to spend a majority of my time thinking about shells.

What is a shell?

December 15, 2025
Patrick Ferris

There is no single definition of what a shell is or isn't.

AI Blues [ai-blues]

December 15, 2025
Patrick Ferris

Undoubtedly last week, I suffered a severe case of "AI blues" (for lack of a better description).

I have been doing some thinking, trying to distil my own thoughts on what aspects of vibecoding seem to irk me so much. Ultimately, it seems it is more about attitudes than anything else.

Exploitative

December 15, 2025
Patrick Ferris

Increasingly, I am finding that other people's vibecoded work is my problem. I have been an active participant in OCaml's outreachy work for going on five years now, and a problem mentors have to face is deciding the appropriate amount of help to give to your intern when they come to you with questions. More than likely, you could answer with specifics and solve the problem at hand. However, this doesn't scale very well and doesn't help your intern learn where else they may look for answers (e.g. Dune's documentation, a project's README page, OCaml's package explorer, heck even just using some simple CLI tools!) or how they might go about debugging a certain problem.

LLMs, in my experience, make this problem worse. When they come up short, they can leave their programmer higher and drier than they would be had they hit a problem on their own. There is an inertia to understanding problems now that there's lots of code lying around. Turning to your colleagues has a few problems that it didn't before:

Your colleagues don't know whose understanding they are correcting: yours or the LLMs?
If it is the latter, there seems to me to be an exploitative quality to even asking for help until you yourself know what's going wrong.

Technological Accelerationism

December 15, 2025
Patrick Ferris

There is no doubt that LLMs are pretty good at writing large amounts of okay code. The generation of this code, and often subsequent publishing of it, completely outpaces the ability to maintain it.

Programmers may be able to release ten times more libraries than they could before, but at the moment, they certainly cannot maintain ten times more libraries than they could before.

Scale --

Modular Explicits in OCaml [modular-explicits]

Patrick Ferris

Abstraction, in OCaml, often refers to hiding a particular type's implementation using a signature. For example:

module S : sig
  type t
  val add : t -> t -> t
end = struct
  type t = int
  let add = Int.add
end

In this post, however, I use abstraction to refer to the notion of code-reuse. Using features of the OCaml language to allow one piece of code to be used across different implementations or instantiations of a particular kind of value.

The goal of this post is to eventually discuss OCaml 5.5's modular explicits which is yet another feature that can allow a single piece of code to be used in various situations.

Polymorphism [abs-polymorphism]

Patrick Ferris

The most prevalent kind of abstraction is polymorphism. For example, List.length will work for all lists regardless of what kind of elements the list contains. However, polymorphism requires you to keep the polymorphic part of the type at arm's length. This can lead to suboptimal performance and/or limited usefulness.

List.mem is a good example; in order to be polymorphic List.mem relies on Stdlib.(=), the polymorphic equality operator. In turn, this operator achieves its polymorphism by performing a structural comparison of the runtime representation of its arguments.

A common way to overcome this is to use higher-order functions (functions that take functions as arguments).

# let rec mem ?(eq=Stdlib.(=)) v = function
  | [] -> false
  | x :: xs -> eq v x || mem ~eq v xs
val mem : ?eq:('a -> 'a -> bool) -> 'a -> 'a list -> bool = <fun>
# let mem_int_list = mem ~eq:Int.equal
val mem_int_list : int -> int list -> bool = <fun>

This works well in plenty of cases (List.map, Hashtbl.iter, etc.), however, it doesn't scale well from a user's perspective if the body of the function requires lots of specific functions (e.g. compare, length, pp). At the same time, we want to avoid the precarious duck-typing of something like Python too.

Functors [abs-functors]

Patrick Ferris

Functors are perhaps the second most prevalent feature of OCaml when it comes to code reuse. In the OCaml sense, a functor is a module that is parameterised by other modules. Set.Make is a functor that requires a module adhereing to the Set.OrderedType interface.

module IntSet = Set.Make (Int)
module StringSet = Set.Make (String)

Functors make it a lot easier to write implementations that depend on a large variety of functionality. Set.OrderedType is relatively tame in this sense, only needing a type t and a compare : t -> t -> int function.

Irmin, on the other hand, has functors that require users to provide Schemas. I have more thoughts about this API design in my Irmin retrospective.

It is important to consider the runtime costs of functors too. Whilst most of the time functors are instantiated at the toplevel, it does not come for free. You can have a play around with js_of_ocamlopt and see the generated output for how functors are applied.

First-class Modules and Existentials [abs-fcm-exist]

Patrick Ferris

Sitting somewhere inbetween polymorphism and functors are first-class modules with existentials. This is a rather advanced use of OCaml's type system to be able to pack together a value of a particular type alongside a module that works with those kinds of values.

type set = Set : ((module Set.S with type t = 'a) * 'a) -> set
let size (Set ((module S), s)) = S.cardinal s

Packaging up integer sets is fairly straight-forward.

# let int_empty_set = Set ((module IntSet), IntSet.empty)
val int_empty_set : set = Set (<module>, <poly>)

From a user's perspective we are now closer to the realm of dynamic typing. Indeed, existentials are the way to do that in OCaml.

# type any = Any : 'a -> any
type any = Any : 'a -> any
# let any v = Any v
val any : 'a -> any = <fun>

Getting back into a typed world often involves some type witness gymnastics or runtime exceptions. Used sparingly, however, it can be very effective particularly when a user is none the wiser about the use of FCMs and existentials. I have made use of this trick to package together Irmin stores with implementations.

You could not, however, simply return the value that was packed away inside the existential.

# let woops (Set (_, s)) = s
Line 1, characters 26-27:
Error: The value s has type $a but an expression was expected of type 'a
       The type constructor $a would escape its scope
       Hint: $a is an existential type bound by the constructor Set.

There is also a runtime cost to the packing (and unpacking) of the module, not to mention the need for the extra allocation to create the existential in the first place. This will likely not be too significant, but is worth considering.

Modular Explicits [abs-modular-explicits]

Patrick Ferris

OCaml 5.5 has module-dependent function types also known as modular explicits. The type of a function can now depend on types that come from a module that is also part of the function definition.

# let set_length : (module S : Set.S) -> S.t -> int =
  fun (module S) v -> S.cardinal v;;
val set_length : (module S : Set.S) -> S.t -> int = <fun>
# set_length (module IntSet) IntSet.empty;;
- : int = 0
# set_length (module StringSet) StringSet.empty;;
- : int = 0

In fact, we can now return values of a particular type that depends on a particular module!

# let empty_set : (module S : Set.S) -> S.t = 
  fun (module S) -> S.empty;;
val empty_set : (module S : Set.S) -> S.t = <fun>
# empty_set (module IntSet);;
- : IntSet.t = <abstr>
# empty_set (module StringSet);;
- : StringSet.t = <abstr>

Or, we can dynamically select implementations and values.

# let with_impl : 
    config:string -> ((module S : Set.S) -> S.t -> 'a) -> 'a = 
  fun ~config fn ->
  match String.lowercase_ascii config with
  | "string" -> fn (module StringSet) StringSet.empty
  | "int" -> fn (module IntSet) IntSet.empty
  | c -> invalid_arg c
val with_impl : config:string -> ((module S : Set.S) -> S.t -> 'a) -> 'a =
  <fun>

This is like a more usable version of the FCM-existential trick, though does require more manual type annotations on functions. For instance, I tried switching out OBuilder's use of existentials with modular explicits:

-type builder = Builder : (module Obuilder.BUILDER with type t = 'a) * 'a -> builder
-
 let log tag msg =
   match tag with
   | `Heading -> Fmt.pr "%a@." Fmt.(styled (`Fg (`Hi `Blue)) string) msg
   | `Note -> Fmt.pr "%a@." Fmt.(styled (`Fg `Yellow) string) msg
   | `Output -> output_string stdout msg; flush stdout

-let create_builder store_spec conf =
+let create_builder :
+  Store_spec.store Lwt.t -> Native_sandbox.config ->
+    ((module B : Obuilder.BUILDER) -> B.t -> 'a) -> 'a Lwt.t =
+  fun store_spec conf fn ->
   store_spec >>= fun (Store_spec.Store ((module Store), store)) ->
   let module Builder = Obuilder.Builder (Store) (Native_sandbox) (Docker_extract) in
   Native_sandbox.create ~state_dir:(Store.state_dir store / "sandbox") conf >|= fun sandbox ->
   let builder = Builder.v ~store ~sandbox in
-  Builder ((module Builder), builder)
+  fn (module Builder) builder

I think the code is more pragmatic; there is no longer an indirection via a type called builder. The extra type annotations do add a little noise. Roll on modular implicits.

Vibecoding Etiquette [vibecoding-etiquette]

March 9, 2026
Patrick Ferris

I am yet to jump head-first into using LLM-based tools like Claude Code or even ChatGPT to help with my programming in any serious way. After using some "free"-tier tools to try to better understand some tricky eBPF problems and to make sense of the semantics of POSIX shells, I was left unimpressed and ultimately felt my time was wasted.

I feel unwilling to send my money to the corporations behind these tools. This is for a variety of reasons including the deeply worrying relationship with the US Department of War (I struggle to find the meaningful difference between "opertational planning" vs. "operational decision-making" given the leaders of the US DoW), environmental concerns, exploitation of human labour and a fast-and-loose approach to copyright laws. All of which leaves the entire industry very unattractive to me.

Everybody can draw their line where they want, and I am, in good faith, trying to find where mine should be.

Unfortunately, my own decision to not use these tools is not completely in my control. What do I do about pull requests to open-source libraries I maintain that include agentic code? I am, after all, participating in the idea of open-source and inclusive collaboration. Agentic code also comes in many flavours, from blatant LLM-generated pull requests to heavily edited, human-in-the-loop contributions.

An immediate first step for me is to develop some policy for contributions to projects I maintain regarding code where LLMs have had a hand. This can help set expectations for contributors. Eio's "AI-generated Code" subsection is succinct and prohibits contributions that solely use AI.

It obfuscates how you think. Purely AI-generated code tells us little about how you think and the problems you might be having. This makes it harder to provide good feedback on PRs and issues.
It is often more work to review. Particularly for the OCaml ecosystem and libraries like Eio, it seems that these tools are not very good and generate a lot of believable code that is in actual fact completely wrong. PR comments and the code submitted with them can say completely different things.
It is a grey area for licensing. Models like ChatGPT have been trained on lots of code with different licenses and has been known to simply copy code as an answer to a prompt. We would like to avoid this headache as best we can.

Additionally, I plead to other developers to own their use of these tools. Please, make Claude a co-author of the commits where Claude has generated any code at the very least. This is the bare minimum. I have strong feelings about the etiquette of sending AI summarised information or LLM-generated code to other people, but before any of that, the decent thing is to remove any uncertainty in the receiving party's mind of how the information they are receiving came to be. In this way, I appreciate the OCaml compiler's AI notice which includes:

If a significant portion of your code, PR description, review comments or messages has been AI-generated, this must be disclosed, stating which tool was used and for what. Reviewing AI-produced code can require a different approach from human-written code. You should have an acknowledgement anyway even if another human contributor did some of the work.

ICFP 2025 [icfp-2025]

October 24, 2025
Patrick Ferris

I must thank my office mate (and friend!) Ryan Gibb for his camaraderie and sage advice: "the best track is hallway track". Not to mention all of his photos too. Thanks Ryan!

From right to left, top to bottom: Anil Madhavapeddy, Sadiq Jaffer, Patrick Ferris, Ryan Gibb and Roy Ang.

Random Hacking

October 24, 2025
Patrick Ferris

Below are just some of the pieces of random hacking I got up to whilst in Singapore.

OxCaml and Uring [oxcaml-and-uring]

Patrick Ferris

After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change:

  module Heap = struct
    type ptr = int32#
    (* ... *)
  end

  type cqe = {
    user_data_id : Heap.ptr;
    res : int32#;
  }

Bibtex Parser

October 24, 2025
Patrick Ferris

Geocaml Libraries

October 24, 2025
Patrick Ferris

There has been some excellent outreachy work happening on the geocaml/ocaml-tiff library which is a potential Outreachy internship project for December 2025. More on this in a future post.

Modern OCaml bindings to PROJ (a coordinate reference system projection library)
Further work on a WKT library for OCaml
Coming soon... ocaml-geotessera

Talks

October 24, 2025
Patrick Ferris

Hazel of OCaml at TyDe

October 24, 2025
Patrick Ferris

Scientific Programming at PROPL

October 24, 2025
Patrick Ferris

TMF Data Discrepancies Redux

October 24, 2025
Patrick Ferris

Land Use Class	Percent Change
Undisturbed	-2.9
Degraded	-5.5
Deforested	8.6
Regrowth	0.5
Water	0.0
Other	-0.6

Shelter Reactions

October 24, 2025
Patrick Ferris

Other Talks and Highlights

October 24, 2025
Patrick Ferris

I was a part of, often in a very small way, some other talks that were presented at ICFP including

Functional Networking for Docker, I encourage you to read Anil's notes on the topic. It is also worth mentioning Anil's post-posix talk (not unrelated to my Uring musings above).
Michael gave an excellent, very live, talk about Yirgacheffe at PROPL which I thoroughly enjoyed. I'm also excited about his recent developments which I'm sure you can clue into over on his blog.

Below is a non-exhaustive list of talks I enjoyed whilst at the conference. Some relevant, most of them not!

By far, one of the best talks was Polynomial-time Program Equivalence for Machine Knitting. I think the talk really nailed the "here's the idea, the details are in the paper"-presenting style as they described an "...algorithm that canonicalizes the algebraic representations of the topological semantics of machine knitting programs".
I attended most of the OxCaml tutorial presented by the incredibly friendly Gavin Gray.
Ryan's talk Spatial Programming for Environmental Monitoring at PROPL was excellent.

Irmin Retrospective [irmin-retro]

August 7, 2025
Patrick Ferris

Irmin is an OCaml library for building branchable and mergeable data stores. The data is mergeable in the sense of mergeable replicated data types.

I have been using Irmin for over five years to build different kinds of interesting data stores including:

A simple markdown-based note-taking web application.
A content-addressed database.
Mentoring an intern who worked on Irmin in the browser.
Most recently, an Irmin-backed shell session manager.

I was asked to provide some feedback recently on Irmin, so I thought writing a little retrospective here would be a good way to do that. The remit for the retrospective was about improving Irmin, so the content is focussed on pain points and areas of improvement.

(* An in-memory Irmin store *)
module Store = Irmin_mem.KV.Make(Irmin.Contents.String)

What is Irmin?

August 7, 2025
Patrick Ferris

Irmin, at its simplest, is a key-value database. Users associate keys with values and can query and update these bindings.

Additionally, this database supports versioning. This means independent snapshots of the database can coexist and users can switch between them and update them without fear of interfering with other versions.

Different versions of the database can be combined by merging. When you set up your instance of an Irmin database, you also provide it with a merge function.

# #show_type Irmin.Merge.f;;
type nonrec 'a f =
    old:'a Irmin.Merge.promise ->
    'a -> 'a -> ('a, Irmin.Merge.conflict) result Lwt.t

Design and API

August 7, 2025
Patrick Ferris

Irmin's API is very git-inspired. There is a large overlap of shared vocabulary and concepts: repositories, branches, commits, heads etc.

Probably the most confusing aspect of this is the notion of a Store. When I was describing Irmin above, I used the term database to help distinguish between some of these concepts. In Irmin's documentation, it is used for multiple related (but different) concepts. In "Creating a Store" stores refer to the entire database, whereas in the Irmin API docs we have that:

There are two kinds of store in Irmin: the ones based on persistent named branches and the ones based temporary detached heads.

For Irmin library users, a Store.t can be thought of as a checkout of the database at a particular revision. This may be from a branch or a specific commit.

# Store.of_branch;;
- : Store.repo -> string -> Store.t Lwt.t = <fun>
# Store.of_commit;;
- : Store.commit -> Store.t Lwt.t = <fun>

The overloading of the term is confusing. I think it leaves users unsure about what other people might mean when they say "store". Being careful with these terms, and the contexts in which they are used, would help avoid this confusion.

Module and Functor Soup

August 7, 2025
Patrick Ferris

Undoubtedly for a majority of use-cases and users, Irmin is over-functorised. Nearly every module requires that you must apply some functor to access any useful code. In general, a user will have to interact with the Schema module when describing the types they want to instantiate their store with.

To counteract this, Irmin has plenty of KV modules that provide a Make functor that only requires a user to provide a suitable content module for their store (i.e. something that provides a type, a runtime representation of that type and a merge function).

# #show_module Irmin_mem.KV;;
module KV :
  sig
    type endpoint = unit
    type metadata = unit
    type hash
    type info = Store.info
    type ('h, _)contents_key = 'h
    type 'h node_key = 'h
    type 'h commit_key = 'h
    module Make : (C : Irmin__.Contents.S) ->sig ... end
  end

I think it is fair to say the documentation is hard to follow as it is a module and functor soup. For example, looking at irmin.3.11.0, starting at the toplevel documentation page our path to finding this paricular module and functor is as follows:

We jump into Irmin_mem from the nicely written landing page.
We scroll down to find the KV module.
We now make sense of the module's signature by following the KV_maker link.
We see a Make (C : Contents.S) : sig ... end at the end of the module, and we navigate through that.
Finally we have come to our journey's end, and find the schema module with type constraints like type Branch.t = string.

There is a big assumption there that you know what the Schema module is telling you and how it relates to your "store".

There is a counter-argument here, in that Irmin is incredibly flexible in terms of what you can use as types for your keys, branches, hashes etc. I was quite easily able to produce Irmin stores that use CIDs (self-describing content identifiers) for example.

I think there is a middle ground. For Irmin to be usable it needs to reduce the complexity of the API. The complexity is due to, in large part, over-functorisation. If the functors cannot be removed then perhaps better documentation or more introductory libraries like irmin-containers would be helpful. Perhaps a standalone library that acts as an interface to Irmin stores would be helpful. I find myself time and again implementing something like that.

Backends Galore

August 7, 2025
Patrick Ferris

Irmin has plenty of backends including a git-compatible one, a MirageOS backend, an in-browser IndexedDB backend and even a FoundationDB backend.

Irmin needs to have fewer, better tested backends. Of course, some of these are likely driven by funding and use-case specific details. However, I think by focusing on a few key backends and striving for solid performance but also strong consitency guarantees, Irmin would become a more viable candidate for potential users. I also believe the browser backend plays a key part to making this work in order to facilate local-first applications.

Syncing Remote Stores

August 7, 2025
Patrick Ferris

The synchronisation mechanisms in Irmin are powerful, but often the API is very confusing. For example, with the git-compatible Unix backend, nested deep in the documentation is the function required to create a remote endpoint. Once they have found this function, using it is not easy as it requires them to learn about the Mimic.ctx, which is an abstraction of the networking stack!

Do Fewer Things and Do Them Well

August 7, 2025
Patrick Ferris

Irmin is simultaneously a database that supports: content-addressing, merging, key-value lookup, branches, transactions, complete OS portability etc. I think, in short, it tries to do too many things and comes up short on some of them in ways that really matter.

When building the content-addressed database I needed strong guarantees about some atomic actions to perform on the underlying store. For example, setting the value and accessing the commit associated with it which was not possible with the API at that time in an atomic way. This kind of missing functionality is probably only understood after users stumble across it, but it also makes the library feel less focused and ready for production use-cases.

Documentation

August 7, 2025
Patrick Ferris

It will probably come as no surprise that one of the main limitations of using Irmin is the lack of documentation and tutorials. This is a particular pain point as the API is not very straightforward.

A while back, I started an Irmin book intending to help document the kinds of things real-world users would need in order to use Irmin in earnest. For example: ways to deal with type migrations or primers on runtime types. I still believe this sort of work would be invaluable, but there is a large cost to completing it, and it is not clear if there is enough interest in the project to warrant such an effort.

The tutorials on the irmin website are still good starting points for most new users. But they quickly lack the depth for real-world scenarios.

Feature Wishlist

August 7, 2025
Patrick Ferris

What follows are additional ideas for improving Irmin.

Heterogeneous Stores

August 7, 2025
Patrick Ferris

For all of [Irmin]'s abstraction and functorisation, there is a very clear missing feature for lots of first-time users of the library: you can only store values of a single type.

This leads to a few common workarounds:

Storing a serealised version of your values, essentially escaping the type-system and implementing a form of dynamic typing.
Growing your value to hold lots of different types via some large variant type.

Both of these options are feasible and have been used in practice. However, they are workarounds. A long time ago CraigFe created an RFC for heterogeneous stores, which I think about a lot. The main idea is to augment Irmin paths (keys) to be GADTs that carry type information about the kinds of values they access (similar to heterogeneuous variants of other data structures). Something like this alongside a simplified API would make Irmin more appealing as a library for persistent data storage.

Real-world Retrospectives

August 7, 2025
Patrick Ferris

This echos some thoughts from the Documentation section. There are some very real-world Irmin use-cases out there. For example, for a long time (I'm not sure if this is still the case) parts of the Tezos blockchain were using Irmin which uses the irmin-pack backend. Does this make irmin-pack the best tested backend and perhaps should be the default backend for new users? Or is it hyper-specific to the Tezos use case? There are blogs on tarides.com about the irmin-pack backend, but they might be outdated and should not be the first place to find advice on which Irmin backend to use.

Active Development and Engagement

August 7, 2025
Patrick Ferris

As far as I know, Irmin was at the forefront of technologies that have come to be described as local-first. There is a growing interest in this area (particularly as it acts as a counter-argument to an increasingly online, centralised model). I highly recommend reading Ink & Switch's essay on the matter. And it would be great to see more research via Irmin at things like the lofi unconference and the PLF workshop at SPLASH 2023!

Thank you for reading! And thank you to all the Irmin contributors. I hope this might be useful in the future for building the next-generation Irmin!

Slipshow x x-ocaml [slipshow-x-xocaml]

July 23, 2025

A short, explanatory post about combining two very fun pieces of work in OCaml.

Paul-Elliot has been building Slipshow for some time now where slides are slips and your presentations run vertically. More recently, Arthur has built x-ocaml, a web component library for executable OCaml cells embedded into OCaml.

Using xocmd, a small tool I built for translating markdown codeblocks to x-ocaml components, your Slipshow's can now be executable!

xocmd learn-effects.md | slipshow compile - > learn-effects.html

Take a look at an example! (or the source markdown).

I really like this light-weight approach to building interactive presentations for explaining things in OCaml (e.g. over running a jupyter notebook server).

Quarterly OCaml Q2 [ocaml-quarterly-q2]

July 18, 2025
Patrick Ferris

Thanks to Tarides sponsorship, I get to work on open-source OCaml. This quarterly is a companion to my weeklies, summarising the last three months of development, peppered with ideas and thoughts about OCaml, its community and its future.

What I wanted to work on?

July 18, 2025
Patrick Ferris

There were two main things I hoped to continue working on: ppxlib and outreachy. These are projects that I was previously working on, and in the case of Outreachy I have now been involved for many years.

In addition to this, all of my Part II projects this year used OCaml in some regard. In general, I want to see more adoption of OCaml. Over the years this has taken many forms including my initial work on developing workflows for OCaml that just turned five years old. This directly fed into the rebranding and rethinking of ocaml.org itself.

What I worked on?

July 18, 2025
Patrick Ferris

Ppxlib

July 18, 2025
Patrick Ferris

Ppxlib is the de facto standard library for building OCaml preprocessors. At the time of writing, opam list --depends-on=ppxlib informs me that there are 267 reverse dependencies. Janestreet is a heavy user of ppxes and has authored many.

One of the main accomplishments this quarter was bumping the internal AST to 5.2. This allows ppx authors to use new OCaml language features in their ppxes. In bumping the AST, we knowingly broke compatability for pretty much every single reverse dependency. As best we can, we have been sending patches to ppx libraries and helping users migrate to the latest ppxlib.

There is an interesting dicussion to had about the nature of open-source, and its interaction with industrial monorepos and community tended package repositories. Package ecosystems thrive whenever there is a dedicated community creating, maintaining and publishing packages. The idea is that the published world should be healthy. The publishing medium can act as natural limiting factor in the churn of breaking changes (in the case of OCaml this is via PRs to the opam-repository). This, I have come to notice, reacts poorly to changes coming from internally consistent monorepos where introducing breaking changes is easily fixed by applying patches there and then. Whatsmore, OCaml is often stated as an incredibly safe language to perform large refactorings thanks to its type system.

Ppxlib sits awkwardly in the space of possible breaking changes. Tied to OCaml's parsetree, impacts of changes there ripple down to ppxlib. The compiler itself can remain internally consistent, and is protected as it need only parse source code. Ppxlib, on the other hand, exposes the parsetree to users and thus any changes to the parsetree will likely be felt by ppx authors. Since I started working on ppxlib, it feels as though the number of syntax changes has gone up (primarily from Janestreet work). Unless we make changes to how we provide support for these, maintainers of ppxlib will forever be stuck doing busy work! Thankfully, Nathan has thoughts on how to improve this.

There are a slew of other features I have added to ppxlib including:

Support for deriving from classes.
Support for deriving from module bindings and signatures.
Fixing compiler and ppxlib dummy locations.
Bumping to 5.3.
Migrations for 5.4.

Outreachy

July 18, 2025
Patrick Ferris

Our two projects this year, one on claudius and one on dune, are going extremely well. At the time of writing, we just had a mid-internship call to catch up, and I was blown away by the progress each intern has made. Unfortunately, Outreachy is struggling with funding and the next round is perhaps not going to happen. This is a real shame and I am hoping that it will not be the case.

Outreachy has been a wonderful source of new, committed OCaml developers. If you haven't already, do peruse the webpage on OCaml.org to see past internships or watch the demo day presentations. For the mentors involved, I believe it has also been a rewarding experience (though at times a challenging one). We are always looking for new mentors and project ideas, please do reach out to me if you are interested.

Hazel

July 18, 2025
Patrick Ferris

OCaml's feature set allows it to shine at writing programming languages (and things of that ilk: compilers, interpretters, static analysis tools). Hazel is a research programming language with typed holes written completely in OCaml (via the reason dialect).

Relating this back to the original intent of my work, to improve OCaml adoption, I believe this also means keeping existing users happy. In the last quarter I developed a compiler from OCaml to Hazel. Whilst new features are still being added to Hazel, I hope this could serve as a tool to help develop test-suites and standard library functions using existing OCaml solutions. In a student's work (Typed Debugging for Hazel), we used this tool to build a corpus of ill-typed Hazel programs to great effect.

OCaml should continue to be a world-class programming language for building other programming languages. I hope to upstream some of this work to Hazel and provide some low effort maintenance to help keep their compiler in good shape.

Systems Programming in OCaml

July 18, 2025
Patrick Ferris

In a cross-over with my own research, I have been developing many tools related to systems programming in OCaml including:

An eBPF-based open syscall tracing tool.
A library in OCaml for spawning void processes.
A shell session manager that uses Irmin to manage sessions. It is nice to see a new push to finally land the direct-style Irmin PR!

This work is means to develop the underlying libraries that support it. For example, I have opened a few PRs to Eio to add new "fork actions" to the spawn API. I also investigated the feasibility of changing the underlying mechanisms in Eio to use Picos. In the future, I think this could be important avoid further splitting the OCaml ecosystem.

OxCaml

July 18, 2025
Patrick Ferris

I dabbled a little with OxCaml and build try-oxcaml to let people take it for a spin without having to perform opam repository gymnastics. It turned into a lot of work to track down some pretty inane bugs (type definitions differeing between js_of_ocaml and the OxCaml compiler, resulting in different Javascript runtime representations...).

This unblocked a few of my colleagues to get OxCaml working on tools like odoc_notebooks and x-ocaml.

Forester

July 18, 2025
Patrick Ferris

A good proportion of my work this quarter has been focused on how to present the very work that I am doing. Jon Sterling has been developing a tool for scientific thought called Forester which seemed like a possible candidate for writing and sharing my work.

Porting my existing blog posts and website content from markdown to Forester's LaTeX-inspired syntax didn't seem like an option. In particular, many of my posts made use of additional markdown-based tools (like ocaml-mdx).

This lead to the development of Graft: a preprocessor for Forester forests, converting markdown and bibtex to trees.

What's next?

July 18, 2025
Patrick Ferris

So, I worked on most of what I wanted to work on and then some! Going forward I hope to keep maintaining ppxlib in some capacity and coordinating the OCaml community's Outreachy efforts.

I hope to continue my small experiments (e.g. converting Eio to Picos). My own research makes heavy use of Irmin, and I would be interested to help with that too. More recently, a new library has been released for working with numerical data in OCaml called Raven: I am interested to use this library in Geocaml, a suite of geospatial tools written in OCaml that I maintain.

I feel conflicted about the OxCaml efforts. I admit, I am not fully aware of the full benefits of the features, but I worry about some proliferation of modes that make the type system in OCaml unbearable to use. On top of this, with my ppxlib hat on, I worry about the impact of changing the compiler so frequently, placing strain on an already small community. That being said, by releasing OxCaml separately I do believe running it as an experimental set of packages will help understand the tool better. But I would not be building anything I intend to maintain or research with right now as that ecosystem is far too volatile.

Finally, over the past two years I have done a lot of teaching. From Part II projects to supervisions. Going into next (academic) year, I intend to reduce my in-person teaching to focus on my research. However, I am interested in producing more materials for learning, maybe some of this in OCaml, perhaps something similar to Beautiful Racket. I have tried in the past to do these sorts of things, for example The Irmin Book.

Thank you

July 18, 2025
Patrick Ferris

Thank you for reading this wrap up! And thank you again to Tarides for letting me work so freely on things that I think are good for the OCaml community.

Opentrace [open-trace]

May 19, 2025
Patrick Ferris
https://tangled.sh/@patrick.sirref.org/opentrace

Thanks to Koonwen's excellent libbpf bindings in OCaml, I have been building a little tool called opentrace to make it easier to track an executable's inputs and outputs.

This work was inspired my Michael's self-proclaimed "gross hack": pyshark. Whilst pyshark achieves its goals by injecting code into commonly used python objects and methods, opentrace uses eBPF. By using a lower-level API (hooks in the kernel), opentrace can remain programming language agnostic. However, less information is none about the user's intent compared to something like pyshark.

Monitoring the System

May 19, 2025
Patrick Ferris

opentrace has an all command that will trace the entire system.

$ sudo opentrace all --flags=O_WRONLY
pid,cgid,comm,kind,flags,mode,filename,return
16324,4417,".nheko-wrapped",openat,577,438,"/home/patrick/.cache/nheko/nheko/curl_alt_svc_cache.txt",47
16324,4417,".nheko-wrapped",openat,193,33188,"/home/patrick/.cache/nheko/nheko/cr3ZUHqBErIOe3PlwJ1SuU8zFKKxL12VzrRoHYMH.tmp",47
16324,4417,".nheko-wrapped",openat,577,438,"/home/patrick/.cache/nheko/nheko/curl_alt_svc_cache.txt",47
16324,4417,".nheko-wrapped",openat,193,33188,"/home/patrick/.cache/nheko/nheko/QfZTdZSuC56NdcDQ3aMxJc3BhMhAj8PmtYW1zFDP.tmp",47
2530,4235,"systemd",openat,524865,438,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/cgroup.subtree_control",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.min",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.low",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.high",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.max",41

The --flags=O_WRONLY argument filters the events where the O_WRONLY flag was set in the call to open.

We also get the name of the current executable linked to the task (comm). The -wrapped is an artefact of using Nix.

Tracing an Executable

May 19, 2025
Patrick Ferris

The primary use case for this tool is to inspect what files your program might be reading and writing.

$ sudo opentrace exec --format=json --flags=O_CREAT -- opam list
$ cat trace.json | jq ".[] | .fname"
"/home/patrick/.opam/cshell/.opam-switch/packages/cache"
"/home/patrick/.opam/log/log-118747-29da3d.out"
"/home/patrick/.opam/log/log-118747-29da3d.err"
"/home/patrick/.opam/log/log-118747-29da3d.env"
"/home/patrick/.opam/log/log-118747-29da3d.info"

The "flags" argument can specify a small boolean formula for checking the open flags of a particular event with | (or), & (and), and ~ (not). Parentheses can be used for precedence.

$ sudo opentrace exec --flags="O_WRONLY|O_RDONLY" -- ocaml --version

Spawning Subprocesses

May 19, 2025
Patrick Ferris

One feature opentrace needs (in this proof-of-concept phase) is the ability to also trace subprocesses.

opentrace is primarily an eBPF program that is loaded into the kernel and communicates with an OCaml program. Events are communicated via a ring buffer and most of the post-processing happens in OCaml. To capture subprocesses, opentrace creates a new control group (cgroup) and places the new process into that group. This gives opentrace a new identifier to track, namely the cgroup.

So consider the following program.

let () =
  Eio_posix.run @@ fun env ->
  Eio.Path.(save ~create:(`Or_truncate 0o664) (env#fs / "hello.txt") "hello");
  Eio.Process.run env#process_mgr
    [ "/bin/bash"; "-c"; "echo 'heya' > heya.txt" ]

It first creates a file using direct calls to functions like openat. Then it spawns a process which creates a new file called heya.txt. This happens in a separate process. However, with the --cgroups flag we can capture both interactions with the operating system.

$ sudo opentrace exec --cgroups --flags="O_CREAT|O_TRUNC" ./main.exe
pid,cgid,comm,kind,flags,mode,filename,return
153187,530807,"main.exe",openat,526914,436,"hello.txt",5
153192,530807,"bash",openat,577,438,"heya.txt",3

Eio's Process API

May 19, 2025
Patrick Ferris

I have used the Eio_unix fork action process API to be able to extend what happens in the child process. Loading most eBPF programs into the kernel requires special privileges hence the need for sudo. When a user requests for a particular program to be executed and traced, opentrace spawns a process via the Eio Process API. Opentrace defines a few new so-called "fork actions", little fragments of C code that are run after the call to fork (clone). Most likely this ends with a call to execve, but other calls are possible for example setuid allowing opentrace to change the user of the child process so it does not run as root. Similarly, this is where (if used) we create the cgroup and place the process into that group.

Limitations: Io_uring

May 19, 2025
Patrick Ferris

Whilst testing opentrace against some of the tools I use nearly daily, I noticed some events were being missed. I tried tracing forester, and only the initial read of forest.toml was logged. It dawned on me that the reason for this was that forester (via eio) was using io_uring to perform most of the IO. Most attempts to open files were bypassing the open system calls, and instead they were being performed by the kernel after reading a submission request for an openat2-style call!

This is not news to seasoned, Linux systems programmers. Io_uring bypasses SECCOMP filters for exactly the same reasons.

$ sudo opentrace exec -- forester build
$ cat trace.csv
pid,cgid,comm,kind,flags,mode,filename,return
155007,535570,"forester",openat,524288,0,"forest.toml",5
155007,535570,"forester",Uring,2621440,0,"",0
155021,535570,"cp",openat,131072,0,"/home/patrick/documents/forest/theme/favicon-32x32.png",4
155007,535570,"forester",Uring,2686976,0,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0

It is interesting to note two things here:

We can tell that forester reads the configuration file probably using something like In_channel in OCaml (it does).
It appears that Uring is performing IO in both worker threads and directly.

The file paths are empty at the moment as I cannot find a clean way to trace openat submission requests into Uring without it sometimes going very wrong. I have tried quite a few methods (e.g. tracing do_filp_open) and at the moment I am tracing io_openat2, but this seems quite brittle, and often the filename is completely garbled, so I do not set it. If anyone has any ideas to trace Io_uring more reliably, I am all ears!

Try OxCaml [try-oxcaml]

May 9, 2025
Patrick Ferris
https://patrick.sirref.org/oxcaml

This week, I have been trying out Janestreet's Oxidised OCaml (see their ICFP paper). This adds a system of modes to OCaml for expressing things like locality.

...we’re introducing a system of modes, which track properties like the locality and uniqueness of OCaml values. Modes allow the compiler to emit better, lower-allocation code, empower users to write safer APIs, and with the advent of multicore, statically guarantee data race freedom—all in a lightweight way that only affects those in need.

For example, you might say:

let is_empty (s : string @@ local) =
  String.length s = 0

To get a feel for how this changes the language, you can follow their instructions on their custom opam-repository. Alternatively, you can give it a go in your browser! This is a full toplevel running with Base installed (which has plenty of OxCaml annotations on Stdlib-like functions).

Happy OxCamling!

A Transpiler from OCaml to Hazel [hazel-of-ocaml]

May 2, 2025
Patrick Ferris

Over the past few months, I have been piecing together a transpiler from Hazel to OCaml. This is, in part, to help one of my third-year undergraduate students who is working on type error debugging in Hazel.

Typed Holes

May 2, 2025
Patrick Ferris

Hazel is a functional programming language with typed holes. Holes are pieces of your program that have not yet been filled in. Holes can appear anywhere in your program both as expression or types. Hazel can still evaluate your program in the presence of holes.

To get a flavour of Hazel, take a regular map function for lists.

let map = fun f -> fun xs -> case xs
  | [] => []
  | x :: xs => f (x) :: map(f)(xs) 
end in
map(fun x -> ?)([1, 2, 3])

The question mark (?) is a hole. The program evaluates to the following expression of type [?] (for people more familiar with OCaml types ? list).

[ ?, ?, ? ]

Hazel supports local type inference but nothing involving unification variables. For example, a simple add_one function in Hazel (fun x -> x + 1) has type ? -> Int.

From OCaml to Hazel

May 2, 2025
Patrick Ferris

The ability to transpile OCaml programs to Hazel programs is motivated by one simple thought: there are more OCaml programs than there are Hazel programs. This could help bootstrap projects by alleviating the need to rewrite boilerplate code (e.g. URI parsing or standard library functions for strings).

A Transformation of Syntax

May 2, 2025
Patrick Ferris

Hazel markets itself as an "Elm/ML-like functional programming language". From the previous example of map, it should be apparent just how close to OCaml the language is.

It turns out that a majority of the transpiler is a transformation of syntax. Take a simple ADT for an arithmetic programming language.

type expr =
  | Float of float
  | Add of expr * expr
  | Sub of expr * expr
  | Mul of expr * expr
  | Div of expr * expr

And when we run hazel_of_ocaml over this OCaml type declaration.

type expr =
  + Float(Float)
  + Add((expr, expr))
  + Sub((expr, expr))
  + Mul((expr, expr))
  + Div((expr, expr))
 in ?

Not much has changed expect some syntax. Hazel does not have a notion of top-level expression so hazel_of_ocaml wraps the program into one set of value bindings. For the most part, Hazel acts as a subset of the pure, functional part of OCaml. At the time of writing, this subset is fairly limited with no support for modules or labelled records out of the box (there are plenty of development branches with these features).

If we try out the same map function but written in OCaml and transpiled to Hazel we get.

let rec map f = function
  | [] -> []
  | x :: xs -> f x :: map f xs

Which becomes the following hazel program.

let map = fun f -> fun x1 -> case x1
  | [] => []
  | x :: xs => f(x) :: map(f)(xs)
end in ?

We could have a field day discussing the syntax of OCaml and Hazel (parentheses for function arguments, well-scoped cases for pattern-matching, a different arrow for pattern-matching etc.). What would be more interesting is taking a look at how to handle polymorphism in Hazel.

Explicit Polymorphism

May 2, 2025
Patrick Ferris

Hazel has explicit polymorphism. So far, we have not seen it as we have let the types have holes in them. The map function in OCaml has the following type.

val map : 
  ('a -> 'b) -> 'a list -> 'b list

We must remind ourselves (by reading Craig's excellent blogpost on the matter) that in OCaml

... type variables in signatures are implicitly universally-quantified

So in reality, we have that map has the following type.

val map : ∀ a b. (a ->  b) -> a list -> b list

In Hazel, we have to explicitly type our map function to be polymorphic. Not only does this mean the type annotation requires universally quantified type variables, but we must also perform type application wherever we choose to apply the map function (whether that be recursively or somewhere later in our program).

let map : forall a -> forall b -> (a -> b) -> [a] -> [b] =
  typfun a -> typfun b -> fun f -> fun xs -> case xs
    | [] => []
    | x :: xs => f (x) :: map@<a>@<b>(f)(xs) 
end in
map@<Int>@<Int>(fun x -> ?)([1, 2, 3])

forall introduces a universally quantified type variable into our type annotation, and typfun introduces it into the function itself (à la System F). Type application requires @<T> where T is some type. This allows hazel to quite easily support higher rank polymorphism, but we will not worry too much about that.

Propagating OCaml Types into Hazel

May 2, 2025
Patrick Ferris

Most often, OCaml users interact with prenex polymorphism (rank-1) where the universal quantifiers are at the front of the type. OCaml does support quantifiers inside certain types like records.

What this means for the transpiler is that we can reuse OCaml's type inference to safely instantiate the correct type annotations and type applications in Hazel! To do this, hazel_of_ocaml uses Merlin to inspect the type of the function in either a value binding or at the point of a function application.

Take a simple, polymorphic length function.

let rec length = function
 | [] -> 0
 | _ :: xs -> 1 + length xs

let int_len = length [ 1; 2; 3 ]
let str_len = length [ "only"; "two" ]

When we run this through hazel_of_ocaml with the -type flag we get.

let length : forall a -> [a] -> Int = typfun a -> fun x1 -> case x1
  | [] => 0
  | _ :: xs => 1 + length@<a>(xs)
end in
let int_len : Int = length@<Int>(1 :: 2 :: [3]) in
let str_len : Int = length@<String>("only" :: ["two"])
in ?

hazel_of_ocaml has correctly instantiated the type for length inside the recursive function and then in each case with the integer list and the string list.

A Corpus of Hazel Programs

May 2, 2025
Patrick Ferris

The impetus for this work was to derive a corpus of ill-typed Hazel programs. Luckily, such a corpus exists for OCaml! Seidel et al. created a corpus of OCaml programs from their undergraduate students at UC San Diego. Some of these programs have been transpiled to Hazel.

Future Work

May 2, 2025
Patrick Ferris

Hazel is a fun, research programming language. Potential third-year students may find it interesting to take this work further. For example, how would this look in terms of a module system? From a purely engineering perspective, plenty of work would be needed to convert a multi-library OCaml project to Hazel (e.g. handling the cmi files).

Another line of research would be to have Hazel target one of the intermediate representations in OCaml which would give Hazel a fully functioning compiler to "native" code?

Bumping Ppxlib's AST to 5.2 [ppxlib-5-2]

February 18, 2025

Ppxlib is a libary for building OCaml preprocessors. Users can generate OCaml code from OCaml code (derivers) or replace parts of OCaml code with other OCaml code (rewriters).

At the core of ppxlib is the OCaml parsetree; a data structure in the compiler that represents OCaml source code. Ppxlib makes a distinction between the source parsetree (based on the version of the OCaml compiler you are using) and ppxlib's parsetree (a copy of some parsetree from the compiler).

In ppxlib.0.36.0, the internal AST was bumped from the AST of OCaml 4.14.0, to the AST of OCaml 5.2.0. This post discusses some of the internal mechanisms that allows ppxlib to support multiple compilers and how this bump broke a lot of backwards compatibility.

Manual Migrations in Ppxlib

February 18, 2025

It is possible to perform migrations between parsetrees manually using the ppxlib.ast package.

open Ppxlib
open Ast_builder.Make(struct let loc = Location.none end)
let pp_expr = Format.printf "%a%!" Pp_ast.(expression ?config:None)

let loc = Location.none
let str txt = { txt; loc }

module To_408 = 
  Ppxlib_ast.Convert (Ppxlib_ast.Js) (Ppxlib_ast__.Versions.OCaml_408)

The Ppxlib_ast.Js module is ppxlib's AST (since ppxlib.0.36.0 OCaml 5.2.0). We can use functions provided by the Convert functor to copy parts of the AST from one version to another. Usually there is very little difference between the versioned parsetrees.

let func arg body =
  pexp_fun Nolabel None (ppat_var (str arg)) body

let add_expr = func "x" (func "y" (eapply (evar "Int.add") [ evar "x"; evar "y" ]))

The above excerpt of OCaml code produces an expression in the language. More accurately, a function that has two arguments. If we print the simplified AST we can see that the arguments are stored together in a list.

# pp_expr add_expr;; 
Pexp_function
  ( [ { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "x")
      }
    ; { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "y")
      }
    ]
  , None
  , Pfunction_body
      (Pexp_apply
         ( Pexp_ident (Ldot ( Lident "Int", "add"))
         , [ ( Nolabel, Pexp_ident (Lident "x"))
           ; ( Nolabel, Pexp_ident (Lident "y"))
           ]
         ))
  )
- : unit = ()

This is a relatively new feature of the parsetree, if we migrate the expression down to 4.08 we see it is represented differently. The output is a little too verbose to be readable, and there's no good way to show the AST, so you will just have to take my word.

let expr_408 = To_408.copy_expression add_expr

Function Arity in the OCaml Parsetree

February 18, 2025

OCaml 5.2 introduced a new way to represent functions. Before 5.2, all functions were represented with single arity. Functions with multiple arguments would return functions as their body. For example:

let add x y = x + y

A two-argument add function would be represented in the parsetree as fun x -> fun y -> x + y. Using the nodes from the parsetree (with some details removed) it would be Pexp_fun (x, Pexp_fun (y, Pexp_apply ...)).

Now, since OCaml 5.2, functions can be expressed with their syntactic arity intact. The function add would look something like Pexp_function ([ x; y ], Pfunction_body (Pexp_apply ...)). Both arguments are stored as a list.

The constructor Pexp_function was not a new addition to the parsetree. It used to represent pattern-matching expressions like function A -> 1. This can now be expressed as a function body of Pfunction_cases.

Whilst these additions and modifications to the parsetree are arguably a better design, it has caused chaos in terms of ppxlib's reverse dependencies!

Coalescing Function Arguments

February 18, 2025

In ppxlib 0.36.0, functions from Ast_builder will help ppx authors produce maximum arity functions. However, functions from Ast_helper will not do this.

let ast_helper_func arg body =
  Ast_helper.Exp.fun_ Nolabel None (ppat_var (str arg)) body

let ast_helper_expr = ast_helper_func "x" (ast_helper_func "y" (eapply (evar "Int.equal") [ evar "x"; evar "y" ]))

Here we recreate our helper function to create function expressions using Ast_helper.Exp.fun_ instead. We can now show the difference between the expr expression and the ast_helper_expr expression.

# pp_expr add_expr;;
Pexp_function
  ( [ { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "x")
      }
    ; { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "y")
      }
    ]
  , None
  , Pfunction_body
      (Pexp_apply
         ( Pexp_ident (Ldot ( Lident "Int", "add"))
         , [ ( Nolabel, Pexp_ident (Lident "x"))
           ; ( Nolabel, Pexp_ident (Lident "y"))
           ]
         ))
  )
- : unit = ()

Above, we see a function expression with arity two. Below, we see nested function expressions each with arity one.

# pp_expr ast_helper_expr;;
Pexp_function
  ( [ { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "x")
      }
    ]
  , None
  , Pfunction_body
      (Pexp_function
         ( [ { pparam_loc = __loc
             ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "y")
             }
           ]
         , None
         , Pfunction_body
             (Pexp_apply
                ( Pexp_ident (Ldot ( Lident "Int", "equal"))
                , [ ( Nolabel, Pexp_ident (Lident "x"))
                  ; ( Nolabel, Pexp_ident (Lident "y"))
                  ]
                ))
         ))
  )
- : unit = ()

Ramifications for Ppxlib Users

February 18, 2025

Many, many, many ppxes broke after migrating the ppxlib AST to 5.2. Mostly, this is due to pattern-matching. Downstream users of ppxlib frequently pattern-match directly against the parsetree. However, nodes such as Pexp_fun no longer exist and Pexp_function has completely different constructor arguments!

Patching all the ppxes has been quite difficult. Most ppxes only deal with single arguments at a time (e.g. Pexp_fun x) but now they must handle zero or more arguments at a time. (In the zero case, this will always correspond to a pattern-matching function).

Here is a non-exhaustive list of some of the ppxes that have been patched. It may useful for other ppx authors who wish to upgrade to ppxlib.0.36.0:

This release has emphasised the tension between using pattern-matching on algebraic datatypes and wanting to reduce downstream breakages (e.g. using abstract datatypes). This tension is not new.

The Future of Ppxlib

February 18, 2025

Ppxlib is not going anywhere anytime soon. We have committed to trying to keep the internal AST up to date with the latest OCaml release. In fact, there is already a PR for the 5.3 bump!

There have been many discussions about simplifying this process, but so far the time and effort to build a ppxlib that will result in fewer breakages is too high (see an attempt using views).

Thank you to Tarides and Jane Street for funding my time to work on this release and all the chaos it has caused.

Happy preprocessing!

Defunctorising VPNKit [vpnkit-upgrade]

February 28, 2025

VPNKit is a core part of the Docker for Mac/Windows stack. It is tasked with translating network activity from the host to the Linux container.

This short post discusses some of the recent changes to MirageOS and how they impact VPNKit.

Dune Virtual Libraries [dune-virt-libs]

Dune, for quite some time, has supported virtual libraries. This allows library authors to define a signature without an implementation (similar to OCaml's module type S = struct ... end). The package is something other dune libraries can build against, still without choosing a particular implementation of the interface. An application author can make the decision about which implementation they would like to build into their program.

This feature has been quite widely used in OCaml. For example, the excellent digestif library of hashing algorithms provides both C and OCaml implementations individually packaged into digestif.c and digestif.ocaml respectively. Consider developing a browser application that needs some hashing algorithm (e.g. Irmin in the browser). We can explicitly use the OCaml backend for digestif which js_of_ocaml will happily compile for us.

There are two distinct limitations to this approach.

Applications can only link exactly one implementation.
Implementations cannot expose additional functionality easily (see dune#5597).

Functors in MirageOS

February 28, 2025

MirageOS is a framework for building unikernels in OCaml. It relies heavily on OCaml's functors to abstract implementation from interface. The mirage tool will then select appropriate instantiations of these functors depending on the backend that is targeted (e.g. unix, hvt etc.)

Recently, there has been a push to use dune virtual libraries instead.

VPNKit's Fake Clock & Time

February 28, 2025

We need to be able to overcome the limitation that implementations cannot expose additional functionality very easily using dune virtual libraries. As it stands, we need to make Mirage's virtual libraries unwrapped (i.e. set (wrapped false)). This allows implementers to expose additional modules alongside the implementation of the virtual library.

For VPNkit, we need this to port some of the DNS forwarding tests which use a fake clock and sleep module. Previously, they could be provided directly as functor arguments to the various DNS modules e.g.

let module R =
  Dns_forward.Resolver.Make(Proto_client)(Fake.Time)(Fake.Clock)

In this new world of dune virtual libraries, our Resolver is significantly more simple.

let module R = Dns_forward.Resolver.Make(Proto_client)

The flexibility of time and clock implementation has been pushed to a build-time decision in a dune file. This means that in order to provide the same fake clock and time we will need to create a private dune library that (a) implements the respective signatures of the virtual libraries and (b) provides the extended interface to allow the test suite to modify the internals of the implementations.

The first thing I had to do was make the mirage-mtime and mirage-sleep virtual libraries unwrapped. This allowed me to make a new private dune library called fake_time with two public modules: the normal interface required by mirage-mtime and another module called Fake_time_state containing:

let timeofday = ref 0L
let c = Lwt_condition.create ()

let advance nsecs =
  timeofday := Int64.add !timeofday nsecs;
  Lwt_condition.broadcast c ()

let reset () =
  timeofday := 0L;
  Lwt_condition.broadcast c ()

I did the same for fake_sleep which depended on the state from the fake_time library (something a normal library wouldn't).

Once I had the implementations for the virtual libraries in place, all that remained was to tell the executable (in this case the test suite) to select these new implementations instead of the defaults. Whereas before, all the tests could live in the same test suite, now they had to be split into those that needed the fake implementation and those that did not. This is another slight limitation of this virtual library approach. You can only link one implementation at a time to an executable (or test). Thus, the dune rules for running the dns_forward test suite became:

; Uses the default  implementations provided by virtual libraries
(executable
 (name test)
 (libraries dns_forward alcotest))

; Override the defaults and select the private fake libraries
(executable
  (name test_fake)
  (libraries dns_forward alcotest fake_sleep fake_time))

With all of this machinery in place, I could now update test_fake to use the new modules that would provide the custom functionality for the clock and time implementations.

(* The bad server should be marked offline and no-one will wait for it *)
Fake_time_state.reset ();
(* avoid the timeouts winning the race with the actual result *)
Fake_time_state.advance Duration.(of_ms 500);

Conclusion

February 28, 2025

The switch to dune virtual libraries seems to be a good one. A majority of applications only require a single clock or time implementation. However, it has become harder to inject custom implementations of various resources into your MirageOS-based applications.

The pull request to upgrade VPNKit is open.

Reviving OCaml R-Tree [ocaml-rtree]

August 9, 2023
Patrick Ferris

Antonin Guttman first proposed R-Trees in 1984.

In order to handle spatial data efficiently, as required in computer aided design and geo-data applications, a database system needs an index mechanism that will help it retrieve data items quickly according to their spatial locations...

There are lots of excellent resources for understanding more about R-Trees and all of the wonderful variations there of (e.g. R*-trees). This blog post seems to be the most popular introduction.

An OCaml Rtree Implementation [ocaml-rtree-001]

Patrick Ferris

In the Energy and Environment Group we are working on various geospatial tools, one of which is written in OCaml and needed a geospatial index to speedup spatial queries over a dataset. Searching Github revealed an implementation in pure OCaml. After reaching out to the author, they were more than happy for their work to be maintained in the Geocaml organisation, updated to the latest OCaml tools and hopefully released on opam.

The dunification went smoothly whilst fast-forwarding 15 OCaml compiler versions with no changes or problems (just dune unused variable warnings).

OCaml Rtree Examples [ocaml-rtree-002]

Patrick Ferris

An R-Tree in OCaml is created from an envelope (the kind of bounding box) and a value (for storing in the tree). The value depends on the envelope as you have to provide a function to calculate an envelope from a value. Here is one way we might define a line using the provided two-dimensional Rectangle envelope.

module Line = struct
  type t =
    { p0 : float * float; p1 : float * float }

  let t =
    let open Repr in
    record "line" (fun p0 p1 -> { p0; p1 })
    |+ field "p0" (pair float float) (fun t -> t.p0)
    |+ field "p1" (pair float float) (fun t -> t.p1)
    |> sealr

  type envelope = Rtree.Rectangle.t

  let envelope { p0 = (x1, y1); p1 = (x2, y2) } =
    let x0 = Float.min x1 x2 in
    let x1 = Float.max x1 x2 in
    let y0 = Float.min y1 y2 in
    let y1 = Float.max y1 y2 in
    Rtree.Rectangle.v ~x0 ~y0 ~x1 ~y1
end

In addition to the envelope function, we also need to provide a runtime representation using Repr. In the example, we've constructed it by hand but I recommend using ppx_repr to derive it for you. From here we can create our index.

module R = Rtree.Make (Rtree.Rectangle) (Line)

Inserting Values [ocaml-rtree-003]

Patrick Ferris

There are two ways to insert values into the index -- either by inserting them one-by-one or by bulk loading them. The latter is the recommended approach where possible as the resulting layout of the R-Tree is much better.

# let r =
    let i = R.empty 4 in
    R.insert i Line.{ p0 = (1., 1.); p1 = (2., 2.) };;
val r : R.t = <abstr>

As you might expect, bulk loading takes a list of values instead.

# let r =
    let lines =
        Line.[
        { p0 = (0., 0.); p1 = (1., 1.) };
        { p0 = (1., 1.); p1 = (2., 2.) };
        { p0 = (2., 2.); p1 = (3., 3.) };
        { p0 = (3., 3.); p1 = (4., 4.) };
        ]
    in
    R.load ~max_node_load:4 lines;;
val r : R.t = <abstr>

Spatial Queries [ocaml-rtree-004]

Patrick Ferris

The lines we loaded into the last index are really just four segments of the line y = x. Let's spatially query for the first two segments.

# R.find r (Rtree.Rectangle.v ~x0:0. ~y0:0. ~x1:2. ~y1:2.);;
- : Line.t list =
[{Line.p0 = (0., 0.); p1 = (1., 1.)}; {Line.p0 = (1., 1.); p1 = (2., 2.)};
 {Line.p0 = (2., 2.); p1 = (3., 3.)}]

You can see the query actually returned the first three, that's because all bound checks are inclusive.

The source repository includes a fairly simple example of rendering an R-Tree using Vg. This can be done outside of the repository using the tree representation.

# #show R.tree;;
val tree : R.t -> R.tree
type nonrec tree =
  R.tree =
    Node of (Line.envelope * R.tree) list
  | Leaf of (Line.envelope * Line.t) list
  | Empty

Here is an example rendering from the repository showing some random lines and the envelopes of the R-tree.

Backlinks

Fellowship Roundup [fellowship-roundup]

February 9, 2026
Patrick Ferris

Overview

February 9, 2026
Patrick Ferris

February is the final month of my current six-month fellowship work with Tarides. A lot of the work can be summarised by reading the roundups (see October, November, December and January). I have been writing throughout the fellowship. However, they do not quite capture all the work I have been doing.

Below, I pick individual projects and expand on them more holistically rather than pointing at individual PRs or issues.

Ppxlib

February 9, 2026
Patrick Ferris

Most of my time over the past six months has been devoted to ppxlib maintenance. A large proportion of this time has been solo development work. Ppxlib is a cornerstone library of the OCaml ecosystem (whether people like it or not).

$ opam list --depends-on=ppxlib --recursive | wc -l
2030

Moreover, it is completely invaluable to Jane Street too. Like other AST-based tools (e.g. ocamlformat, merlin), ppxlib must follow a regular release schedule to support newer compilers. During the years of multicore OCaml development, ppxlib maintenance was a little easier as the Parsetree rarely changed. However, since then, the same cannot be said. Since OCaml 5.2, here are just some of the Parsetree changes:

Functions are represented according to their arity.
Local module opens on types.
Effect syntax.
Modular explicits.
External types.
Locations for all parts of long identifiers.
Labeled tuples.

In a perfect world, each of these require a codec for serialising the feature into ASTs that do not support the feature, plenty of tests and new Ast_builder/Ast_pattern functions for using the feature. Nathan and I have been managing pretty well I would say, though we are not getting any technical debt work done.

Outreachy

February 9, 2026
Patrick Ferris

As the OCaml coordinator for Outreachy I have been running the latest December 2025 round. We are lucky to have four projects on the go!

Thibaut Mattio is mentoring two projects: an ML dashboard for Raven and an OxCaml backend for Raven.
Xvw is mentoring a Yocaml project.
I am mentoring a Geocaml project.

Eio

February 9, 2026
Patrick Ferris

During my work on tools like Shelter I have made a few improvements to Eio including:

OxCaml

February 9, 2026
Patrick Ferris

I have enjoyed my work on OxCaml went it has happend. I think a few tools have been useful including the try-oxcaml work. David and I also had fun at ICFP discussing some quick experiments related to io_uring and OxCaml.

One road-blocker here is the comparative amount of time spent trying to keep the OxCaml ecosystem working. Given my limited time working on fellowship projects, the rate of change of OxCaml (including the opam ecosystem) meant most of the time was spent making it just work, not exploring or experimenting with the features themselves. I think this has got better in recent months and perhaps jumping back in I would be surprised at the progress that has been made there.

Forester

February 9, 2026
Patrick Ferris

Tarides are funding some work on forester. This site uses forester by way of graft and I have spent some time working on these tools during the fellowship.

This includes a bytesrw-based bibtex library in OCaml that could be released soon.

Papers

February 9, 2026
Patrick Ferris

I had the great fortune of attending ICFP where I presented two talks and was co-author on a few others too.

Reference. Functional Networking for Millions of Docker Desktops (Experience Report) [madhavapeddy2025docker]

2025
Anil Madhavapeddy, David J. Scott, Patrick Ferris, Ryan T. Gibb, Thomas Gazagnaire
10.1145/3747525
https://dl.acm.org/doi/10.1145/3747525

Abstract

2025
Anil Madhavapeddy, David J. Scott, Patrick Ferris, Ryan T. Gibb, Thomas Gazagnaire

Docker is a developer tool used by millions of developers to build, share and run software stacks. The Docker Desktop clients for Mac and Windows have long used a novel combination of virtualisation and OCaml unikernels to seamlessly run Linux containers on these non-Linux hosts. We reflect on a decade of shipping this functional OCaml code into production across hundreds of millions of developer desktops, and discuss the lessons learnt from our experiences in integrating OCaml deeply into the container architecture that now drives much of the global cloud. We conclude by observing just how good a fit for systems programming that the unikernel approach has been, particularly when combined with the OCaml module and type system.

Reference. What we talk about when we talk about scientific programming [ferris2025scientific_programming]

2025
Patrick Ferris
https://watch.eeg.cl.cam.ac.uk/w/aYXqXLtgQawYMjVXjSQtjx

Abstract

2025
Patrick Ferris

Programming for the planet undoubtedly involves programming scientifically, but what kind of programming are we talking about and what makes it scientific? In what ways does it differ from other forms of programming, if at all? Is scientific programming, data science or machine learning fundamentally different to constructing a compiler or building a high-throughput web server? By considering how the scientific method (with its falsifiable hypotheses and repeatable and reproducible experiments) relates to scientific programming, I hope to explore how computer science and traditional programming techniques are coming up short in meeting the requirements of scientific programmers.

Reference. Generating a corpus of Hazel programs from ill-typed OCaml programs [ferris2025hazel_of_ocaml]

2025
Patrick Ferris, Anil Madhavapeddy
https://patrick.sirref.org/var/tyde2025.pdf

Abstract

2025
Patrick Ferris, Anil Madhavapeddy

When developing a new programming language, having a large corpus of both correct and incorrect programs allows language designers to test and explore the capabilities of their new language. However, bootstrapping such a corpus of incorrect programs is time-consuming and arduous. We therefore explore how to reuse code from more mature languages to generate a corpus of ill-typed code for newer ones. We have developed a compiler to Hazel, an emerging language with typed holes, from the more mature OCaml ecosystem. We find it practical to generate a comprehensive corpus of ill-typed programs for Hazel development, and discuss future larger scale efforts towards bridging ecosystems.

Reference. Decomposable Type Highlighting for Bidirectional Type and Cast Systems [carroll2025decomposable_type_highlighting]

2025
Max Carroll, Anil Madhavapeddy, Patrick Ferris
https://maxcarroll0.github.io/papers/workshops/HATRA-decomposable-type-highlighting/

Abstract

2025
Max Carroll, Anil Madhavapeddy, Patrick Ferris

We explore how to provide programmers with an interactive interface for explaining the process by which static types and dynamic casts are derived, with the goal of improving the debugging of static and dynamic type errors. To this end, we define mathematical foundations for a decomposable highlighting system within a bidirectional system, and show how these can be propagated through dynamic types in a cast system. Our prototype implementation in the gradually typed Hazel language includes a web-based user interface, through which we highlight the importance of type level debugging.

Reference. Yirgacheffe: A Declarative Approach to Geospatial Data [dales2025yirgacheffe]

2025
Michael Winston Dales, Alison Eyres, Patrick Ferris, Francesca A. Ridley, Simon Tarr, Anil Madhavapeddy
10.1145/3759536.3763806
https://dl.acm.org/doi/10.1145/3759536.3763806

Abstract

2025
Michael Winston Dales, Alison Eyres, Patrick Ferris, Francesca A. Ridley, Simon Tarr, Anil Madhavapeddy

We present Yirgacheffe, a declarative geospatial library that allows spatial algorithms to be implemented concisely, supports parallel execution, and avoids common errors by automatically handling data (large geospatial rasters) and resources (cores, memory, GPUs). Our primary user domain comprises ecologists, where a typical problem involves cleaning messy occurrence data, overlaying it over tiled rasters, combining layers, and deriving actionable insights from the results. We describe the successes of this approach towards driving key pipelines related to global biodiversity and describe the capability gaps that remain, hoping to motivate more research into geospatial domain-specific languages.

I had some great conversations with KC too with respect to my current research.

Future Work

February 9, 2026
Patrick Ferris

As I enter my final year of the PhD, my goal would be to try and align more of my fellowship work with my own research wherever it makes sense. This research focuses on the following hypothesis:

Embedding deep provenance tracking, reversible execution and mergeable histories directly into an interactive programming environment (a POSIX-like shell) greatly decreases the gap between exploratory scientific work and reproducible, publishable results; all whilst leaving existing workflows intact and being programming language agnostic.

I want to focus on building these tools using Irmin, Eio, OxCaml etc. There are other projects I wish to continue regardless, like Outreachy, though just as a coordinator (not a mentor).

Overlapping PhD Work

February 9, 2026
Patrick Ferris

There are multiple projects that overlap with my work. For example, Irmin, which I wrote a retrospective for. This is at the core of Shelter and suggests an extremely valid use case for a brancheable, mergeable database.

In addition to this, continuing to work on Eio makes sense to me too. Eio currently has no maintainer and no active development, but the issues are piling up. In particular, I am making heavy use of Eio in Merry, a POSIX(ish) shell written in OCaml. Aside from research angles on brancheable and mergeable shells, I am also quite interested in how this might look in terms of OxCaml. I met with Anil and Thomas recently to discuss the future of Eio and there was some agreement that we are all heavy users of it and hope to keep up its maintenance and set of features.

One large piece of work that is sorely needed here is moving Eio to bytes and not bigarray-backed Cstructs. Any libraries using bytesrw (e.g. jsont) incur an extra copy of data into and out of the bytes. Anil has been working on some of the pieces recently to make this better!

Maintaining and Releasing my own libraries

February 9, 2026
Patrick Ferris

I have published quite a few OCaml libraries in my time, below is just a few libraries (filtered to not include OCurrent projects).

$ opam list --all --no-switch --columns=name,authors: | grep -E "sirref|Ferris" | cut -d " " -f1
carbon
geojson
geojsone
graft
hilite
ISO3166
jekyll-format
ppx_deriving_ezjsonm
ppx_deriving_yaml
rtree
search
topojson
topojsone
cid
multibase
multicodec
multihash
multihash-digestif

Some of these are used in the community including hilite, ppx_deriving_yaml and the multi-codecs. I would like to do some general maintenance of some of these tools and release a few iterations of graft which now makes use of bib.

Ppxlib Maintenance

February 9, 2026
Patrick Ferris

By far, maintenance of ppxlib takes up a majority of my time. I think, given the other work I need to focus on, it would be great to try to minimise the amount of time working on ppxlib. In particular, perhaps only maintaining ppxlib as far as OCaml release-readiness is a possibility and important bug fixes (like the OOM bug reported by Jane Street).

Forester, Graft and Writing

February 9, 2026
Patrick Ferris

With all of this work, I am eager to communicate what I am up to and my thoughts on open source, OCaml, OxCaml etc. I am glad to be using Forester to do this, particularly as it is written in OCaml (and could make great use of my own work on Eio).

I hope to continue working on Graft and related tools like bib. But I would also like to start writing some long-form content similar to the Irmin retrospective.

Taking stock & Graft updates [weekly-2025-w45]

November 10, 2025
Patrick Ferris

I ended up working on a real mix of things last week. I managed to get two large posts uploaded to this site: an ICFP recap and a deeper-dive into the random bits of OCaml hacking. This was pretty useful in terms of taking stock of where I was at, I always find post-conference, I'm a little lost. The conversations and ideas from ICFP are simultaneously exciting but also pretty inactionable.

Graft Updates [graft-updates]

November 10, 2025
Patrick Ferris

It would seem that there is a potential second user of Graft! A great source of motivation for some tidying and feature implementing. At the end of last week I fixed some little bugs in Graft and (more importantly) allowed users to name their subtrees. Upstream Graft also now pins and uses my Bibtex implementation.

OCaml Geotessera

November 10, 2025
Patrick Ferris

As I discussed in my OCaml roundup, I spent a bit of time polishing the OCaml geotessera code. In particular this meant starting to handle low-level geospatial transformations allowing users to project latitudes and longitudes into the rows and columns of their raster data. I feel like I'm stumbling onto a path Michael has carved out with Yirgacheffe. I'm following along with his work excitedly and we had a good catch-up last week discussing how libraries should handle things like automatic, numeric precision conversion.

As a fully paid-up member of the static types fan club, I'm often evangelising the benefits of types. However, I hit a little quirk of the Nx library in OCaml (a numpy equivalent). Take a look at this sum function:

let sum nx = Nx.sum nx |> Nx.item [];;
val sum : ('a, 'b) Nx.t -> 'a
let arr = Nx.create Nx.int8 [| 2 |] [| 64; 64 |];;
val arr : (int, Bigarray.int8_signed_elt) Nx.t
sum arr;;
- : int = -128

Even though sum takes an ('a, 'b) Nx.t array that stores elements with representation 'b and allows you to read/write to the elements with OCaml values of type 'a, the sum function performs the calculation in the world of the 'b representation type (here signed 8-bit integers). So we get an overflow! Perhaps not too surprising but easy to overlook! The moral of the story is that we still need to be careful in the OCaml world too.

Shelter

November 10, 2025
Patrick Ferris

I did a little cleaning up of the Shelter code too. I noticed a few bugs from the demos I did at ICFP. Most were pretty straight-forward to fix, and the test-suite is now checking parallel for-loops and conditional execution are working.

This week

November 10, 2025
Patrick Ferris

I hope to get a few things done this week:

Run some part of the LIFE pipeline through Shelter.
Deploy the Shelter daemon allowing other people to connect and take it for a spin.
Spend some time writing and reading, pulling ideas together rather than always writing code!
I have a slightly harebrained scheme that combines Shelter, atproto, Forester, capn(proto/web) and webcomponents... I want to see if I can pull that together.

OCaml Roundup: October 2025 [ocaml-roundup-october-2025]

November 5, 2025
Patrick Ferris

Welcome to a monthly roundup of OCaml-related open-source work I have been involved with. If you haven't already, a quick look at my ICFP roundup might be a good preface to what follows.

Outreachy December 2025

November 5, 2025
Patrick Ferris

The contribution period for this year's Outreachy round took place for most of the month of October. This year I am excited to be going back to a Geocaml project, working primarily to add writing capabilities to ocaml-tiff.

As per usual, I have been extremely pleased with the level of interest in the various OCaml projects being offered. Whilst we are still deliberating on our choices for interns this year, I won't say much more on the contributions, but if you want to get involved please do reach out! OCaml is a great language for geospatial work, striking a balance between reasonable performance for numerical code and also a rich type system for expressing complicated geospatial data-structures.

Ppxlib and the Future of Compiler Support

November 5, 2025
Patrick Ferris

Nathan has put a tremendous amount of effort into laying a roadmap this month for how ppxlib, going forward, will support future compilers. This is in response to the calamity that was bumping the internal AST to 5.2, and whilst I think we are adding a good deal more complexity, it will hopefully keep the ppx universe on an even keel for longer periods of time.

The first substantial PR for this modern world of ppxlib is in review!

Aside from that I also debugged an issue that lead to ppxlib OOM-ing machines... of course, any recursive fold-like loop where the accumulator keeps growing is likely to do that! It was nice to be able to crack out memtrace to be able to hunt it down:

Incremental Directory Reading for Eio

November 5, 2025
Patrick Ferris

A common tasks for many a CLI tool is to scan a directory of files, filter out the ones you are looking for and apply some transformation to said files. In fact, that is what Graft and Forester both do! A standard approach to this involves a recursive dance of readdir and stat, the former lists the entries in a directory whilst the latter lets you know what kind of file is at a particular path (e.g. symbolic link, directory etc.). This has a few downsides:

We're making two systems calls (pretty much one of the most expensive things your program can do).
If a directory contains thousands of files, we end up allocating a big list of strings after returning from readdir.

On Linux, we can get around most of that with getdents64(2) which allows us to incrementally read a directory. Whatsomore, this system call also returns the file type so there's no need to do an extra stat!

I prototyped an implementation of this called Eio.Path.walk which looks like:

val walk : _ t -> ((File.Stat.kind * string) Seq.t -> 'a) -> 'a
(** [walk t] traverses the directory [t] producing a sequence of results.*)

We effectively halve the number of system calls. In a benchmark that traverse deeply-nested directories with plenty of files, a full traversal whent from 1.46s to 110ms. I have since rejigged the implementation to automatically recurse for the user into subdirectories (not that the signature is unchanged) which does make it tricky to control the number of open file descriptors compared to if the user were in control.

OxCaml Experiments [icfp-oxcaml-uring]

November 5, 2025
Patrick Ferris

I have included here a transclusion from my ICFP post about an experiment using unboxed 32-bit integers in OCaml's Uring library.

OxCaml and Uring [oxcaml-and-uring]

Patrick Ferris

After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change:

  module Heap = struct
    type ptr = int32#
    (* ... *)
  end

  type cqe = {
    user_data_id : Heap.ptr;
    res : int32#;
  }

x-ocaml with OxCaml

November 5, 2025
Patrick Ferris

I sent a few PRs to the OxCaml repository to get the toplevel working again and lucky the underlying culprit for why patches were disappearing was found. Watch this space for a new toplevel in the browser soon.

Geospatial OCaml

November 5, 2025
Patrick Ferris

In addition to the Outreachy work I mentioned above, I also found the time to keep the Geocaml project ticking along. A few projects saw some much-needed love including:

PROJ in OCaml

November 5, 2025
Patrick Ferris

A modernisation of the bindings to PROJ. PROJ is probably the most used and tested code for working with geospatial projections. Any time you get a sateillite image or a blob of GeoJSON, the geospatial data will normally have some coordinate reference system (CRS) with it detailing exactly what the numbers that tell you where something is mean. One of the most common being WGS84 (used by GPS and GeoJSON for example). Each CRS has its benefits and drawbacks, and often you will need to convert between them. That is what PROJ does.

WKT in OCaml

November 5, 2025
Patrick Ferris

Well-known Text (WKT) is a very simple encoding of geospatial objects and CRSs. For example, a polygon might look like:

Polygon ((10 10, 10 20, 20 20, 20 15, 10 10))

I used bytesrw to build a simple codec for the WKT format. It is probably about time to write the basic geospatial object library too, something similar to Rust's geo library.

Geotessera

November 5, 2025
Patrick Ferris

A lot of people in my group are excited about Tessera. They have made it easy to used the Tessera embeddings via a python library called geotessera. In true OCaml-at-heart spirit, I tried my hand at porting this library. I had a good deal of success, converting the data into Nx array, extracting the CRS and transform out of the GeoTIFF landmasks using ocaml-tiff, reading datasets using ocaml-geojson etc. This made me excited but also keen to improve the APIs of most of these libraries too.

I feel geocaml is at a critical mass where we can start to have this useful feedback loop for the libraries because there is enough functionality to actually use them!

OCaml Roundup: October 2025 › OxCaml Experiments [icfp-oxcaml-uring]

November 5, 2025
Patrick Ferris

I have included here a transclusion from my ICFP post about an experiment using unboxed 32-bit integers in OCaml's Uring library.

OxCaml and Uring [oxcaml-and-uring]

Patrick Ferris

After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change:

  module Heap = struct
    type ptr = int32#
    (* ... *)
  end

  type cqe = {
    user_data_id : Heap.ptr;
    res : int32#;
  }

x-ocaml with OxCaml

November 5, 2025
Patrick Ferris

Graft [graft]

June 2, 2025
Patrick Ferris
https://github.com/patricoferris/graft

Graft is a Forester preprocessor.

It takes a forester (a directory of trees) written in a mixture of Markdown, Bibtex and Forester syntax and produces a new forest completely written in Forester syntax.

Usage

June 2, 2025
Patrick Ferris

graft simply preprocesses a forest generating Forester trees from .md, .bib and .tree files. It will copy the structure of the input directory in the output directory.

$ graft preprocess --output=grafted-trees trees
$ forester build

This assumes that you have updated your Forester toml file to put the grafted-trees directory as your source of trees.

[forest]
trees = [ "grafted-trees" ]

Example

June 2, 2025
Patrick Ferris

A typical "tree" might look something like

---
title: Opentrace and Supervisions
date: 2025-05-26
author: Patrick Ferris
---

Over the past two weeks I have mainly split my time (amongst many things) developing [opentrace](open-trace)
and doing revision supervisions.

```forester
\put\transclude/numbered{false}
\transclude{open-trace}
```

A few things to note:

The yaml frontmatter allows you to add some of the metadata fields from Forester.
At any point in your markdown there is an escape hatch to Forester using a forester codeblock.

A Transpiler from OCaml to Hazel [hazel-of-ocaml]

May 2, 2025
Patrick Ferris

Typed Holes

May 2, 2025
Patrick Ferris

To get a flavour of Hazel, take a regular map function for lists.

let map = fun f -> fun xs -> case xs
  | [] => []
  | x :: xs => f (x) :: map(f)(xs) 
end in
map(fun x -> ?)([1, 2, 3])

The question mark (?) is a hole. The program evaluates to the following expression of type [?] (for people more familiar with OCaml types ? list).

[ ?, ?, ? ]

Hazel supports local type inference but nothing involving unification variables. For example, a simple add_one function in Hazel (fun x -> x + 1) has type ? -> Int.

From OCaml to Hazel

May 2, 2025
Patrick Ferris

A Transformation of Syntax

May 2, 2025
Patrick Ferris

Hazel markets itself as an "Elm/ML-like functional programming language". From the previous example of map, it should be apparent just how close to OCaml the language is.

It turns out that a majority of the transpiler is a transformation of syntax. Take a simple ADT for an arithmetic programming language.

type expr =
  | Float of float
  | Add of expr * expr
  | Sub of expr * expr
  | Mul of expr * expr
  | Div of expr * expr

And when we run hazel_of_ocaml over this OCaml type declaration.

type expr =
  + Float(Float)
  + Add((expr, expr))
  + Sub((expr, expr))
  + Mul((expr, expr))
  + Div((expr, expr))
 in ?

If we try out the same map function but written in OCaml and transpiled to Hazel we get.

let rec map f = function
  | [] -> []
  | x :: xs -> f x :: map f xs

Which becomes the following hazel program.

let map = fun f -> fun x1 -> case x1
  | [] => []
  | x :: xs => f(x) :: map(f)(xs)
end in ?

Explicit Polymorphism

May 2, 2025
Patrick Ferris

Hazel has explicit polymorphism. So far, we have not seen it as we have let the types have holes in them. The map function in OCaml has the following type.

val map : 
  ('a -> 'b) -> 'a list -> 'b list

We must remind ourselves (by reading Craig's excellent blogpost on the matter) that in OCaml

... type variables in signatures are implicitly universally-quantified

So in reality, we have that map has the following type.

val map : ∀ a b. (a ->  b) -> a list -> b list

let map : forall a -> forall b -> (a -> b) -> [a] -> [b] =
  typfun a -> typfun b -> fun f -> fun xs -> case xs
    | [] => []
    | x :: xs => f (x) :: map@<a>@<b>(f)(xs) 
end in
map@<Int>@<Int>(fun x -> ?)([1, 2, 3])

Propagating OCaml Types into Hazel

May 2, 2025
Patrick Ferris

Most often, OCaml users interact with prenex polymorphism (rank-1) where the universal quantifiers are at the front of the type. OCaml does support quantifiers inside certain types like records.

Take a simple, polymorphic length function.

let rec length = function
 | [] -> 0
 | _ :: xs -> 1 + length xs

let int_len = length [ 1; 2; 3 ]
let str_len = length [ "only"; "two" ]

When we run this through hazel_of_ocaml with the -type flag we get.

let length : forall a -> [a] -> Int = typfun a -> fun x1 -> case x1
  | [] => 0
  | _ :: xs => 1 + length@<a>(xs)
end in
let int_len : Int = length@<Int>(1 :: 2 :: [3]) in
let str_len : Int = length@<String>("only" :: ["two"])
in ?

hazel_of_ocaml has correctly instantiated the type for length inside the recursive function and then in each case with the integer list and the string list.

A Corpus of Hazel Programs

May 2, 2025
Patrick Ferris

Future Work

May 2, 2025
Patrick Ferris

Another line of research would be to have Hazel target one of the intermediate representations in OCaml which would give Hazel a fully functioning compiler to "native" code?

person. Anil Madhavapeddy [anilmadhavapeddy]

https://anil.recoil.org

Professor of Planetary Computing at University of Cambridge.

Software. Bib [bib]

Patrick Ferris
https://git.sirref.org/ocaml-bibtex

Bib is a simple bibtex codec for OCaml.

Documentation

Patrick Ferris

The documentation is available online.

Releases

Patrick Ferris

All versions of Bib are available:

Bib: Initial release (0.1.0) [bib-release-0-1-0]

August 21, 2026
Patrick Ferris
https://patrick.sirref.org/software/bib/releases/bib-0.1.0.tbz

Initial release of Bib, a bytesrw-codec for reading and writing bibtex files.

Please consult the documentation for how to use the library.

Forester [forester]

Michael W. Dales [mdales]

https://digitalflapjack.com

Project. Outreachy [outreachy]

https://www.outreachy.org/

Outreachy provides internships in open source and open science. Outreachy provides internships to people subject to systemic bias and impacted by underrepresentation in the technical industry where they are living.

I am one of the coordinators for the OCaml community for Outreachy. I have also mentored multiple three-month internships for the community.

Write support in OCaml TIFF library [outreachy-ocaml-tiff]

February 9, 2026
Patrick Ferris

I am mentoring Tambe Salome during the December 2025 Outreachy round to add support for writing TIFF files in the ocaml-tiff library.

You can now see the video of the demonstration day presentation:

Implement a non-blocking, streaming codec for TopoJSON [topojson-0001]

December 1, 2022
https://github.com/geocaml/ocaml-geojson

GeoJSON is a JSON standardised format for handling simple geographical data. One common use of GeoJSON is to add map overlays such as with the popular JavaScript library leaflet. One issue with GeoJSON is redundant data, for example when two polygons share boundaries ideally only one would contain the full data about the boundary, however, with GeoJSON the data is duplicated.

TopoJSON is an extension to GeoJSON to encode topology. This allows for redundant data to be removed and file sizes to be greatly reduced. This is often very desirable especially when working with data in the browser. In a previous Outreachy internship a new OCaml library was implemented to provide an OCaml library for TopoJSON, this project will build on this adding more functionality to the library and providing a non-blocking, streaming codec version similar to the geojsone library.

Extend OCaml's GeoJSON library to support TopoJSON [topojson-0002]

June 1, 2022
https://github.com/geocaml/ocaml-geojson

Improve the ocaml.org Website [ocamlorg-0001]

June 1, 2021
https://ocaml.org

OCaml.org is the main website for OCaml, a functional, typed, high-level programming language. This project revolves around improving the website on multiple different fronts including: layout, accessibility and content.

Because a lot of the content of the website is about OCaml and the website itself is written in OCaml, this would be a great opportunity for interns to learn the language and functional programming concepts.

Ryan Gibb [ryangibb]

https://ryan.freumh.org

Fellow FN06 hacker.

Shelter [shelter]

https://tangled.sh/@patrick.sirref.org/shelter

A shim between a user and the shell that provides greater reproducibility and insight into what your code is doing!

Give me Shelter!

Shelter combines the caching and reproducibility of Dockerfiles with the ease of interaction of the shell. I'd be interested to use Ryan's package management work as a means to specify the base environment users want.

Here is an example of running Shelter and making use of the branchable sessions.

shelter> echo hello > hello.txt
shelter[main#bbd691a] : { mode:rw }> @ session exp-1shelter[exp-1#bbd691a] : { mode:rw }> echo world >> hello.txtshelter[exp-1#dd5bab8] : { mode:rw }> @ session mainshelter[main#bbd691a] : { mode:rw }> echo "to the" >> hello.txtshelter[main#b9abef4] : { mode:rw }> cat hello.txt
hello
to theshelter[main#d1c1728] : { mode:rw }> @ session exp-1shelter[exp-1#dd5bab8] : { mode:rw }> @ replay mainshelter[exp-1#df1b4e1] : { mode:rw }> cat hello.txt
hello
to the
worldshelter[exp-1#67162b7] : { mode:rw }>

The @ character allows the user to interact directly with Shelter's build-in operators. @ session exp-1 creates a new session as there is no exp-1 session. It points to the head commit of the branch it came from (git checkout -b exp-1).

From there we make changes and switch between branches until we perform a replay. A replay is a rebase without any merge conflict checks, it simply finds the least common ancestor of the two branches and re-applies the commits from your current branch (here exp-1) onto the head of the target branch (here main).

Under the hood

Shelter is a light-weight, reimplementation of obuilder. It uses the similar ideas, like snapshotting filesystems (e.g. ZFS) and Linux namespaces (e.g. Void). Between the low-level store and the interactive shell parts, there is an Irmin store.

Type Error Debugging in Hazel [part-ii-hazel]

This project will add some features to the Hazel language. Hazel is a functional research language that makes use of gradual types to support unusual features such as: holes (code place-holders) to give type meaning to incomplete programs. Importantly for this project, all Hazel programs, even ill-typed or incomplete programs, are evaluable. This allows dynamic reasoning about ill-typed programs via evaluation traces with the potential to improve the user’s understanding of why ill-typed programs go wrong.
This project aims to exploit further this potential by providing some extra features to both: aid with finding values/inputs that demonstrate why type-errors were found (type-error witnesses) and linking the evaluation traces back to source code.