Forester [forester]

Backlinks

Weekly 2026 W43 [weekly-2026-w43]

October 26, 2026
Patrick Ferris

This week was a bit of a flop as I ended up coming down with something and was unable to work for about half of it. A shorter note this week due to this.

Paper Writing

October 26, 2026
Patrick Ferris

I had a productive conversation with Anil about the direction of the paper I am currently writing. Plenty of work to do with the deadline looming, but I am starting to think the framework we're building could have wider use than the applications I first thought of and that is somewhat exciting.

More on this during the week as I'm back to operational strength!

Forester 6.0

October 26, 2026
Patrick Ferris

A new release of Forester is imminent and with that comes upgrades and fixes to Graft. If you are reading this post then you are enjoying the delights of Forester 6.0 compiling artefacts from an in-the-works version of Graft.

The upgrade was mostly smooth, though some of my CSS was a bit broken, and it was easier to do a merge of the most important bits with the new Forester stylesheets.

Depending on the internals of Forester is a little tricky as it is first and foremost a CLI tool. The library APIs are likely to be less stable and are a bit of a moving target. The changes, however, were pleasantly small as far as I can tell, limited mostly to some changes in the reporting infrastructure and the strictness of all Forester codes needing a location (somewhat tricky as there is no one-to-one mapping from markdown/yaml/bibtex to these codes). For the latter case, we do our best and where possible try to map the generated codes back to the fragment of the source language Graft has transpiled.

Now that I am dog-fooding Graft and Forester, it will be easier to keep things working ahead of the next release.

Thanks to Kento and Jon for their work on improving Forester.

Outreachy

October 26, 2026
Patrick Ferris

I have lots of thoughts about Outreachy this year. The volume and quality of contributions is quite unlike anything I have seen before, in large part due to the current ease of accessibility and quality of LLMs. This does fly in the face of somewhat strong policies set out by Outreachy to help maintainers handle AI contributions.

Going forward, I imagine it will be difficult to balance pedagogical values with financial rewards given wide-spread AI usage (regardless of project-specific or organisational AI policies). Not something that quite exists, in the same way, say, for university undergraduate students.

I am still trying to find my feet in the middle of this. I have never felt quite this alone in the open-source software world.

Misc.

October 26, 2026
Patrick Ferris

Claudius Playground

October 26, 2026
Patrick Ferris

Michael spoke to me about some accessible ways to play around with his fantasy console library Claudius. At this point, I fancy myself a bit of an expert at shoving OCaml code into browsers, so I pulled together a rough and ready Claudius Playground (code is freely available on tangled). Happy coding!

Music

October 26, 2026
Patrick Ferris

I was able to go see the Mountain Goats before the start of last week, with Craig Finn supporting. I enjoyed Isaiah 45:23.

The Mountain Goats at Vicar St. Dublin, 2026-10-18.

Uniform Representations [uniform-repr]

October 12, 2026
Patrick Ferris

Whilst working on things related to Merry and Shelter, I couldn't help but think how much easier parts of this problem would be if we were using Plan9 for one simple reason: the uniform access to OS resources. Half the battle on Linux is trying to trace or sandbox a process with respect to some set of OS resources (in an ideal world, all of them). Syscalls are perhaps the closest thing we have, but there isn't a one-to-one mapping between user actions and syscalls (e.g. opening a file using io_uring does not call open(2) or openat(2)).

I think there is often a trade-off here in terms of performance and simplicity. Plan9's 9P protocol was not exactly known for its performance. But, I still find myself, to use a phrase my colleague Ryan once used when explaining his use of emacs, comfy with uniform representations. Another example, a tree in Forester.

It is pleasant when these representations have a defined structure, even if the contents are more malleable. For instance, I don't want everything to be schema-less JSON. Forester trees have tags and that allows me to impose a schema in a structured way that doesn't lead to parsing errors. Similarly, 9P offers a simple protocol on which all things are built. As I understand it, the semantics are given by the interpretation of the file server that is resolved, but the protocol is uniform. Anyway, just some thoughts.

Shells, SOOCon26 and Part IIs [weekly-2026-w40]

October 9, 2026
Patrick Ferris

Last week I gave a precise meaning to the "mergeability" of container images. The further I go down this rabbit hole, the less convinced I am of the exact benefits of such an approach. However, there are lots of off-shoots to explore as well, which all seem very worthwhile and might have better semantics than merging (e.g. incremental computation and provenance-tracking).

Uniform Representations [uniform-repr]

October 12, 2026
Patrick Ferris

State of Open Conference 2026 -- Belfast [soocon2026-belfast]

October 9, 2026
Patrick Ferris
https://stateofopencon.com/belfast-soocon26/

This week I was fortunate enough to attend (as a volunteer) SOOCon26 in Belfast in the Ulster Bank building, just opposite the City Hall. Thanks to Anil Madhavapeddy for linking me up with Amanda Brock to make it happen at the last hour.

If I can boil down the conference into a general topic it would be: how are large language models and coding agents impacting commercial, business-oriented open-source software. I found the conference's near-total focus on AI a little surprising and a little disappointing, but I suppose it reflects the current state of open source. For example, most, if not all, the working developers and business owners at this conference, for one reason or another, felt the need to relate their work to AI, both in their talks and in hallway conversations. Something I continue to struggle with, though, is that almost all references to "AI" pointed to large, expensive, closed frontier models (such as Claude, ChatGPT, Copilot and Gemini). That being said, there were talks on data (and AI) sovereignty as well as mentions of the increasingly capabale open models (e.g. Qwen).

I would like to add that the conference was essentially offline: no recordings or streaming. With that in mind, I want to respect the fact that most attendees and presenters could speak freely and candidly, so I will only focus on some of the higher level threads of discussion without attributing too much to any individual.

AI: Productivity

October 9, 2026
Patrick Ferris

The morning focused mainly on the question: what is a software developer's job in 2026? This culminated in a panel discussion on "building trust in AI". Already, I would say the conference's mind was made up, AI is here to stay; it is just a tool; we should learn how to hold it properly. I think, in part, this was mainly due to the group of speakers who were all, more or less, executives within for-profit businesses that incorporate open source in some way: Github, BIMP.ai and Budibase. The only exception was a speaker from the Institute of Directors. I think this gave the morning session a commercial flavour which suggested the tech industry's large-scale adoption of AI tooling, a position I'm not sure is as common among, or available to, smaller open-source maintainers.

One stream of thought that was popular was arguing that languages like C, where it is easy to introduce memory bugs and vulnerabilities, are on the way out. Copilot's runtime rewrite in Rust was championed as a success story for the kinds of productivity that can be unlocked for highly-skilled developers. A pinch of salt is needed though, as Microsoft did sponsor the event. My own thoughts, however, are that there are quite a few success stories we can learn from where software was translated from one programming language to another using an LLM. Migration seems well-defined and well-scoped, often with large, pre-existing test suites that can easily guide a tool towards some form of correctness. In the same breath, it is not necessarily that straightforward, for example, see some lobste.rs commentary on the Bun rewrite in Rust.

Another undercurrent throughout the morning talks was how LLMs could free up developers to actively engage with users. Some quotes from the talks included "experiment with empathy" and "AI doesn't write the story". On the surface, this seems like a reasonable argument. But, shouldn't these large teams of people have already been thinking about their users, with or without AI? I'm also not convinced this free time will appear. When the metric of success is at least proportional to the rate of new features, it would seem that a tool, like an LLM, would only crank out more widgets rather than better widgets.

A final worrying conversation from the panel, discussed the fact that companies are not hiring graduates. This was self-evident from a few other people I met in and around the conference: computer science graduates still looking for jobs.

AI: Community, Data Centres and Vulnerabilities

October 9, 2026
Patrick Ferris

The post-lunch session included a talk about the future of community open source, a talk about AI data centres and finally a panel about the "vulnpocalypse".

The first talk focused on the realities of doing community open source and how, in effect, AI and LLMs are not (yet) a part of our shared open culture, citing large, anti-AI open-source projects, some which have banned AI contributions (e.g. NLNet Labs, Codemirror) and pro-AI projects which have closed contributions (e.g. Ladybird). Of course, open source never meant the right to contribute, but it still signals a, perhaps, worrying shift in open-source, cultural norms. An arXiv paper collected some AI policies on popular repositories which is worth a skim.

Another topic in the talk was the "time to open". The speaker argued that it was now possible to completely reverse engineer closed-source products and tools in as little as 48 hours. This could lead to more open source business models, as closed models are no longer viable. The speaker suggested that the open-source community around a project is now more important than the code itself. I think I fall somewhere closer to believing that they are both important. I worry about the proliferation of agentically-managed software stacks which companies and individuals use with little incentive to upstream patches back to the community (or release their new code for the community). This kind of isolationism seems more likely given the ability to create your own implementation of any piece of software.

Data Centres

October 9, 2026
Patrick Ferris

The talk on hyperscalar data centres presented some great work from the folks at Action to Protect Rural Scotland (APRS). With some serious energy projections:

This gives an estimated around 7,306 MW currently in the planning system or in pre-application consultation. Peak winter electricity demand in Scotland is just over 4,000 MW.

It seemed like the feeling in the room was that these data centres are inevitable, in spite of these jarring numbers; and that somewhere down the line, the ends will be just so good, that the means will be justified.

It seems, once again, I must reach for Pete Seeger.

Perhaps the biggest change will come when we don't have to change much at all When maniacs holler "grow, grow, grow, grow!"; we can choose to be small.

Vulnerabilities

October 9, 2026
Patrick Ferris

I don't think this needs too much explaining, but visit Daniel Stenberg's mastodon feed and that will catch you up. Edit: I wanted to also add a more recent post from Daniel highlighting the other work of the core curl team.

Conclusion of sorts

October 9, 2026
Patrick Ferris

Unfortunately, I missed a good portion of the final session as I was helping clear up. The talks were: "The cost of Sovereignty", "Knowledge Graphs for Digital Transformation" and "You can't download trust".

I managed to have a good chat with Phil Weir, director of Flax and Teal, about some of their work including Alizarin, a tool for working with knowledge graphs and Arches data. It was refreshing to hear about the details of an actual project someone was building.

Ultimately, I would say the day left me disheartened for the immediate future of (open-source) software. What I found most unsettling about SOOCon is what I find unsettling about the tech industry at large. It would seem that the industry is leaving new graduates in the lurch. Additionally, environmental concerns are falling somewhere (far?) below commerical concerns.

I'm also anxious that open source's ethos is being eclipsed by the need for companies (and individual employees) to demonstrate their relevance in an AI-focused economy, and that that performance of relevance is really about integrating, into whatever you're doing, the products coming out of a few powerful companies, which are not invested in the values which make open source make sense.

I think conferences like State of Open are important, and we need more conversations from all kinds of open-source community members. I'd like to see more talks from students, sole maintainers of highly depended-on software, folks from other non-profit-driven open-source initiatives like Outreachy or the Processing Foundation (these are just examples I know). I recognise that, perhaps, SOOCon26 Belfast was suffering from being in Belfast, hardly the most connected metropolis in the world... but it is the home of The Big Fish. I'm excited to hopefully see it return in the years to come.

My friends, Maddie and Emma, enjoying the splendour of Belfast!

Thanks to Meredith, my partner, for helping me edit this and form some much more coherent ideas.

Part II Projects

October 9, 2026
Patrick Ferris

After some super-human administration by Anil Madhavapeddy, I have closed this week with three part II projects to supervise.

An Embedded DSL for Querying OpenStreetMap [part-ii-edsl-osm]

Matthew Occleshaw

This project, originally conceived by Anil Madhavapeddy, hopes to make it easier to answer questions like "What is the safest route to Pembroke College from Cambridge train station?". Here, "safest", can have lots of meanings: the best lit road, a route with tactile paving or accessible pedestrian signals etc.

In addition to this, we may wish to provide suggestions for how to improve the current folksonomy of information OSM stores.

From OCaml to Graphics Shader Languages [part-ii-shaders]

Rohan Wadhawan

This project wants to reuse OCaml's typechecker to provide a safe way to create graphics shaders. As an extension, a ppx rewriter will provide a translation layer from OCaml to the embedded DSL making it possible to write shaders in a subset of the OCaml language.

When interacting with the GPU for graphics programming, parts of the rendering pipeline are written as standalone shader programs. The standard method of using these is to store the shader source code as a string, which is then compiled at runtime by the GPU driver. This is not a performance issue, as shader programs are usually small enough that compilation time is negligible. However, it does mean simple syntax errors or incorrect types, which would normally be caught at compilation, won’t manifest until the entire program is run.
I propose a new library, Shaders_of_ocaml, allowing for shader programs to be written inline as native OCaml, which is then compiled to shader code strings to be passed to the existing graphics libraries. This would allow the OCaml compiler to provide type checking for every shader written.
-- From Rohan's Proposal

Ripple: An OCaml Library for Incremental Computation [part-ii-incr]

Boris Hall

Following the work of Adaptive Functional Programming and Implicit self-adjusting computation, this project will build an incremental computation engine in OCaml alongside an expressive combinator library for build such computations. As an extension, a small DSL will be built that makes use of the engine to build an implicitly incremental programming language.

The core of this project is to implement an eDSL and a runtime in OCaml for self-adjusting computation. I also propose a new TDT which allows for generic monoidal folds over modifiable sequences: the library will include a native foldMap primitive that allows performant reduction over a modifiable sequence of modifiable values for any general monoid, giving logarithmic updates with a small trace. I will then compare foldMap to the generic fold implemented in plain SAC.
-- From Boris' Proposal

Ppxlib

October 9, 2026
Patrick Ferris

Just a quick footnote about [Ppxlib]: I opened a PR tomerge multiple instances of a deriver. I threw together this OCaml script for checking implementation and interface files. Running it on Forester, it picked up two instances which can be avoided.

open Ppxlib

let attr_compare (a1 : attribute) (a2 : attribute) =
  String.compare a1.attr_name.txt a2.attr_name.txt

let has_derivings_recursively =
  object
    inherit [bool] Ast_traverse.fold as super

    method! structure_item s acc =
      match s.pstr_desc with
      | Pstr_type (Recursive, (_ :: _ as tds)) ->
        let attributes =
          List.fold_left (fun acc td -> td.ptype_attributes @ acc) [] tds
        in
        let uniq = List.sort_uniq attr_compare attributes in
        List.length attributes <> List.length uniq || super#structure_item s acc
      | _ -> acc

    method! signature_item s acc =
      match s.psig_desc with
      | Psig_type (Recursive, (_ :: _ as tds)) ->
        let attributes =
          List.fold_left (fun acc td -> td.ptype_attributes @ acc) [] tds
        in
        let uniq = List.sort_uniq attr_compare attributes in
        List.length attributes <> List.length uniq || super#signature_item s acc
      | _ -> acc
  end

let main f =
  let is_ml = Filename.extension f = ".ml" in
  let is_mli = Filename.extension f = ".mli" in
  let kind =
    if is_ml then Some Ppxlib_private.Utils.Kind.Impl
    else if is_mli then Some Intf
    else None
  in
  match kind with
  | None -> Format.eprintf "Skipping %s@,%!" f
  | Some kind -> (
    match Driver.load_input ~kind ~relocate:false ~input_name:f f with
    | Error (e, _) ->
      Format.eprintf "Input err %a: %s" Location.print (Location.Error.get_location e) (Location.Error.message e);
      exit 127
    | Ok (_, _, Impl s) ->
      if has_derivings_recursively#structure s false then
        Format.printf ">>> Maybe fix: %s@,%!" f
      else ()
    | Ok (_, _, Intf s) ->
      if has_derivings_recursively#signature s false then
        Format.printf ">>> Maybe fix: %s@,%!" f
      else ())

Note, that given the implementation of attr_compare, this will be an overapproximation.

Shells, SOOCon26 and Part IIs [weekly-2026-w41]

October 9, 2026
Patrick Ferris

Uniform Representations [uniform-repr]

October 12, 2026
Patrick Ferris

State of Open Conference 2026 -- Belfast [soocon2026-belfast]

October 9, 2026
Patrick Ferris
https://stateofopencon.com/belfast-soocon26/

AI: Productivity

October 9, 2026
Patrick Ferris

AI: Community, Data Centres and Vulnerabilities

October 9, 2026
Patrick Ferris

The post-lunch session included a talk about the future of community open source, a talk about AI data centres and finally a panel about the "vulnpocalypse".

Data Centres

October 9, 2026
Patrick Ferris

The talk on hyperscalar data centres presented some great work from the folks at Action to Protect Rural Scotland (APRS). With some serious energy projections:

This gives an estimated around 7,306 MW currently in the planning system or in pre-application consultation. Peak winter electricity demand in Scotland is just over 4,000 MW.

It seems, once again, I must reach for Pete Seeger.

Perhaps the biggest change will come when we don't have to change much at all When maniacs holler "grow, grow, grow, grow!"; we can choose to be small.

Vulnerabilities

October 9, 2026
Patrick Ferris

Conclusion of sorts

October 9, 2026
Patrick Ferris

My friends, Maddie and Emma, enjoying the splendour of Belfast!

Thanks to Meredith, my partner, for helping me edit this and form some much more coherent ideas.

Part II Projects

October 9, 2026
Patrick Ferris

After some super-human administration by Anil Madhavapeddy, I have closed this week with three part II projects to supervise.

An Embedded DSL for Querying OpenStreetMap [part-ii-edsl-osm]

Matthew Occleshaw

In addition to this, we may wish to provide suggestions for how to improve the current folksonomy of information OSM stores.

From OCaml to Graphics Shader Languages [part-ii-shaders]

Rohan Wadhawan

When interacting with the GPU for graphics programming, parts of the rendering pipeline are written as standalone shader programs. The standard method of using these is to store the shader source code as a string, which is then compiled at runtime by the GPU driver. This is not a performance issue, as shader programs are usually small enough that compilation time is negligible. However, it does mean simple syntax errors or incorrect types, which would normally be caught at compilation, won’t manifest until the entire program is run.
I propose a new library, Shaders_of_ocaml, allowing for shader programs to be written inline as native OCaml, which is then compiled to shader code strings to be passed to the existing graphics libraries. This would allow the OCaml compiler to provide type checking for every shader written.
-- From Rohan's Proposal

Ripple: An OCaml Library for Incremental Computation [part-ii-incr]

Boris Hall

The core of this project is to implement an eDSL and a runtime in OCaml for self-adjusting computation. I also propose a new TDT which allows for generic monoidal folds over modifiable sequences: the library will include a native foldMap primitive that allows performant reduction over a modifiable sequence of modifiable values for any general monoid, giving logarithmic updates with a small trace. I will then compare foldMap to the generic fold implemented in plain SAC.
-- From Boris' Proposal

Ppxlib

October 9, 2026
Patrick Ferris

open Ppxlib

let attr_compare (a1 : attribute) (a2 : attribute) =
  String.compare a1.attr_name.txt a2.attr_name.txt

let has_derivings_recursively =
  object
    inherit [bool] Ast_traverse.fold as super

    method! structure_item s acc =
      match s.pstr_desc with
      | Pstr_type (Recursive, (_ :: _ as tds)) ->
        let attributes =
          List.fold_left (fun acc td -> td.ptype_attributes @ acc) [] tds
        in
        let uniq = List.sort_uniq attr_compare attributes in
        List.length attributes <> List.length uniq || super#structure_item s acc
      | _ -> acc

    method! signature_item s acc =
      match s.psig_desc with
      | Psig_type (Recursive, (_ :: _ as tds)) ->
        let attributes =
          List.fold_left (fun acc td -> td.ptype_attributes @ acc) [] tds
        in
        let uniq = List.sort_uniq attr_compare attributes in
        List.length attributes <> List.length uniq || super#signature_item s acc
      | _ -> acc
  end

let main f =
  let is_ml = Filename.extension f = ".ml" in
  let is_mli = Filename.extension f = ".mli" in
  let kind =
    if is_ml then Some Ppxlib_private.Utils.Kind.Impl
    else if is_mli then Some Intf
    else None
  in
  match kind with
  | None -> Format.eprintf "Skipping %s@,%!" f
  | Some kind -> (
    match Driver.load_input ~kind ~relocate:false ~input_name:f f with
    | Error (e, _) ->
      Format.eprintf "Input err %a: %s" Location.print (Location.Error.get_location e) (Location.Error.message e);
      exit 127
    | Ok (_, _, Impl s) ->
      if has_derivings_recursively#structure s false then
        Format.printf ">>> Maybe fix: %s@,%!" f
      else ()
    | Ok (_, _, Intf s) ->
      if has_derivings_recursively#signature s false then
        Format.printf ">>> Maybe fix: %s@,%!" f
      else ())

Note, that given the implementation of attr_compare, this will be an overapproximation.

OCaml Roundup: September 2026 [ocaml-roundup-september-2026]

October 7, 2026
Patrick Ferris

Plenty of OCaml work in September, including a few releases of libraries I maintain:

rtree got a new release.
ppx_deriving_yamlx saw its first release.
I managed to upstream some Forester work that was living inside Graft.
Some work on ppxlib that is on its way into the world.

Weeklies

October 7, 2026
Patrick Ferris

Weeknotes 2026-w38 [weekly-2026-w38]

September 18, 2026
Patrick Ferris

Msh log 2026-38 [msh-log-2026-w38]

September 18, 2026
Patrick Ferris

After running some Dockerfile analysis last week, I have focused on formalising some operational semantics for the core POSIX-shell inspired language for a "mergeable shell".

This work borrows from Smoosh, but extends the semantics to include reasoning about namespaces, filesystems and effects.

For instance, here are two inference rules for field splitting and parameter substitution. Here, $\sigma$ tracks the shell state, $\phi$ is the filesystem and $\tau$ tracks the side effects.

There are lots of interesting examples of this, like assignment during parameter substitution (e.g. ${FOO:=BAR}) or the side effects that take place during program name resolution.

OCaml Maintenance [ocaml-maintenance-2026-w38]

September 18, 2026
Patrick Ferris

A mixture of work in ppxlib and geocaml.

Ppxlib migration woes

September 18, 2026
Patrick Ferris

Ppxlib continues to prove itself as a tricky library to maintain. This week, a report came in from David about package_type attributes being dropped. These are new attributes that are attached to package expressions which simply did not have them before.

The solution here was to store these attributes on parent nodes (e.g. the core_type a Ptyp_pack might be a part of) when migrating from 5.4 to 5.3 alongside a ppxlib.migration attribute to distinguish them from the core_type's attributes. When migrating back up we then re-split the attributes.

David also landed a new syntax feature arriving in OCaml 5.6 that adds upstream support for expression "holes", something Merlin supports. It is now possible to replace any OCaml expression with _, for example, let x = 1 + _.

This needed explicit ppxlib support which, thankfully, was relatively straightforward as hole expressions do not have a lot of associated metadata or arguments.

Our benchmarking infrastructure is also back on its feet.

Thanks to Nathan for all the reviewing.

`rtree.0.3.0` release [rtree-0-3-0]

September 18, 2026
Patrick Ferris

I cut a release of ocaml-rtree that removes the repr dependency. See the relevant discuss post.

This was a feature that Michael had asked for when building laserver. I took the time to open a PR to laserver to use rtree.0.3.0 which does simplify the code.

I wanted to ensure the library was still working, so I grabbed some of the LAS data for Lough Neagh, provided by OpenDataNI for 2017. After converting it to cloud-optimised point cloud (COPC) data via pdal (yes, two different "cloud" meanings) I managed to view the south-east portion of Lough Neagh.

This is overlooking the Lough Neagh Discovery Centre and Croaghan Island.

Outreachy

September 18, 2026
Patrick Ferris

After securing funding for this Outreachy round, I submitted an ocaml-tiff project. A year ago, we had an intern successfully add write support to the library.

This time, I hope to continue building out the library with an intern who is interested in learning OCaml, functional programming and geospatial data formats! One subgoal of the internship will be to create a tiffinfo-like command line tool that is distributed as part of the library. I also hope to add more compression algorithm support and iron out some API kinks.

Weekly 2026 w39 [weekly-2026-w39]

September 30, 2026
Patrick Ferris

Shells

September 30, 2026
Patrick Ferris

Work on the msh paper continued this week. One interesting result (for an appendix) was seeing what were the most commonly used commands in the Dockerfile dataset:

Which again, leads to some notion of overkill in terms of using a shell as the means to create container images when all we do is move some files around and install packages! I'm wondering if we're missing a real opportunity to streamline this process via something like the package calculus -- a package manager-agnostic means for declaring your dependencies.

Embed that as a programmable notion in your container image language alongside a container image-aware HTTP GET tool (to replace a vast majority of curl calls) and a handful of busybox commands, and you are left with (nearly) the raw shell only being used for the long tail of domain-specific things (e.g. invoking a build system).

Joy in old documents

September 30, 2026
Patrick Ferris

I'll talk about the joy I found in reading some historical bits and bobs tracking down information about shells, containers and operating systems more generally.

Part of this archaeological effort was trying to find when CHROOT was introduced into UNIX. Various online tools, including AI summaries, point to UNIX v7 but the system call is not obviously defined there. A bit more digging reveals it is listed under CHDIR(2) and not on its own.

Anyway, flicking through the manual was somewhat of a joy. The preface reads:

This new form of the Seventh Edition manual attests to the gratifying popularity of the UNIX Operating System. Delivered with no support and with an amateurishly published manual, this elegant and productive system has nevertheless earned worldwide respect, sometimes bordering on adulation...
Thousands of users today work from faint reproductions of reproductions of the original manual, whose ragged pages dangle and slide out of beat-up ring binders...

This bordering-on-poetic prose reminds me of some passages written for the MULTICS manual too (which I have talked about before).

All to say, I'm often thinking about the importance of writing these days and injecting some personality into it.

Part II Discussions

September 30, 2026
Patrick Ferris

I am hopefully going to supervise a part II project this year either on a fun ergonomic frontend to eBPF programs or a new spin on the geospatial DSL. More to follow once things are finalised.

Upstreaming Forester Pretty-printer

September 30, 2026
Patrick Ferris

I finally got around to upstreaming the Forester pretty-printing API I have been using in Graft. This allows you to print, hopefully, syntactically correct Forester code directly from the AST (a Code.t in forester parlance).

Happily, this all took place on tangled. It will be nice to keep this updated alongside Forester and point Graft to it. It feels like many people are moving away from a community-oriented model of software development, instead building a hybrid bazaarian cathedral monorepo powered by AI. I'm not here to really say what is right and what is wrong, but it seems to me that the sense of community is fragmenting and undue stress is being placed on many maintainers who no longer enjoy their job.

New release of ppx_deriving_ [ppx_deriving_yamlx-release]

September 30, 2026
Patrick Ferris

Raphael has proposed a new yamlx backend for ppx_deriving_yaml. It seems Ahrefs have been using his patch. I rejigged that PR to parameterise ppx_deriving_yaml over a small set of constructors and patterns so that the core codec logic could be reused.

I am fairly pleased with this abstraction. It allows any data format that supports objects and list to reuse the codec. For example, it would be trivial now to add a ppx_deriving_cbor.

Weekly 2026 w39 [weekly-2026-w39]

September 30, 2026
Patrick Ferris

Shells

September 30, 2026
Patrick Ferris

Work on the msh paper continued this week. One interesting result (for an appendix) was seeing what were the most commonly used commands in the Dockerfile dataset:

Joy in old documents

September 30, 2026
Patrick Ferris

I'll talk about the joy I found in reading some historical bits and bobs tracking down information about shells, containers and operating systems more generally.

Anyway, flicking through the manual was somewhat of a joy. The preface reads:

This new form of the Seventh Edition manual attests to the gratifying popularity of the UNIX Operating System. Delivered with no support and with an amateurishly published manual, this elegant and productive system has nevertheless earned worldwide respect, sometimes bordering on adulation...
Thousands of users today work from faint reproductions of reproductions of the original manual, whose ragged pages dangle and slide out of beat-up ring binders...

This bordering-on-poetic prose reminds me of some passages written for the MULTICS manual too (which I have talked about before).

All to say, I'm often thinking about the importance of writing these days and injecting some personality into it.

Part II Discussions

September 30, 2026
Patrick Ferris

I am hopefully going to supervise a part II project this year either on a fun ergonomic frontend to eBPF programs or a new spin on the geospatial DSL. More to follow once things are finalised.

Upstreaming Forester Pretty-printer

September 30, 2026
Patrick Ferris

New release of ppx_deriving_ [ppx_deriving_yamlx-release]

September 30, 2026
Patrick Ferris

I am fairly pleased with this abstraction. It allows any data format that supports objects and list to reuse the codec. For example, it would be trivial now to add a ppx_deriving_cbor.

Fellowship Roundup [fellowship-roundup]

February 9, 2026
Patrick Ferris

Overview

February 9, 2026
Patrick Ferris

February is the final month of my current six-month fellowship work with Tarides. A lot of the work can be summarised by reading the roundups (see October, November, December and January). I have been writing throughout the fellowship. However, they do not quite capture all the work I have been doing.

Below, I pick individual projects and expand on them more holistically rather than pointing at individual PRs or issues.

Ppxlib

February 9, 2026
Patrick Ferris

Most of my time over the past six months has been devoted to ppxlib maintenance. A large proportion of this time has been solo development work. Ppxlib is a cornerstone library of the OCaml ecosystem (whether people like it or not).

$ opam list --depends-on=ppxlib --recursive | wc -l
2030

Moreover, it is completely invaluable to Jane Street too. Like other AST-based tools (e.g. ocamlformat, merlin), ppxlib must follow a regular release schedule to support newer compilers. During the years of multicore OCaml development, ppxlib maintenance was a little easier as the Parsetree rarely changed. However, since then, the same cannot be said. Since OCaml 5.2, here are just some of the Parsetree changes:

Functions are represented according to their arity.
Local module opens on types.
Effect syntax.
Modular explicits.
External types.
Locations for all parts of long identifiers.
Labeled tuples.

In a perfect world, each of these require a codec for serialising the feature into ASTs that do not support the feature, plenty of tests and new Ast_builder/Ast_pattern functions for using the feature. Nathan and I have been managing pretty well I would say, though we are not getting any technical debt work done.

Outreachy

February 9, 2026
Patrick Ferris

As the OCaml coordinator for Outreachy I have been running the latest December 2025 round. We are lucky to have four projects on the go!

Thibaut Mattio is mentoring two projects: an ML dashboard for Raven and an OxCaml backend for Raven.
Xvw is mentoring a Yocaml project.
I am mentoring a Geocaml project.

Eio

February 9, 2026
Patrick Ferris

During my work on tools like Shelter I have made a few improvements to Eio including:

OxCaml

February 9, 2026
Patrick Ferris

I have enjoyed my work on OxCaml went it has happend. I think a few tools have been useful including the try-oxcaml work. David and I also had fun at ICFP discussing some quick experiments related to io_uring and OxCaml.

One road-blocker here is the comparative amount of time spent trying to keep the OxCaml ecosystem working. Given my limited time working on fellowship projects, the rate of change of OxCaml (including the opam ecosystem) meant most of the time was spent making it just work, not exploring or experimenting with the features themselves. I think this has got better in recent months and perhaps jumping back in I would be surprised at the progress that has been made there.

Forester

February 9, 2026
Patrick Ferris

Tarides are funding some work on forester. This site uses forester by way of graft and I have spent some time working on these tools during the fellowship.

This includes a bytesrw-based bibtex library in OCaml that could be released soon.

Papers

February 9, 2026
Patrick Ferris

I had the great fortune of attending ICFP where I presented two talks and was co-author on a few others too.

Reference. Functional Networking for Millions of Docker Desktops (Experience Report) [madhavapeddy2025docker]

2025
Anil Madhavapeddy, David J. Scott, Patrick Ferris, Ryan T. Gibb, Thomas Gazagnaire
10.1145/3747525
https://dl.acm.org/doi/10.1145/3747525

Abstract

2025
Anil Madhavapeddy, David J. Scott, Patrick Ferris, Ryan T. Gibb, Thomas Gazagnaire

Docker is a developer tool used by millions of developers to build, share and run software stacks. The Docker Desktop clients for Mac and Windows have long used a novel combination of virtualisation and OCaml unikernels to seamlessly run Linux containers on these non-Linux hosts. We reflect on a decade of shipping this functional OCaml code into production across hundreds of millions of developer desktops, and discuss the lessons learnt from our experiences in integrating OCaml deeply into the container architecture that now drives much of the global cloud. We conclude by observing just how good a fit for systems programming that the unikernel approach has been, particularly when combined with the OCaml module and type system.

Reference. What we talk about when we talk about scientific programming [ferris2025scientific_programming]

2025
Patrick Ferris
https://watch.eeg.cl.cam.ac.uk/w/aYXqXLtgQawYMjVXjSQtjx

Abstract

2025
Patrick Ferris

Programming for the planet undoubtedly involves programming scientifically, but what kind of programming are we talking about and what makes it scientific? In what ways does it differ from other forms of programming, if at all? Is scientific programming, data science or machine learning fundamentally different to constructing a compiler or building a high-throughput web server? By considering how the scientific method (with its falsifiable hypotheses and repeatable and reproducible experiments) relates to scientific programming, I hope to explore how computer science and traditional programming techniques are coming up short in meeting the requirements of scientific programmers.

Reference. Generating a corpus of Hazel programs from ill-typed OCaml programs [ferris2025hazel_of_ocaml]

2025
Patrick Ferris, Anil Madhavapeddy
https://patrick.sirref.org/var/tyde2025.pdf

Abstract

2025
Patrick Ferris, Anil Madhavapeddy

When developing a new programming language, having a large corpus of both correct and incorrect programs allows language designers to test and explore the capabilities of their new language. However, bootstrapping such a corpus of incorrect programs is time-consuming and arduous. We therefore explore how to reuse code from more mature languages to generate a corpus of ill-typed code for newer ones. We have developed a compiler to Hazel, an emerging language with typed holes, from the more mature OCaml ecosystem. We find it practical to generate a comprehensive corpus of ill-typed programs for Hazel development, and discuss future larger scale efforts towards bridging ecosystems.

Reference. Decomposable Type Highlighting for Bidirectional Type and Cast Systems [carroll2025decomposable_type_highlighting]

2025
Max Carroll, Anil Madhavapeddy, Patrick Ferris
https://maxcarroll0.github.io/papers/workshops/HATRA-decomposable-type-highlighting/

Abstract

2025
Max Carroll, Anil Madhavapeddy, Patrick Ferris

We explore how to provide programmers with an interactive interface for explaining the process by which static types and dynamic casts are derived, with the goal of improving the debugging of static and dynamic type errors. To this end, we define mathematical foundations for a decomposable highlighting system within a bidirectional system, and show how these can be propagated through dynamic types in a cast system. Our prototype implementation in the gradually typed Hazel language includes a web-based user interface, through which we highlight the importance of type level debugging.

Reference. Yirgacheffe: A Declarative Approach to Geospatial Data [dales2025yirgacheffe]

2025
Michael Winston Dales, Alison Eyres, Patrick Ferris, Francesca A. Ridley, Simon Tarr, Anil Madhavapeddy
10.1145/3759536.3763806
https://dl.acm.org/doi/10.1145/3759536.3763806

Abstract

2025
Michael Winston Dales, Alison Eyres, Patrick Ferris, Francesca A. Ridley, Simon Tarr, Anil Madhavapeddy

We present Yirgacheffe, a declarative geospatial library that allows spatial algorithms to be implemented concisely, supports parallel execution, and avoids common errors by automatically handling data (large geospatial rasters) and resources (cores, memory, GPUs). Our primary user domain comprises ecologists, where a typical problem involves cleaning messy occurrence data, overlaying it over tiled rasters, combining layers, and deriving actionable insights from the results. We describe the successes of this approach towards driving key pipelines related to global biodiversity and describe the capability gaps that remain, hoping to motivate more research into geospatial domain-specific languages.

I had some great conversations with KC too with respect to my current research.

Future Work

February 9, 2026
Patrick Ferris

As I enter my final year of the PhD, my goal would be to try and align more of my fellowship work with my own research wherever it makes sense. This research focuses on the following hypothesis:

Embedding deep provenance tracking, reversible execution and mergeable histories directly into an interactive programming environment (a POSIX-like shell) greatly decreases the gap between exploratory scientific work and reproducible, publishable results; all whilst leaving existing workflows intact and being programming language agnostic.

I want to focus on building these tools using Irmin, Eio, OxCaml etc. There are other projects I wish to continue regardless, like Outreachy, though just as a coordinator (not a mentor).

Overlapping PhD Work

February 9, 2026
Patrick Ferris

There are multiple projects that overlap with my work. For example, Irmin, which I wrote a retrospective for. This is at the core of Shelter and suggests an extremely valid use case for a brancheable, mergeable database.

In addition to this, continuing to work on Eio makes sense to me too. Eio currently has no maintainer and no active development, but the issues are piling up. In particular, I am making heavy use of Eio in Merry, a POSIX(ish) shell written in OCaml. Aside from research angles on brancheable and mergeable shells, I am also quite interested in how this might look in terms of OxCaml. I met with Anil and Thomas recently to discuss the future of Eio and there was some agreement that we are all heavy users of it and hope to keep up its maintenance and set of features.

One large piece of work that is sorely needed here is moving Eio to bytes and not bigarray-backed Cstructs. Any libraries using bytesrw (e.g. jsont) incur an extra copy of data into and out of the bytes. Anil has been working on some of the pieces recently to make this better!

Maintaining and Releasing my own libraries

February 9, 2026
Patrick Ferris

I have published quite a few OCaml libraries in my time, below is just a few libraries (filtered to not include OCurrent projects).

$ opam list --all --no-switch --columns=name,authors: | grep -E "sirref|Ferris" | cut -d " " -f1
carbon
geojson
geojsone
graft
hilite
ISO3166
jekyll-format
ppx_deriving_ezjsonm
ppx_deriving_yaml
rtree
search
topojson
topojsone
cid
multibase
multicodec
multihash
multihash-digestif

Some of these are used in the community including hilite, ppx_deriving_yaml and the multi-codecs. I would like to do some general maintenance of some of these tools and release a few iterations of graft which now makes use of bib.

Ppxlib Maintenance

February 9, 2026
Patrick Ferris

By far, maintenance of ppxlib takes up a majority of my time. I think, given the other work I need to focus on, it would be great to try to minimise the amount of time working on ppxlib. In particular, perhaps only maintaining ppxlib as far as OCaml release-readiness is a possibility and important bug fixes (like the OOM bug reported by Jane Street).

Forester, Graft and Writing

February 9, 2026
Patrick Ferris

With all of this work, I am eager to communicate what I am up to and my thoughts on open source, OCaml, OxCaml etc. I am glad to be using Forester to do this, particularly as it is written in OCaml (and could make great use of my own work on Eio).

I hope to continue working on Graft and related tools like bib. But I would also like to start writing some long-form content similar to the Irmin retrospective.

Shelter imports [weekly-2025-w46]

November 15, 2025
Patrick Ferris

Most of this week was spent fixing bugs in Shelter in an attempt to get it ready to run the LIFE pipeline.

To do so, Shelter requires a mechanism by which to pull in files from the outside world. That could be configuration files, source trees, data etc. For now, this has taken the form of a crude @ import statement. In true Forester fashion, I have transcluded the foreign tree from my Shelter logs.

failed to render transclusion https://shelter.sirref.org/shelter-log-001/

Eio Debugging [eio-cap-debugging]

November 15, 2025
Patrick Ferris

One of the promises of Eio's capability model is an improved debugging experience. Your program must be designed in such a way as to make resources explicit. If you have a load_config function, more than likely it will need access to the file-system.

val load_config : _ Eio.Path.t -> Config.t
(** [load_config path] loads a configuration from [path].*)

Once all of your resources are explicit and threaded through your program, it becomes much easier to swap out one resource for another (provided they have the same interface).

This week I was trying to get the LIFE pipeline running in shelter. Part of that involves pulling the appropriate geospatial container image, but my code was terrifically broken, failing with what looked to me with a Docker error.

With only this clue, I knew that it was probably in the execution of some command in a child process, but which one? To trace this quickly, I could plug in a slightly different instance of the _ Eio.Process.mgr capability. Essentially unchanged from the one provided by the unix backend except that it would also print to stderr the command it was trying to run!

let debug_process_mgr (mgr : 'a Eio_unix.Process.mgr) : 'a Eio_unix.Process.mgr
    =
  let module D = struct
    type t = unit

    let spawn_unix () ~sw ?cwd ?pgid ?uid ?gid ~env ~fds ~executable args =
      Eio.traceln "Spawning subprocess... %a" Fmt.(list string) args;
      Eio_unix.Process.spawn_unix ~sw ?cwd ?pgid ?uid ?gid mgr ~env ~fds
        ~executable args
  end in
  let module V = Eio_unix.Process.Make_mgr (D) in
  Eio.Resource.T ((), Eio_unix.Process.Pi.mgr_unix (module V))

Whist there is a fair amount of jumping through first-class-module-shaped hoops here, all I am doing is rewrapping an existing "method" to a spawn a unix child process. Then using this helper function, I can push that into my standard environment.

let env =
  Eio_unix.Stdenv.with_env
  ~process_mgr:(debug_process_mgr env#process_mgr)
  env
in
Shelter.main config (env :> _ Shelter.env) dir cmd_file

And sure enough I was querying for the Docker user before pulling the image! I hadn't run into this, as for the last while I have been using alpine and debian images that already exists locally.

Taking stock & Graft updates [weekly-2025-w45]

November 10, 2025
Patrick Ferris

I ended up working on a real mix of things last week. I managed to get two large posts uploaded to this site: an ICFP recap and a deeper-dive into the random bits of OCaml hacking. This was pretty useful in terms of taking stock of where I was at, I always find post-conference, I'm a little lost. The conversations and ideas from ICFP are simultaneously exciting but also pretty inactionable.

Graft Updates [graft-updates]

November 10, 2025
Patrick Ferris

It would seem that there is a potential second user of Graft! A great source of motivation for some tidying and feature implementing. At the end of last week I fixed some little bugs in Graft and (more importantly) allowed users to name their subtrees. Upstream Graft also now pins and uses my Bibtex implementation.

OCaml Geotessera

November 10, 2025
Patrick Ferris

As I discussed in my OCaml roundup, I spent a bit of time polishing the OCaml geotessera code. In particular this meant starting to handle low-level geospatial transformations allowing users to project latitudes and longitudes into the rows and columns of their raster data. I feel like I'm stumbling onto a path Michael has carved out with Yirgacheffe. I'm following along with his work excitedly and we had a good catch-up last week discussing how libraries should handle things like automatic, numeric precision conversion.

As a fully paid-up member of the static types fan club, I'm often evangelising the benefits of types. However, I hit a little quirk of the Nx library in OCaml (a numpy equivalent). Take a look at this sum function:

let sum nx = Nx.sum nx |> Nx.item [];;
val sum : ('a, 'b) Nx.t -> 'a
let arr = Nx.create Nx.int8 [| 2 |] [| 64; 64 |];;
val arr : (int, Bigarray.int8_signed_elt) Nx.t
sum arr;;
- : int = -128

Even though sum takes an ('a, 'b) Nx.t array that stores elements with representation 'b and allows you to read/write to the elements with OCaml values of type 'a, the sum function performs the calculation in the world of the 'b representation type (here signed 8-bit integers). So we get an overflow! Perhaps not too surprising but easy to overlook! The moral of the story is that we still need to be careful in the OCaml world too.

Shelter

November 10, 2025
Patrick Ferris

I did a little cleaning up of the Shelter code too. I noticed a few bugs from the demos I did at ICFP. Most were pretty straight-forward to fix, and the test-suite is now checking parallel for-loops and conditional execution are working.

This week

November 10, 2025
Patrick Ferris

I hope to get a few things done this week:

Run some part of the LIFE pipeline through Shelter.
Deploy the Shelter daemon allowing other people to connect and take it for a spin.
Spend some time writing and reading, pulling ideas together rather than always writing code!
I have a slightly harebrained scheme that combines Shelter, atproto, Forester, capn(proto/web) and webcomponents... I want to see if I can pull that together.

OCaml Roundup: October 2025 [ocaml-roundup-october-2025]

November 5, 2025
Patrick Ferris

Welcome to a monthly roundup of OCaml-related open-source work I have been involved with. If you haven't already, a quick look at my ICFP roundup might be a good preface to what follows.

Outreachy December 2025

November 5, 2025
Patrick Ferris

The contribution period for this year's Outreachy round took place for most of the month of October. This year I am excited to be going back to a Geocaml project, working primarily to add writing capabilities to ocaml-tiff.

As per usual, I have been extremely pleased with the level of interest in the various OCaml projects being offered. Whilst we are still deliberating on our choices for interns this year, I won't say much more on the contributions, but if you want to get involved please do reach out! OCaml is a great language for geospatial work, striking a balance between reasonable performance for numerical code and also a rich type system for expressing complicated geospatial data-structures.

Ppxlib and the Future of Compiler Support

November 5, 2025
Patrick Ferris

Nathan has put a tremendous amount of effort into laying a roadmap this month for how ppxlib, going forward, will support future compilers. This is in response to the calamity that was bumping the internal AST to 5.2, and whilst I think we are adding a good deal more complexity, it will hopefully keep the ppx universe on an even keel for longer periods of time.

The first substantial PR for this modern world of ppxlib is in review!

Aside from that I also debugged an issue that lead to ppxlib OOM-ing machines... of course, any recursive fold-like loop where the accumulator keeps growing is likely to do that! It was nice to be able to crack out memtrace to be able to hunt it down:

Incremental Directory Reading for Eio

November 5, 2025
Patrick Ferris

A common tasks for many a CLI tool is to scan a directory of files, filter out the ones you are looking for and apply some transformation to said files. In fact, that is what Graft and Forester both do! A standard approach to this involves a recursive dance of readdir and stat, the former lists the entries in a directory whilst the latter lets you know what kind of file is at a particular path (e.g. symbolic link, directory etc.). This has a few downsides:

We're making two systems calls (pretty much one of the most expensive things your program can do).
If a directory contains thousands of files, we end up allocating a big list of strings after returning from readdir.

On Linux, we can get around most of that with getdents64(2) which allows us to incrementally read a directory. Whatsomore, this system call also returns the file type so there's no need to do an extra stat!

I prototyped an implementation of this called Eio.Path.walk which looks like:

val walk : _ t -> ((File.Stat.kind * string) Seq.t -> 'a) -> 'a
(** [walk t] traverses the directory [t] producing a sequence of results.*)

We effectively halve the number of system calls. In a benchmark that traverse deeply-nested directories with plenty of files, a full traversal whent from 1.46s to 110ms. I have since rejigged the implementation to automatically recurse for the user into subdirectories (not that the signature is unchanged) which does make it tricky to control the number of open file descriptors compared to if the user were in control.

OxCaml Experiments [icfp-oxcaml-uring]

November 5, 2025
Patrick Ferris

I have included here a transclusion from my ICFP post about an experiment using unboxed 32-bit integers in OCaml's Uring library.

OxCaml and Uring [oxcaml-and-uring]

Patrick Ferris

Jane Street were a big presence at ICFP 2025, carting along with them a shiny new OCaml compiler: OxCaml. If have been playing around with OxCaml recently but nothing outside toplevels in Javascript. Until now!

After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change:

  module Heap = struct
    type ptr = int32#
    (* ... *)
  end

  type cqe = {
    user_data_id : Heap.ptr;
    res : int32#;
  }

The idea being that a completion queue entry (a notification that some operation has completed) could be fully represented using 64 bits (two 32-bit, unboxed values). You can see how this impacted the library! I'm not certain about this change (and I'm sure I did it wrong) but it was nice to realise OxCaml gives you this kind of control. However, I am worried about the ergonomics of manipulating values like int32# and the temptation to case it into an int (presumably losing a good portion of the value of having an unboxed value in the first place).

x-ocaml with OxCaml

November 5, 2025
Patrick Ferris

I sent a few PRs to the OxCaml repository to get the toplevel working again and lucky the underlying culprit for why patches were disappearing was found. Watch this space for a new toplevel in the browser soon.

Geospatial OCaml

November 5, 2025
Patrick Ferris

In addition to the Outreachy work I mentioned above, I also found the time to keep the Geocaml project ticking along. A few projects saw some much-needed love including:

PROJ in OCaml

November 5, 2025
Patrick Ferris

A modernisation of the bindings to PROJ. PROJ is probably the most used and tested code for working with geospatial projections. Any time you get a sateillite image or a blob of GeoJSON, the geospatial data will normally have some coordinate reference system (CRS) with it detailing exactly what the numbers that tell you where something is mean. One of the most common being WGS84 (used by GPS and GeoJSON for example). Each CRS has its benefits and drawbacks, and often you will need to convert between them. That is what PROJ does.

WKT in OCaml

November 5, 2025
Patrick Ferris

Well-known Text (WKT) is a very simple encoding of geospatial objects and CRSs. For example, a polygon might look like:

Polygon ((10 10, 10 20, 20 20, 20 15, 10 10))

I used bytesrw to build a simple codec for the WKT format. It is probably about time to write the basic geospatial object library too, something similar to Rust's geo library.

Geotessera

November 5, 2025
Patrick Ferris

A lot of people in my group are excited about Tessera. They have made it easy to used the Tessera embeddings via a python library called geotessera. In true OCaml-at-heart spirit, I tried my hand at porting this library. I had a good deal of success, converting the data into Nx array, extracting the CRS and transform out of the GeoTIFF landmasks using ocaml-tiff, reading datasets using ocaml-geojson etc. This made me excited but also keen to improve the APIs of most of these libraries too.

I feel geocaml is at a critical mass where we can start to have this useful feedback loop for the libraries because there is enough functionality to actually use them!

ICFP 2025 [icfp-2025]

October 24, 2025
Patrick Ferris

Two weeks ago I was fortunate enough to attend the International Conference on Functional Programming in Singapore. My first time in Asia and my second time at the conference, what follows are some thoughts and presentations I enjoyed whilst I was there.

I must thank my office mate (and friend!) Ryan Gibb for his camaraderie and sage advice: "the best track is hallway track". Not to mention all of his photos too. Thanks Ryan!

From right to left, top to bottom: Anil Madhavapeddy, Sadiq Jaffer, Patrick Ferris, Ryan Gibb and Roy Ang.

Random Hacking

October 24, 2025
Patrick Ferris

Below are just some of the pieces of random hacking I got up to whilst in Singapore.

OxCaml and Uring [oxcaml-and-uring]

Patrick Ferris

After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change:

  module Heap = struct
    type ptr = int32#
    (* ... *)
  end

  type cqe = {
    user_data_id : Heap.ptr;
    res : int32#;
  }

Bibtex Parser

October 24, 2025
Patrick Ferris

On the flight over to Singapore, I started to write a post for this website. Some readers may remember that I use forester after a preprocessing step via graft to build this site. Part of that preprocessing consumes bibtex files and outputs forester trees. For the initial graft release I used the excellent Talaria Bibtex library. Unfortunately, it presented a pretty confusing API that, whilst nicely typed, felt a little cumbersome to use. Additionally, it was too specific in what it would accept as valid Bibtex. I wanted something that was a little more straight-forward to use (if a little fast-and-loose when it comes to types...). And so I built bib.

This was my first real attempt at using the Bytesrw library. I was happy with the results, although I did have to copy a substantial amount of code from other bytesrw-compatible libraries. I'd be happy to shift to this bytes-first world if it wasn't for the ever-present-in-my-mind issue of relocatable values in the OCaml heap!

Geocaml Libraries

October 24, 2025
Patrick Ferris

There has been some excellent outreachy work happening on the geocaml/ocaml-tiff library which is a potential Outreachy internship project for December 2025. More on this in a future post.

On the way back (and a little during the conference), I spent some time building an OCaml counterpart to geotessera, a library for working with Tessera. The library is not ready for people to use directly, but I was working on various constituent parts including:

Modern OCaml bindings to PROJ (a coordinate reference system projection library)
Further work on a WKT library for OCaml
Coming soon... ocaml-geotessera

Talks

October 24, 2025
Patrick Ferris

Hazel of OCaml at TyDe

October 24, 2025
Patrick Ferris

The type-driven development (TyDe) workshop was held on the first day of ICFP at the NUS School of Computing. I find the workshop name to be a little misleading, but perhaps this comes from my engineering background. The talks focused a lot on type theory, presenting the formal mathematics of interesting type systems with less focus on why it might be useful to have such an exotic type system... but then again, that was the nature of the entire conference.

I presented a talk on hazel_of_ocaml. The slides are here, along with the extended abstract. My talk ended up focusing a lot on the pedagogical benefits of having a type system that helped explain the most important thing a type system does... catch type errors! This work represents a small piece of work inside Max Carroll's excellent part II project, which he presented at the HATRA workshop: Decomposable Type Highlighting for Bidirectional Type and Cast Systems.

Scientific Programming at PROPL

October 24, 2025
Patrick Ferris

I think it is important to be honest whenever you get a talk accepted that you otherwise feel should not have been. My talk "what we talk about when we talk about scientific programming" is one such talk. The slides are here. The talk itself was nothing to write home about, but I think there were a few nice ideas that came out of it and the conversations I had with fellow PROPL attenders.

TMF Data Discrepancies Redux

October 24, 2025
Patrick Ferris

On the eve of my talk, I decided to re-run some analysis I did a few years ago on the tropical moist forest dataset, knowing that since then a few more years had become available. After fighting with some bit-rot in the Google Earth Engine script I had used, the new results were not very uplifting.

Land Use Class	Percent Change
Undisturbed	-2.9
Degraded	-5.5
Deforested	8.6
Regrowth	0.5
Water	0.0
Other	-0.6

The data in the table corresponds to the land use class values in Indonesia in the year 2008. The analysis computed the difference across each of the classes between the dataset released in 2021 and the dataset released in 2024. Most notably, the amount of deforested land has gone up by 8.6 percent!

As a somewhat tangential, though related, aside, I had the pleasure of talking to Sam Westrick, Seong-Heon Jung and Sundara Vishnu Satish of the ParCour research group. They were telling me about MaPLe, a programming language from the MLs built for parallelism. We discussed some ideas of using real-world, geospatial algorithms as benchmarks for MaPLe. Michael, if you got this far remind me to chat to you about this!

Shelter Reactions

October 24, 2025
Patrick Ferris

At the end of my talk, I briefly mentioned shelter. By far, this was the topic I got the most engagement on post-talk. During the conference, this was also true when people asked what I was working on. The tag line "a shell with undo" seemed to capture the attention of most people.

I'm feeling a renewed sense of interest in my own work thanks to that, even if the "research" contributions are not very apparent. However, I also had a good discussion with KC about the underlying MRDT in Shelter and how we might think about collaborating in the future on MRDT-related work.

Related to that, I managed to spend the evening in Chinatown with Roly Perera. Roly's work has focused on Fluid, a transparency-based programming language that allows authors to reveal their computation to their user. Excitingly, Roly presented a new text-based part of the language at PROPL. Our work is in the same universe, loosely orbiting some notion of reproducibility and explainability, but quite different in engineering work and how users might actually interact with our systems. I thoroughly enjoyed talking about solutions or research ideas that preserve existing workflows. Roly was great at pulling that out of the current work I have been doing.

Other Talks and Highlights

October 24, 2025
Patrick Ferris

I was a part of, often in a very small way, some other talks that were presented at ICFP including

Functional Networking for Docker, I encourage you to read Anil's notes on the topic. It is also worth mentioning Anil's post-posix talk (not unrelated to my Uring musings above).
Michael gave an excellent, very live, talk about Yirgacheffe at PROPL which I thoroughly enjoyed. I'm also excited about his recent developments which I'm sure you can clue into over on his blog.

Below is a non-exhaustive list of talks I enjoyed whilst at the conference. Some relevant, most of them not!

By far, one of the best talks was Polynomial-time Program Equivalence for Machine Knitting. I think the talk really nailed the "here's the idea, the details are in the paper"-presenting style as they described an "...algorithm that canonicalizes the algebraic representations of the topological semantics of machine knitting programs".
I attended most of the OxCaml tutorial presented by the incredibly friendly Gavin Gray.
Ryan's talk Spatial Programming for Environmental Monitoring at PROPL was excellent.

There's plenty more to talk about, and I'm sure my future posts will reference other parts of my experience at ICFP 2025. Thanks for reading, and if anybody is interested in any aspects of this do reach out!

Quarterly OCaml Q2 [ocaml-quarterly-q2]

July 18, 2025
Patrick Ferris

Thanks to Tarides sponsorship, I get to work on open-source OCaml. This quarterly is a companion to my weeklies, summarising the last three months of development, peppered with ideas and thoughts about OCaml, its community and its future.

What I wanted to work on?

July 18, 2025
Patrick Ferris

There were two main things I hoped to continue working on: ppxlib and outreachy. These are projects that I was previously working on, and in the case of Outreachy I have now been involved for many years.

In addition to this, all of my Part II projects this year used OCaml in some regard. In general, I want to see more adoption of OCaml. Over the years this has taken many forms including my initial work on developing workflows for OCaml that just turned five years old. This directly fed into the rebranding and rethinking of ocaml.org itself.

What I worked on?

July 18, 2025
Patrick Ferris

Ppxlib

July 18, 2025
Patrick Ferris

Ppxlib is the de facto standard library for building OCaml preprocessors. At the time of writing, opam list --depends-on=ppxlib informs me that there are 267 reverse dependencies. Janestreet is a heavy user of ppxes and has authored many.

One of the main accomplishments this quarter was bumping the internal AST to 5.2. This allows ppx authors to use new OCaml language features in their ppxes. In bumping the AST, we knowingly broke compatability for pretty much every single reverse dependency. As best we can, we have been sending patches to ppx libraries and helping users migrate to the latest ppxlib.

There is an interesting dicussion to had about the nature of open-source, and its interaction with industrial monorepos and community tended package repositories. Package ecosystems thrive whenever there is a dedicated community creating, maintaining and publishing packages. The idea is that the published world should be healthy. The publishing medium can act as natural limiting factor in the churn of breaking changes (in the case of OCaml this is via PRs to the opam-repository). This, I have come to notice, reacts poorly to changes coming from internally consistent monorepos where introducing breaking changes is easily fixed by applying patches there and then. Whatsmore, OCaml is often stated as an incredibly safe language to perform large refactorings thanks to its type system.

Ppxlib sits awkwardly in the space of possible breaking changes. Tied to OCaml's parsetree, impacts of changes there ripple down to ppxlib. The compiler itself can remain internally consistent, and is protected as it need only parse source code. Ppxlib, on the other hand, exposes the parsetree to users and thus any changes to the parsetree will likely be felt by ppx authors. Since I started working on ppxlib, it feels as though the number of syntax changes has gone up (primarily from Janestreet work). Unless we make changes to how we provide support for these, maintainers of ppxlib will forever be stuck doing busy work! Thankfully, Nathan has thoughts on how to improve this.

There are a slew of other features I have added to ppxlib including:

Support for deriving from classes.
Support for deriving from module bindings and signatures.
Fixing compiler and ppxlib dummy locations.
Bumping to 5.3.
Migrations for 5.4.

Outreachy

July 18, 2025
Patrick Ferris

Our two projects this year, one on claudius and one on dune, are going extremely well. At the time of writing, we just had a mid-internship call to catch up, and I was blown away by the progress each intern has made. Unfortunately, Outreachy is struggling with funding and the next round is perhaps not going to happen. This is a real shame and I am hoping that it will not be the case.

Outreachy has been a wonderful source of new, committed OCaml developers. If you haven't already, do peruse the webpage on OCaml.org to see past internships or watch the demo day presentations. For the mentors involved, I believe it has also been a rewarding experience (though at times a challenging one). We are always looking for new mentors and project ideas, please do reach out to me if you are interested.

Hazel

July 18, 2025
Patrick Ferris

OCaml's feature set allows it to shine at writing programming languages (and things of that ilk: compilers, interpretters, static analysis tools). Hazel is a research programming language with typed holes written completely in OCaml (via the reason dialect).

Relating this back to the original intent of my work, to improve OCaml adoption, I believe this also means keeping existing users happy. In the last quarter I developed a compiler from OCaml to Hazel. Whilst new features are still being added to Hazel, I hope this could serve as a tool to help develop test-suites and standard library functions using existing OCaml solutions. In a student's work (Typed Debugging for Hazel), we used this tool to build a corpus of ill-typed Hazel programs to great effect.

OCaml should continue to be a world-class programming language for building other programming languages. I hope to upstream some of this work to Hazel and provide some low effort maintenance to help keep their compiler in good shape.

Systems Programming in OCaml

July 18, 2025
Patrick Ferris

In a cross-over with my own research, I have been developing many tools related to systems programming in OCaml including:

An eBPF-based open syscall tracing tool.
A library in OCaml for spawning void processes.
A shell session manager that uses Irmin to manage sessions. It is nice to see a new push to finally land the direct-style Irmin PR!

This work is means to develop the underlying libraries that support it. For example, I have opened a few PRs to Eio to add new "fork actions" to the spawn API. I also investigated the feasibility of changing the underlying mechanisms in Eio to use Picos. In the future, I think this could be important avoid further splitting the OCaml ecosystem.

OxCaml

July 18, 2025
Patrick Ferris

I dabbled a little with OxCaml and build try-oxcaml to let people take it for a spin without having to perform opam repository gymnastics. It turned into a lot of work to track down some pretty inane bugs (type definitions differeing between js_of_ocaml and the OxCaml compiler, resulting in different Javascript runtime representations...).

This unblocked a few of my colleagues to get OxCaml working on tools like odoc_notebooks and x-ocaml.

Forester

July 18, 2025
Patrick Ferris

A good proportion of my work this quarter has been focused on how to present the very work that I am doing. Jon Sterling has been developing a tool for scientific thought called Forester which seemed like a possible candidate for writing and sharing my work.

Porting my existing blog posts and website content from markdown to Forester's LaTeX-inspired syntax didn't seem like an option. In particular, many of my posts made use of additional markdown-based tools (like ocaml-mdx).

This lead to the development of Graft: a preprocessor for Forester forests, converting markdown and bibtex to trees.

What's next?

July 18, 2025
Patrick Ferris

So, I worked on most of what I wanted to work on and then some! Going forward I hope to keep maintaining ppxlib in some capacity and coordinating the OCaml community's Outreachy efforts.

I hope to continue my small experiments (e.g. converting Eio to Picos). My own research makes heavy use of Irmin, and I would be interested to help with that too. More recently, a new library has been released for working with numerical data in OCaml called Raven: I am interested to use this library in Geocaml, a suite of geospatial tools written in OCaml that I maintain.

I feel conflicted about the OxCaml efforts. I admit, I am not fully aware of the full benefits of the features, but I worry about some proliferation of modes that make the type system in OCaml unbearable to use. On top of this, with my ppxlib hat on, I worry about the impact of changing the compiler so frequently, placing strain on an already small community. That being said, by releasing OxCaml separately I do believe running it as an experimental set of packages will help understand the tool better. But I would not be building anything I intend to maintain or research with right now as that ecosystem is far too volatile.

Finally, over the past two years I have done a lot of teaching. From Part II projects to supervisions. Going into next (academic) year, I intend to reduce my in-person teaching to focus on my research. However, I am interested in producing more materials for learning, maybe some of this in OCaml, perhaps something similar to Beautiful Racket. I have tried in the past to do these sorts of things, for example The Irmin Book.

Thank you

July 18, 2025
Patrick Ferris

Thank you for reading this wrap up! And thank you again to Tarides for letting me work so freely on things that I think are good for the OCaml community.

Forester as a Target Syntax [weekly-2025-06-02]

June 2, 2025
Patrick Ferris

This week included some time finishing opentrace and subsequently folding it into shelter. I have been writing up some more of the draft paper for shelter which I am excited to share in the near future.

I revisited the upgrading vpnkit PR and pushed some more fixes. I have been thinking, again, about the promise of a direct-style world for OCaml that just hasn't quite landed yet. C'est la vie.

Forester and Graft

June 2, 2025
Patrick Ferris

I spent a bit of time finally pulling out my changes to Forester to add markdown and bibtex support into a standalone tool: graft.

Graft [graft]

June 2, 2025
Patrick Ferris
https://github.com/patricoferris/graft

Graft is a Forester preprocessor.

It takes a forester (a directory of trees) written in a mixture of Markdown, Bibtex and Forester syntax and produces a new forest completely written in Forester syntax.

Usage

June 2, 2025
Patrick Ferris

graft simply preprocesses a forest generating Forester trees from .md, .bib and .tree files. It will copy the structure of the input directory in the output directory.

$ graft preprocess --output=grafted-trees trees
$ forester build

This assumes that you have updated your Forester toml file to put the grafted-trees directory as your source of trees.

[forest]
trees = [ "grafted-trees" ]

Example

June 2, 2025
Patrick Ferris

A typical "tree" might look something like

---
title: Opentrace and Supervisions
date: 2025-05-26
author: Patrick Ferris
---

Over the past two weeks I have mainly split my time (amongst many things) developing [opentrace](open-trace)
and doing revision supervisions.

```forester
\put\transclude/numbered{false}
\transclude{open-trace}
```

A few things to note:

The yaml frontmatter allows you to add some of the metadata fields from Forester.
At any point in your markdown there is an escape hatch to Forester using a forester codeblock.

It is very satisfying to find the separation of concerns works quite well. For a while I had been rebasing my development branch on Forester. I was also worried about trying to get the code upstream as it pulled in many dependencies. It seems that I have a very workable solution. I welcome contributions to graft including extra input formats. I am also considering extending the Forester configuration to contain some graft configuration for how it should generate new trees (e.g. at the moment every entry in a bibtex file is given a new tree).

Maths Support

June 2, 2025
Patrick Ferris

As part of that process, graft now supports Markdown KaTeX. For example:

Definition. Mergeable Replicated Data Type Implementation [mrdts]

Patrick Ferris

A mergeable replicated data type (MRDT) implementation for a data type $\tau$ is a tuple $D_{\tau } = (\Sigma , \sigma _{0}, do, merge)$ where:

$\Sigma$ is the set of all possible states at a branch,
$\sigma _{0} \in \Sigma$ is the initial state,
$do : Op_{\tau } \times \Sigma \times Timestamp \rightarrow \Sigma \times Val_{\tau }$ implements every data type operation,
$merge : \Sigma \times \Sigma \times \Sigma \rightarrow \Sigma$ implements the three-way merge strategy.

Definition 2.1 from "Certified Mergeable Replicated Data Types".

Revision Supervisions

June 2, 2025
Patrick Ferris

I have been doing revision supervisions for Discrete Maths and Foundations of Computer Science. I have also been marking Operating Systems past paper questions for the same group of first years.

Graft [graft]

June 2, 2025
Patrick Ferris
https://github.com/patricoferris/graft

Graft is a Forester preprocessor.

It takes a forester (a directory of trees) written in a mixture of Markdown, Bibtex and Forester syntax and produces a new forest completely written in Forester syntax.

Usage

June 2, 2025
Patrick Ferris

graft simply preprocesses a forest generating Forester trees from .md, .bib and .tree files. It will copy the structure of the input directory in the output directory.

$ graft preprocess --output=grafted-trees trees
$ forester build

This assumes that you have updated your Forester toml file to put the grafted-trees directory as your source of trees.

[forest]
trees = [ "grafted-trees" ]

Example

June 2, 2025
Patrick Ferris

A typical "tree" might look something like

---
title: Opentrace and Supervisions
date: 2025-05-26
author: Patrick Ferris
---

Over the past two weeks I have mainly split my time (amongst many things) developing [opentrace](open-trace)
and doing revision supervisions.

```forester
\put\transclude/numbered{false}
\transclude{open-trace}
```

A few things to note:

The yaml frontmatter allows you to add some of the metadata fields from Forester.
At any point in your markdown there is an escape hatch to Forester using a forester codeblock.

Opentrace [open-trace]

May 19, 2025
Patrick Ferris
https://tangled.sh/@patrick.sirref.org/opentrace

Thanks to Koonwen's excellent libbpf bindings in OCaml, I have been building a little tool called opentrace to make it easier to track an executable's inputs and outputs.

This work was inspired my Michael's self-proclaimed "gross hack": pyshark. Whilst pyshark achieves its goals by injecting code into commonly used python objects and methods, opentrace uses eBPF. By using a lower-level API (hooks in the kernel), opentrace can remain programming language agnostic. However, less information is none about the user's intent compared to something like pyshark.

Monitoring the System

May 19, 2025
Patrick Ferris

opentrace has an all command that will trace the entire system.

$ sudo opentrace all --flags=O_WRONLY
pid,cgid,comm,kind,flags,mode,filename,return
16324,4417,".nheko-wrapped",openat,577,438,"/home/patrick/.cache/nheko/nheko/curl_alt_svc_cache.txt",47
16324,4417,".nheko-wrapped",openat,193,33188,"/home/patrick/.cache/nheko/nheko/cr3ZUHqBErIOe3PlwJ1SuU8zFKKxL12VzrRoHYMH.tmp",47
16324,4417,".nheko-wrapped",openat,577,438,"/home/patrick/.cache/nheko/nheko/curl_alt_svc_cache.txt",47
16324,4417,".nheko-wrapped",openat,193,33188,"/home/patrick/.cache/nheko/nheko/QfZTdZSuC56NdcDQ3aMxJc3BhMhAj8PmtYW1zFDP.tmp",47
2530,4235,"systemd",openat,524865,438,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/cgroup.subtree_control",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.min",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.low",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.high",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.max",41

The --flags=O_WRONLY argument filters the events where the O_WRONLY flag was set in the call to open.

We also get the name of the current executable linked to the task (comm). The -wrapped is an artefact of using Nix.

Tracing an Executable

May 19, 2025
Patrick Ferris

The primary use case for this tool is to inspect what files your program might be reading and writing.

$ sudo opentrace exec --format=json --flags=O_CREAT -- opam list
$ cat trace.json | jq ".[] | .fname"
"/home/patrick/.opam/cshell/.opam-switch/packages/cache"
"/home/patrick/.opam/log/log-118747-29da3d.out"
"/home/patrick/.opam/log/log-118747-29da3d.err"
"/home/patrick/.opam/log/log-118747-29da3d.env"
"/home/patrick/.opam/log/log-118747-29da3d.info"

The "flags" argument can specify a small boolean formula for checking the open flags of a particular event with | (or), & (and), and ~ (not). Parentheses can be used for precedence.

$ sudo opentrace exec --flags="O_WRONLY|O_RDONLY" -- ocaml --version

Spawning Subprocesses

May 19, 2025
Patrick Ferris

One feature opentrace needs (in this proof-of-concept phase) is the ability to also trace subprocesses.

opentrace is primarily an eBPF program that is loaded into the kernel and communicates with an OCaml program. Events are communicated via a ring buffer and most of the post-processing happens in OCaml. To capture subprocesses, opentrace creates a new control group (cgroup) and places the new process into that group. This gives opentrace a new identifier to track, namely the cgroup.

So consider the following program.

let () =
  Eio_posix.run @@ fun env ->
  Eio.Path.(save ~create:(`Or_truncate 0o664) (env#fs / "hello.txt") "hello");
  Eio.Process.run env#process_mgr
    [ "/bin/bash"; "-c"; "echo 'heya' > heya.txt" ]

It first creates a file using direct calls to functions like openat. Then it spawns a process which creates a new file called heya.txt. This happens in a separate process. However, with the --cgroups flag we can capture both interactions with the operating system.

$ sudo opentrace exec --cgroups --flags="O_CREAT|O_TRUNC" ./main.exe
pid,cgid,comm,kind,flags,mode,filename,return
153187,530807,"main.exe",openat,526914,436,"hello.txt",5
153192,530807,"bash",openat,577,438,"heya.txt",3

Eio's Process API

May 19, 2025
Patrick Ferris

I have used the Eio_unix fork action process API to be able to extend what happens in the child process. Loading most eBPF programs into the kernel requires special privileges hence the need for sudo. When a user requests for a particular program to be executed and traced, opentrace spawns a process via the Eio Process API. Opentrace defines a few new so-called "fork actions", little fragments of C code that are run after the call to fork (clone). Most likely this ends with a call to execve, but other calls are possible for example setuid allowing opentrace to change the user of the child process so it does not run as root. Similarly, this is where (if used) we create the cgroup and place the process into that group.

Limitations: Io_uring

May 19, 2025
Patrick Ferris

Whilst testing opentrace against some of the tools I use nearly daily, I noticed some events were being missed. I tried tracing forester, and only the initial read of forest.toml was logged. It dawned on me that the reason for this was that forester (via eio) was using io_uring to perform most of the IO. Most attempts to open files were bypassing the open system calls, and instead they were being performed by the kernel after reading a submission request for an openat2-style call!

This is not news to seasoned, Linux systems programmers. Io_uring bypasses SECCOMP filters for exactly the same reasons.

$ sudo opentrace exec -- forester build
$ cat trace.csv
pid,cgid,comm,kind,flags,mode,filename,return
155007,535570,"forester",openat,524288,0,"forest.toml",5
155007,535570,"forester",Uring,2621440,0,"",0
155021,535570,"cp",openat,131072,0,"/home/patrick/documents/forest/theme/favicon-32x32.png",4
155007,535570,"forester",Uring,2686976,0,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0

It is interesting to note two things here:

We can tell that forester reads the configuration file probably using something like In_channel in OCaml (it does).
It appears that Uring is performing IO in both worker threads and directly.

The file paths are empty at the moment as I cannot find a clean way to trace openat submission requests into Uring without it sometimes going very wrong. I have tried quite a few methods (e.g. tracing do_filp_open) and at the moment I am tracing io_openat2, but this seems quite brittle, and often the filename is completely garbled, so I do not set it. If anyone has any ideas to trace Io_uring more reliably, I am all ears!

Shelter in the Forest [weekly-2025-04-21]

April 21, 2025
Patrick Ferris

I spent much of this week working on Shelter and things related to it. Some of that time was also spent on Forester.

Forester

April 21, 2025
Patrick Ferris

I added two important quality-of-life features to my frontend to Forester this week.

Bibtex Support

April 21, 2025
Patrick Ferris

I had previously mentioned adding support to Forester for Markdown. This week I added support for Bibtex too. From any *.bib file in your forest, Forester will now dutifully recognise it as a Bibtex file and convert, as best it can, all the entries into Reference trees.

I'm becoming quite convinced of this model at the moment. I'm using Forester's Code.t as a target representation. In fact, to ease the process, I really shouldn't spend all my time on my website, I have reused my Yaml.t -> Code.t and Markdown.t -> Code.t functions in the Bibtex parser.

To see it in action, you could have a look at the Build Systems à la Carte paper which is generated completely from Bibtex.

Full Heading Support

April 21, 2025
Patrick Ferris

The eagled-eyed viewer may have noticed that the table of contents for this page has more than one level. I finally caved and spent an evening rejigging my Cmarkit.Doc.t -> Tree code which was hacky and broken and is now less hacky and less broken.

In addition, headings support links and emphasis etc.

Lunch with Jon Sterling

April 21, 2025
Patrick Ferris

I had a delightful lunch with Jon Sterling discussing the future of Forester, the nature of the Web (old and new) and the success posting weekly updates for our colleagues. Thanks Jon.

Shelter Fixes

April 21, 2025
Patrick Ferris

I spent a good chunk of my week fixing bugs in Shelter with the aim to perhaps setup a VM somewhere and let people kick the tyres of what we've got so far.

The first bug is pretty annoying. At the moment, our filsystem backend is ZFS and we make heavy use of snapshots and cloning in order to provide time-travelling capabilities. Unfortunately, ZFS will take a snapshot before data has fully made it to disk (or whatever is the equivalent point it should reach in ZFS). Commands that generated lots of disk activity would be snapshotted in a half finished state and this would cause all sorts of problems. Thanks to Mark Elvers for the pointer to how OBuilder uses ZFS for the OCaml macOS builders which unmount datasets immediately therefore inducing a flush of sorts. Shelter now follows a similar model with all of the slow downs that create. Anil and I discussed some amalgamation of overlayfs, tmpfs and ZFS to alleviate some of this but for now that's a premature optimisation.

A small eDSL for Shelter

April 21, 2025
Patrick Ferris

Whilst testing Shelter, I ended up wanting a way to programmatically invoke the different run commands. This is similar to say a Dockerfile, but maybe with a little more expressivity.

This lead me to revisit the Build systems à la Carte paper and rediscover selective applicative functors.

I started playing around with a selective applicative interface to Shelter, this would allow you to express your dependencies statically but select them dynamically (as the paper says).

module D = Shl (Identity)

let shelterfile =
  let open D in
  let base_image = from "alpine" in
  let is_node_lst img = String.equal "v22.15.0" (stdout img) in
  let cmds base =
    let node_version = run "node --version" base in
    Select.if'
      (Select.map ~f:is_node_lst node_version)
      (run "node -e 'console.log('success!')")
      (run "node -e 'console.log('failure!')")
      base
  in
  with_session "node" (cmds base_image)

From this, we get a slightly more expressive way to describe images.

Geocaml TIFF Library

April 21, 2025
Patrick Ferris

I was pleasantly surprised to receive a pull request from Michael adding support to ocaml-tiff for reading TIFF files compressed using LZW. I was also surprised to hear the TIFF LZW is a little different to others.

In trying to get this PR merged, I moved the initialisation of the Eio eventloop to outside each individual test case. This one change then completely broke the entire test suite. After a period of debugging and help from Thomas Leonard the root cause was OCaml's OUnit2 library using process-level parallelism (via Unix.fork), sharing the ring between the parent and the child lead to the issues.

Part II Students

April 21, 2025
Patrick Ferris

As the new term begins, it signals that there are only just over two weeks for the final year undegrads at Cambridge to submit their dissertations.

The four students that I help supervise have been sending me drafts of their work (and reminder you can read about their projects) and I'm very impressed. I'm sure the next two weeks will be stressful, but I'm proud of what they have accomplished over the past academic year.

Shelter, Hazel and More! [weekly-2025-03-31]

March 31, 2025
Patrick Ferris

Last week I focused on Shelter -- our idea that shells should have the same ability as reproducible build tools like Nix or Docker. To this end I now have a fairly fleshed out prototype.

Shelter Prototype

March 31, 2025
Patrick Ferris

Shelter is a spin-off from the work Michael and I started with Shark. It takes the same ideas but applies them directly to a shell-like interface.

We're still in the middle of working all of this, but you can read more about it at Shelter.

Forester Hacking

March 31, 2025
Patrick Ferris

As you can probably tell, my website is still using Forester. I rebased my Markdown branch to include the new Atom syndication feature.

Alongside that I added support for arbitrary HTML injection into Forester via codeblocks in Markdown. This was actually very straightforward thanks to Markup and being able to re-parse Forester syntax in the middle of converting a Markdown document. The HTML for the shell in Shelter uses this feature.

If you are interested in taking this custom Forester for a spin, there's a branch on Github. In fact, nearly the only change beyond letting the core engine know about markdown files, is adding a new parser frontend.

Hazel

March 31, 2025
Patrick Ferris

For one of my Part II students, I've been prototyping a transpiler from OCaml to Hazel. This has gone pretty well and now supports type annotations as well as straight-forward implementation translation.

Consider the following OCaml map function.

let rec map f = function
  | [] -> []
  | x :: xs -> (f x) :: map f xs

The tool, hazel_of_ocaml can translate this to Hazel code, including making the polymorphism explicit.

let map : forall a -> forall b -> (a -> b) -> [a] -> [b] 
  = typfun a -> typfun b -> fun f -> fun x1 -> case x1
  | [] => []
  | x :: xs => f(x) :: map(f)(xs)
end in ?

You can copy and paste that codeblock into the hazel playground. But do note that it still needs some manual editing to add the type applications in directly (map@<a>@<b>). With the right amount of type-inferencing and scoping I actually think that you could place those type applications in yourself. This could make a nice Part II project I think.

Ppxlib

March 31, 2025
Patrick Ferris

I recently wrote about the painful experience of miragrating ppxlib to the 5.2 OCaml AST. This week, Nathan Rebours and I merged a PR to add the 5.3 AST! Ppxlib has been playing catch-up with the compiler and we decided it was best to try to catch up quickly and deal with the ecosystem fallout all at once rather than incrementally. With this new AST merged, ppx authors can now use the new Pexp_effect parsetree node. I'll write a little more about this in a separate post soon.

Vpnkit, Void Processes, LSP Servers [weekly-2025-02-17]

February 17, 2025

Previous weeklies used strong emphasis to distinguish sections. This comes from Forester's philosophy about atomicity of the content in your forest.

However, subtrees are supported! I quickly hacked together the ability to use subheadings to indicate subtrees. This is strictly less expressive than the \subtree{} of Forester's default syntax as we cannot close heading sections in Markdown.

This weekly uses subtrees.

Vpnkit

February 17, 2025

I spent some time this week trying to upgrade vpnkit to OCaml 5. I was originally working on a paper idea which might need benchmarks, but Anil and I decided we could simply point to the port I did and show how it has simplified much of the code.

Void Processes [void-processes]

February 17, 2025

Work continued on implementing (and fully exploring) void processes. A lot of the groundwork already exists in Jake Hillion's master's thesis.

This week I added a feature that I needed to help build the processes we need for the shell I'm building: mount points with modes!

In addition to the root mount (taken care of with pivot_root), we need to be able to add additional mounts into the process' environment. These can now be added. All mount points can be mounted readonly or readwrite.

Here is the "Hello, World!" example (the /say/hey program has been statically compiled using musl-gcc).

let status =
  let void =
    empty 
    |> mount ~mode:R ~src:hey_dir ~tgt:"say"
    |> exec [ "/say/hey" ]
  in
  let t = Void.spawn ~sw void in
  Promise.await (Void.exit_status t)

There really is nothing else in there. Without specifying a root mount, the void process is started with an empty tmpfs root. Next on the list is networking!

LSP Servers

February 17, 2025

I got a little side-tracked building a library for writing LSP servers in OCaml: mlsp. This may seem a little unrelated, but it isn't. The LSP has become the de facto standard for communicating between an editor and a programming language environment. If you have used VSCode to write a program in Python, chances are you are using the official extension which gives you linting, formatting, code navigation etc. All of these features are communicating using the LSP.

It seems Code Mirror can already proxy over a websocket for LSP support too (we might not even need that as we can compile OCaml directly to JavaScript/Webassembly and have the whole thing running locally!).

Open-Source & Community

February 17, 2025

Anil and I had a great conversation this week about building community especially as it pertains to open-source and OCaml.

I've been going back over Governing the Commons, but have already discovered The Real World of Technology!

More thoughts on all of this soon... maybe

Forester, ICFP, Wikis [weekly-2025-01-20]

January 20, 2025

This week was my first full week back from the break and I found it challenging trying to get back into what I had been working on previously.

ICFP Papers

In conversation with Anil, we looked at options for submitting a paper to ICFP. I wrote up some notes on some options we discussed.

Forester

I spent some time this week converting this website to using Forester. I'm not a huge fan of the syntax, especially as a lot of my site's content was already in markdown. So I wrote a markdown frontend to forester which is available on Github.

The markdown frontend integrates very nicely and only a few changes were needed in the core logic of Forester itself. Additionally, for any features not directly supported in markdown there is an escape hatch using code blocks such as:

```forester
\put\transclude/numbered{false}

\transclude{pxf-1000}
```

Personally, I'm still getting to grips with the bottom-up approach to building this site, atomically creating notes and reference cards that then are linked in many places.

I'm excited to see how I can integrate some of the Forester concepts into "Shark".

OCaml

In the OCaml world, I spent time on ppxlib, reviewing PRs to update the lower bounds of the library, fixing effect syntax problems and bumping internal AST to 5.2. I've also spent some time looking into de-objecting Eio, making the OCaml types more friendly to new users. I need to revive my port of Vpnkit to Eio for the thoughts on ICFP 2025 too.

I also want to modernise and make more public my OCaml code for creating little shells in OCaml too -- I think the ideas here really have legs and would like to find a conference to submit them too.

I also met with the single, OCaml Outreachy intern working on ocaml-api-watch.

Misc.

I met with most of my Part II students this week which was nice to catch up and see how their projects are going. I also started marking work for my first year students who are at the induction part of their Discrete Maths course.

Thoughts on KC's "Foundations for Hacking on OCaml" › Self-host, self-host, self-host [self-host-love]

November 10, 2025
Patrick Ferris

Self-hosting is another great way to build those skills. My website has had at least five different incarnations at this point. Some highlights include when it was completely made with GatsbyJS (when that was the new hottness), when it was a mishmash of OCurrent and Omd, to now, when it is one part Graft and one part Forester.

Before I had a website, I was also blogging on Medium (I won't pretend they were the best blogs ever, but I learnt a lot by writing them). I have since moved past Medium as their values do not align with my own, but it was an important stepping stone.

Thanks to Ryan, who is very generous with his time, I now self-host a variety of services using NixOS.