Michael W. Dales [mdales]

https://digitalflapjack.com

Backlinks

Weekly 2026 W43 [weekly-2026-w43]

October 26, 2026
Patrick Ferris

This week was a bit of a flop as I ended up coming down with something and was unable to work for about half of it. A shorter note this week due to this.

Paper Writing

October 26, 2026
Patrick Ferris

I had a productive conversation with Anil about the direction of the paper I am currently writing. Plenty of work to do with the deadline looming, but I am starting to think the framework we're building could have wider use than the applications I first thought of and that is somewhat exciting.

More on this during the week as I'm back to operational strength!

Forester 6.0

October 26, 2026
Patrick Ferris

A new release of Forester is imminent and with that comes upgrades and fixes to Graft. If you are reading this post then you are enjoying the delights of Forester 6.0 compiling artefacts from an in-the-works version of Graft.

The upgrade was mostly smooth, though some of my CSS was a bit broken, and it was easier to do a merge of the most important bits with the new Forester stylesheets.

Depending on the internals of Forester is a little tricky as it is first and foremost a CLI tool. The library APIs are likely to be less stable and are a bit of a moving target. The changes, however, were pleasantly small as far as I can tell, limited mostly to some changes in the reporting infrastructure and the strictness of all Forester codes needing a location (somewhat tricky as there is no one-to-one mapping from markdown/yaml/bibtex to these codes). For the latter case, we do our best and where possible try to map the generated codes back to the fragment of the source language Graft has transpiled.

Now that I am dog-fooding Graft and Forester, it will be easier to keep things working ahead of the next release.

Thanks to Kento and Jon for their work on improving Forester.

Outreachy

October 26, 2026
Patrick Ferris

I have lots of thoughts about Outreachy this year. The volume and quality of contributions is quite unlike anything I have seen before, in large part due to the current ease of accessibility and quality of LLMs. This does fly in the face of somewhat strong policies set out by Outreachy to help maintainers handle AI contributions.

Going forward, I imagine it will be difficult to balance pedagogical values with financial rewards given wide-spread AI usage (regardless of project-specific or organisational AI policies). Not something that quite exists, in the same way, say, for university undergraduate students.

I am still trying to find my feet in the middle of this. I have never felt quite this alone in the open-source software world.

Misc.

October 26, 2026
Patrick Ferris

Claudius Playground

October 26, 2026
Patrick Ferris

Michael spoke to me about some accessible ways to play around with his fantasy console library Claudius. At this point, I fancy myself a bit of an expert at shoving OCaml code into browsers, so I pulled together a rough and ready Claudius Playground (code is freely available on tangled). Happy coding!

Music

October 26, 2026
Patrick Ferris

I was able to go see the Mountain Goats before the start of last week, with Craig Finn supporting. I enjoyed Isaiah 45:23.

The Mountain Goats at Vicar St. Dublin, 2026-10-18.

Catching up in May [weekly-2026-w22]

June 1, 2026
Patrick Ferris

I never quite found the time in May to write weeknotes. I think that's, in some ways, quite a good thing. My philosophy on weeknotes is not that they are a public punchcard of my progress. They are a tool for organising thoughts and keeping my colleagues abreast of what I am up to. Only the latter suggests any necessity for timeliness and given I frequently message, call and zulip them, if a few weeks fall between the cracks then that's okay!

All that being said, I have tried to provide some thoughts and things I have been up to over the course of May and not just week 22 of the year.

Tools for Shells

June 1, 2026
Patrick Ferris

Some tools and libraries related to Merry.

Meio

June 1, 2026
Patrick Ferris

The shell work continues, and as the complexity internally increases I have found myself wishing for some more tools for debugging concurrent Eio programs. Given enough pipes and compound commands, it is fairly easy to introduce a fair deal of concurrency to your shell programs. To this end I hope I have improved meio somewhat in this fork.

In particular, there were several things I wanted to improve.

Simple UI improvements -- the set of tools for building TUI UIs from nottui are, on closer inspection, rather good and simple. For example, turning a particular UI component into a scrollable pane is as simple as wrapping it in a call to this function.
Logging more runtime events -- Eio programs emit information via OCaml's runtime events. This is how meio knows which fiber is currently active for example. It also emits information like "the current fiber created a new promise" and "this fiber is blocked waiting for a promise". I have not figured out an intuitive way for representing this in meio's tree-view, but I did turn these into "system logs" that appear in the per-fiber logs which is already an improvement. Thanks to Thomas for helping understand the runtime event sequence for logging this information.
Timestamping where possible -- all runtime events come with a timestamp, these were mostly discarded by meio.

Some aspects of meio made it clear that labelling Eio.Switches and Eio.Promises might be a good practice. After reading Michael's Eio forays, I can't help but wonder if a more opinionated Eio for Systems Programmers guide might be useful. I think the Eio README does a wonderful job of introducing most of the low-level concepts, but not quite how they are used by the working systems programmer.

Glob

June 1, 2026
Patrick Ferris

A little dismayed at the lack of progress I was making with Merry, I switched to a specific problem I knew I wanted to fix -- removing the dependency on glob(3), replacing it with a pure OCaml implementation.

let get_txts fs =
  Glob_eio.glob fs Glob.(of_component [ dir_sep; string "home"; dir_sep; star; string ".txt" ])

Anil provided some issues with fixes too which was very appreciated.

Beyond factoring out fork+exec, Merry should now be portable to the browser. Of course there are lots of other refactorings needed to (e.g. a fake-filesystem) -- but I'm interested in how I can use Eio's filesystem capabilities to do this pretty seemlessly. Having an emeddable, pure OCaml shell may also unlock some cool debugging capabilities for unikernels too!

Geocaml

June 1, 2026
Patrick Ferris

Michael has been valiantly trying some of the Geocaml libraries in his work on rendering LiDAR point clouds. Where possible, I have been reviewing Michael's PRs and helping implement features he might find useful. I also took his Laserver for a spin, and suggested a few changes.

Rtree

June 1, 2026
Patrick Ferris

After a comical series of attempts, I (or rather Anil) released rtree.0.2.0 which contained:

A bug fix where not all the tree values were being returned.
A new bounds function for getting an Rtree's maximum bound.
A set of simple removal functions.

PROJ

June 1, 2026
Patrick Ferris

Michael added some new functions to ocaml-proj. Previously, I had worked on making PROJ more portable by using virtual libraries. This was a little restrictive, as it forced users to always implement a JS version of any functionality that exists in the native version of PROJ.

To make it easier to contribute (and allow the C bindings to contain more specific bindings) I carved out the two virtual library implementations into standalone libraries, and reimplemented the virtual library support using these libraries. Now projects can depend directly on proj_c if they wish to have more features in exchange for being less portable.

Ppxlib

June 1, 2026
Patrick Ferris

Thankfully, work on ppxlib has been relatively quiet. Nathan also opened a PR for initial OCaml 5.6 support. Keeping on top of 5.6 features is probably the right way to go about this (rather than one giant PR during the 5.6 release cycle).

I'm particularly happy to have some better migration error handling. The previous errors, though uncommon, were confusing ppx users and hopefully this will help ppx authors and ppxlib maintainers triaging important encoding bugs or feature wishes.

Self-hosting

June 1, 2026
Patrick Ferris

I now have a fairly workable setup for self-hosting source code with git.sirref.org and tests.sirref.org. One piece that is missing, is a publishing workflow. I spent a little time building a YOCaml static-site generator for packaging OCaml libraries. Unfortunately, this was full of rabbit holes and busy work which was frustrating.

Tar-eio

June 1, 2026
Patrick Ferris

One such rabbit hole was tar-eio. It was in a bit of a sorry state, so I took the opportunity to improve the API for new users. Tarring a directory is now as simple as:

let foo = Eio.Path.(cwd / "foo") in
let foo_tar = Eio.Path.(cwd / "foo.tar") in
Eio.Path.with_open_out ~create:(`Or_truncate 0o755) foo_tar @@ fun foo_tar ->
Eio.Switch.run @@ fun sw ->
let t = Tar_eio.create ~sw foo in
Tar_eio.run t (Tar_eio.flow_of_file tar)

There was also a series of security issues that the tar library had that were fixed too, you can read about those in Thomas's weeknotes.

Forking in Shells & Library Maintenance [weekly-2026-w14]

April 3, 2026
Patrick Ferris

This week, Merry got close to executing apt-get install ca-certificates and running Debian's debootstrap scripts correctly. Nathan and I released Ppxlib 0.38.0, I added support to ocaml-proj for compiling to the browser and I fixed a long-standing bug in hilite.

Forking in Merry

April 3, 2026
Patrick Ferris

So far, Merry has managed to support many features of the POSIX shell specification without needing to do a fork(2) without an exec. Or, to put it another way, all the forking that Merry does, will only be followed by running C code. This is actually a feature of Eio's Process API. It is required to make process execution safe in the context of multiple domains in OCaml (mostly due to ensuring consistency within the garbage collector). However, it is perfectly okay to fork and run OCaml code provided there is only a single domain.

In a shell, many features of the shell language require that commands (or built-ins, function applications etc.) execute within a subshell. The specification does not go into detail about how shells should implement this, but many choose to fork in order to preserve some state in the parent (for example, the file descriptor table). Some shells try to minimise the number of forks as an optimisation. The korn shell (ksh) is one such shell.

Using the notation $(command) will cause command to execute in a subshell of the current ksh. In many instances, ksh will not actually fork/exec a subshell when command is a built-in or a shell function.

By trying to adhere to Multicore OCaml's "thou shalt not fork" commandment, Merry is much more similar to ksh in this regard. However, whenever there is an interaction with a shell built-in and semantics that need a child process, things get tricky very quickly. Consider the following:

while echo "hello"; do
  :
done | head -n 3

We have a mix of shell built-ins, normal commands, a compound command (the while loop) and a pipeline. Each individual command in the pipeline requires you to run them in a subshell and set them up before executing them. Without a fork for that first command, you may end up looping forever which is not the intended behaviour here. I had a good conversation with Michael about the implications of this for reproducibility.

For now, I have started implementing a fork for some of the shell features whilst still trying to maintain the functional core of the implementation.

An OCaml 5.5 compatible Ppxlib

April 3, 2026
Patrick Ferris

Nathan and I released an OCaml 5.5 compatible version of ppxlib.

Ppxlib Releases › 0.38.0 [ppxlib-release-0-38-0]

March 30, 2026
Patrick Ferris

Nathan and I released ppxlib.0.38.0 last week. Its main feature is full migration support for the upcoming OCaml 5.5 compiler (currently in its alpha3 release). This means supporting the handful of new features landing in OCaml 5.5: modular explicits, external type declarations and arbitrary "local" structure items.

As Nathan and I continue to find a plausible maintenance story for ppxlib, Nathan has opened an issue on the OCaml compiler to discuss the idea of adding additional extension points to the language.

This comes from the desire to be able to store encoded versions of new OCaml features inside older abstract syntax trees.

There are also some nice bug fixes in there too:

A potential OOM loop has now been removed.
Locations have been restored to long identifiers!

Browser support for ocaml-proj [ocaml-proj-virt]

April 3, 2026
Patrick Ferris

A while ago I built some "modern" bindings to PROJ4 in OCaml. After reading Jon Ludlam's weeknotes (and speaking with him and Anil, I thought it might be nice to add a Javascript backend to those bindings). This was relatively straight-forward using Dune's virtual libraries and is available on Github.

With virtual libraries, your own data analysis could (if you wished) depend solely on the proj library and later choose to either link it with proj.c or proj.js depending on where the analysis is being deployed.

I briefly looked at WASM support, but quickly realised there was not much appetite for it and trying to compile around the C FFI was going to be hard.

Hilite updates

April 3, 2026
Patrick Ferris

Since I helped relaunch the ocaml.org website a few years ago, I have maintained hilite, a tool for doing build-time syntax highligting.

It is being used on the ocaml.org website, in yocaml_markdown and also in Michael's webplats (when it works...).

And it was Michael who led me to finally fixing and adding syntax highlighting support for Python and Go. Once I was on this roll, I finally took a look at fixing the long-standing bug of trying to highlight ocaml-mdx code, like the following:

# 50.123;;
# 1 "file.ml";;

In fact, this code now highlights fine, but was the source of the bug. The syntax highlighting grammer confuses the ocaml-mdx # as a toplevel directive and trouble ensues.

Taking stock & Graft updates [weekly-2025-w45]

November 10, 2025
Patrick Ferris

I ended up working on a real mix of things last week. I managed to get two large posts uploaded to this site: an ICFP recap and a deeper-dive into the random bits of OCaml hacking. This was pretty useful in terms of taking stock of where I was at, I always find post-conference, I'm a little lost. The conversations and ideas from ICFP are simultaneously exciting but also pretty inactionable.

Graft Updates [graft-updates]

November 10, 2025
Patrick Ferris

It would seem that there is a potential second user of Graft! A great source of motivation for some tidying and feature implementing. At the end of last week I fixed some little bugs in Graft and (more importantly) allowed users to name their subtrees. Upstream Graft also now pins and uses my Bibtex implementation.

OCaml Geotessera

November 10, 2025
Patrick Ferris

As I discussed in my OCaml roundup, I spent a bit of time polishing the OCaml geotessera code. In particular this meant starting to handle low-level geospatial transformations allowing users to project latitudes and longitudes into the rows and columns of their raster data. I feel like I'm stumbling onto a path Michael has carved out with Yirgacheffe. I'm following along with his work excitedly and we had a good catch-up last week discussing how libraries should handle things like automatic, numeric precision conversion.

As a fully paid-up member of the static types fan club, I'm often evangelising the benefits of types. However, I hit a little quirk of the Nx library in OCaml (a numpy equivalent). Take a look at this sum function:

let sum nx = Nx.sum nx |> Nx.item [];;
val sum : ('a, 'b) Nx.t -> 'a
let arr = Nx.create Nx.int8 [| 2 |] [| 64; 64 |];;
val arr : (int, Bigarray.int8_signed_elt) Nx.t
sum arr;;
- : int = -128

Even though sum takes an ('a, 'b) Nx.t array that stores elements with representation 'b and allows you to read/write to the elements with OCaml values of type 'a, the sum function performs the calculation in the world of the 'b representation type (here signed 8-bit integers). So we get an overflow! Perhaps not too surprising but easy to overlook! The moral of the story is that we still need to be careful in the OCaml world too.

Shelter

November 10, 2025
Patrick Ferris

I did a little cleaning up of the Shelter code too. I noticed a few bugs from the demos I did at ICFP. Most were pretty straight-forward to fix, and the test-suite is now checking parallel for-loops and conditional execution are working.

This week

November 10, 2025
Patrick Ferris

I hope to get a few things done this week:

Run some part of the LIFE pipeline through Shelter.
Deploy the Shelter daemon allowing other people to connect and take it for a spin.
Spend some time writing and reading, pulling ideas together rather than always writing code!
I have a slightly harebrained scheme that combines Shelter, atproto, Forester, capn(proto/web) and webcomponents... I want to see if I can pull that together.

ICFP 2025 [icfp-2025]

October 24, 2025
Patrick Ferris

Two weeks ago I was fortunate enough to attend the International Conference on Functional Programming in Singapore. My first time in Asia and my second time at the conference, what follows are some thoughts and presentations I enjoyed whilst I was there.

I must thank my office mate (and friend!) Ryan Gibb for his camaraderie and sage advice: "the best track is hallway track". Not to mention all of his photos too. Thanks Ryan!

From right to left, top to bottom: Anil Madhavapeddy, Sadiq Jaffer, Patrick Ferris, Ryan Gibb and Roy Ang.

Random Hacking

October 24, 2025
Patrick Ferris

Below are just some of the pieces of random hacking I got up to whilst in Singapore.

OxCaml and Uring [oxcaml-and-uring]

Patrick Ferris

Jane Street were a big presence at ICFP 2025, carting along with them a shiny new OCaml compiler: OxCaml. If have been playing around with OxCaml recently but nothing outside toplevels in Javascript. Until now!

After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change:

  module Heap = struct
    type ptr = int32#
    (* ... *)
  end

  type cqe = {
    user_data_id : Heap.ptr;
    res : int32#;
  }

The idea being that a completion queue entry (a notification that some operation has completed) could be fully represented using 64 bits (two 32-bit, unboxed values). You can see how this impacted the library! I'm not certain about this change (and I'm sure I did it wrong) but it was nice to realise OxCaml gives you this kind of control. However, I am worried about the ergonomics of manipulating values like int32# and the temptation to case it into an int (presumably losing a good portion of the value of having an unboxed value in the first place).

Bibtex Parser

October 24, 2025
Patrick Ferris

On the flight over to Singapore, I started to write a post for this website. Some readers may remember that I use forester after a preprocessing step via graft to build this site. Part of that preprocessing consumes bibtex files and outputs forester trees. For the initial graft release I used the excellent Talaria Bibtex library. Unfortunately, it presented a pretty confusing API that, whilst nicely typed, felt a little cumbersome to use. Additionally, it was too specific in what it would accept as valid Bibtex. I wanted something that was a little more straight-forward to use (if a little fast-and-loose when it comes to types...). And so I built bib.

This was my first real attempt at using the Bytesrw library. I was happy with the results, although I did have to copy a substantial amount of code from other bytesrw-compatible libraries. I'd be happy to shift to this bytes-first world if it wasn't for the ever-present-in-my-mind issue of relocatable values in the OCaml heap!

Geocaml Libraries

October 24, 2025
Patrick Ferris

There has been some excellent outreachy work happening on the geocaml/ocaml-tiff library which is a potential Outreachy internship project for December 2025. More on this in a future post.

On the way back (and a little during the conference), I spent some time building an OCaml counterpart to geotessera, a library for working with Tessera. The library is not ready for people to use directly, but I was working on various constituent parts including:

Modern OCaml bindings to PROJ (a coordinate reference system projection library)
Further work on a WKT library for OCaml
Coming soon... ocaml-geotessera

Talks

October 24, 2025
Patrick Ferris

Hazel of OCaml at TyDe

October 24, 2025
Patrick Ferris

The type-driven development (TyDe) workshop was held on the first day of ICFP at the NUS School of Computing. I find the workshop name to be a little misleading, but perhaps this comes from my engineering background. The talks focused a lot on type theory, presenting the formal mathematics of interesting type systems with less focus on why it might be useful to have such an exotic type system... but then again, that was the nature of the entire conference.

I presented a talk on hazel_of_ocaml. The slides are here, along with the extended abstract. My talk ended up focusing a lot on the pedagogical benefits of having a type system that helped explain the most important thing a type system does... catch type errors! This work represents a small piece of work inside Max Carroll's excellent part II project, which he presented at the HATRA workshop: Decomposable Type Highlighting for Bidirectional Type and Cast Systems.

Scientific Programming at PROPL

October 24, 2025
Patrick Ferris

I think it is important to be honest whenever you get a talk accepted that you otherwise feel should not have been. My talk "what we talk about when we talk about scientific programming" is one such talk. The slides are here. The talk itself was nothing to write home about, but I think there were a few nice ideas that came out of it and the conversations I had with fellow PROPL attenders.

TMF Data Discrepancies Redux

October 24, 2025
Patrick Ferris

On the eve of my talk, I decided to re-run some analysis I did a few years ago on the tropical moist forest dataset, knowing that since then a few more years had become available. After fighting with some bit-rot in the Google Earth Engine script I had used, the new results were not very uplifting.

Land Use Class	Percent Change
Undisturbed	-2.9
Degraded	-5.5
Deforested	8.6
Regrowth	0.5
Water	0.0
Other	-0.6

The data in the table corresponds to the land use class values in Indonesia in the year 2008. The analysis computed the difference across each of the classes between the dataset released in 2021 and the dataset released in 2024. Most notably, the amount of deforested land has gone up by 8.6 percent!

As a somewhat tangential, though related, aside, I had the pleasure of talking to Sam Westrick, Seong-Heon Jung and Sundara Vishnu Satish of the ParCour research group. They were telling me about MaPLe, a programming language from the MLs built for parallelism. We discussed some ideas of using real-world, geospatial algorithms as benchmarks for MaPLe. Michael, if you got this far remind me to chat to you about this!

Shelter Reactions

October 24, 2025
Patrick Ferris

At the end of my talk, I briefly mentioned shelter. By far, this was the topic I got the most engagement on post-talk. During the conference, this was also true when people asked what I was working on. The tag line "a shell with undo" seemed to capture the attention of most people.

I'm feeling a renewed sense of interest in my own work thanks to that, even if the "research" contributions are not very apparent. However, I also had a good discussion with KC about the underlying MRDT in Shelter and how we might think about collaborating in the future on MRDT-related work.

Related to that, I managed to spend the evening in Chinatown with Roly Perera. Roly's work has focused on Fluid, a transparency-based programming language that allows authors to reveal their computation to their user. Excitingly, Roly presented a new text-based part of the language at PROPL. Our work is in the same universe, loosely orbiting some notion of reproducibility and explainability, but quite different in engineering work and how users might actually interact with our systems. I thoroughly enjoyed talking about solutions or research ideas that preserve existing workflows. Roly was great at pulling that out of the current work I have been doing.

Other Talks and Highlights

October 24, 2025
Patrick Ferris

I was a part of, often in a very small way, some other talks that were presented at ICFP including

Functional Networking for Docker, I encourage you to read Anil's notes on the topic. It is also worth mentioning Anil's post-posix talk (not unrelated to my Uring musings above).
Michael gave an excellent, very live, talk about Yirgacheffe at PROPL which I thoroughly enjoyed. I'm also excited about his recent developments which I'm sure you can clue into over on his blog.

Below is a non-exhaustive list of talks I enjoyed whilst at the conference. Some relevant, most of them not!

By far, one of the best talks was Polynomial-time Program Equivalence for Machine Knitting. I think the talk really nailed the "here's the idea, the details are in the paper"-presenting style as they described an "...algorithm that canonicalizes the algebraic representations of the topological semantics of machine knitting programs".
I attended most of the OxCaml tutorial presented by the incredibly friendly Gavin Gray.
Ryan's talk Spatial Programming for Environmental Monitoring at PROPL was excellent.

There's plenty more to talk about, and I'm sure my future posts will reference other parts of my experience at ICFP 2025. Thanks for reading, and if anybody is interested in any aspects of this do reach out!

OCaml Weekly 2025 w32 to w35 [ocaml-weekly-2025-w32-w35]

September 2, 2025
Patrick Ferris

I have been working on a few different OCaml-related projects over the last few weeks. This is also coinciding with my partner and I moving across the UK, which has made finding time to write weeklies and posts a little tricky. Nevertheless, here are some of the things I have been thinking about and working on!

I managed to publish one signficant post this month: a retrospective on Irmin.

Eio

September 2, 2025
Patrick Ferris

Increasingly, I'm feeling the dream of a unified framework for asynchronous IO slipping through the OCaml community's fingers. It is perhaps not such a bad thing, and I think with the right library authoring we can at least get to a place where it isn't so bad, for example providing read functions as opposed to using an opinionated IO library directly.

That being said, I am a very happy user of Eio when those choices do not matter, as is the case in building your own application (e.g. Shelter). To this end, I have spent a good bit of time upstreaming support for various missing pieces in Eio's API including:

Setuid and setgid fork action's for the process API.
Set process group support for job control in the process API.
Responding to Buf_write.of_flow request, and tinkering with the example there. I think this does highlight the awkwardness of making code portable across concurrency mechanisms, particularly with Eio's structured concurrency.
I did some investigating into EINTR bug which seems to be stemming from a known-issue on Uring in that writes are not buffered which usually does not matter except perhaps when there are parallel writes to stdout.
Spent some time thinking about the fiber local storage across domains issue, I've passed on some thoughts to folks working on this.

Vpnkit

September 2, 2025
Patrick Ferris

You might recall I was interested in using vpnkit. Hannes has done an amazing amount of work (patching and releasing) a series of packages to get this into a place that is much better and could be considered soon for merging. This defunctorisation is actually very useful for the Eio port I wrote a long time ago.

Papers and Talks at ICFP

September 2, 2025
Patrick Ferris

Somehow, I have ended up on lots of papers and talks at ICFP and the co-located events in October. The vaguely OCaml-related ones include:

Essentially a Vpnkit Experience Report.
An extended abstract on generating a corpus of ill-typed Hazel programs was accepted into TyDe workshop.
Relatedly, the work that project supported was accepted into HATRA which was the Part II project I supervised: Decomposable Type Highlighting for Bidirectional Type and Cast Systems.
And two PROPL talks!

Outreachy

September 2, 2025
Patrick Ferris

We have come to the end of another Outreachy round! I will write more on this soon in its own separate post. But for now I am very grateful to this round's mentors gridbugs and mdales, and also our fantastic interns. If you are interested, please do watch our demo day presentations.

The next round is fast approaching and we still need to work out the logistics. But I had a good conversation with mdales about possible Geocaml projects that I intend to submit!

OCaml Weekly 2025 w30 and w31 [ocaml-weekly-2025-w30-w31]

July 31, 2025
Patrick Ferris

A bumper edition today, a fortnightly.

Opam Releases

July 31, 2025
Patrick Ferris

I did a little personal spring-cleaning of packages I maintain and pushed a few releases to opam. This included a 0.1.0 release of graft! There is some light documentation online at https://graft.sirref.org.

Elsewhere, Hilite got a 0.5.0 release. This release makes the core syntax highlighting separate from the markdown part of the library. You can now generate pairs of tokens and identifiers to plug into any "OCaml sourcecode to format" tool you might be building (e.g. graft!).

Interactive OCaml Presentations

July 31, 2025
Patrick Ferris

I had fun trying to fuse slipshow and x-ocaml!

Slipshow x x-ocaml [slipshow-x-xocaml]

July 23, 2025

A short, explanatory post about combining two very fun pieces of work in OCaml.

Paul-Elliot has been building Slipshow for some time now where slides are slips and your presentations run vertically. More recently, Arthur has built x-ocaml, a web component library for executable OCaml cells embedded into OCaml.

Using xocmd, a small tool I built for translating markdown codeblocks to x-ocaml components, your Slipshow's can now be executable!

xocmd learn-effects.md | slipshow compile - > learn-effects.html

Take a look at an example! (or the source markdown).

I really like this light-weight approach to building interactive presentations for explaining things in OCaml (e.g. over running a jupyter notebook server).

I tried using x-ocaml with Irmin and something is not quite right with some of the runtime JS code being generated (see the issue in case you can help). This was intended to be used alongside a longer-form retrospective I am writing on my use of Irmin over the years.

Outreachy

July 31, 2025
Patrick Ferris

I have been helping Michael's intern whilst he has been away on their project Claudius. It has reminded me, again, how inpenetretable some of OCaml's tooling is. In this case the generation of opam files from dune-project files, and in particular, the required manual steps when you wish to remove a dependency (1. remove it from the dune-project file, 2. run dune build to update the opam file, 3. remove all occurences of the library from dune files across your project).

Ppxlib

July 31, 2025
Patrick Ferris

Work on Ppxlib has been fairly varied this past two weeks. I mentioned before about an exiting plan Nathan has planned to help ease the burden on ppxlib maintainers and ppx authors whenever there is an OCaml parsetree bump. That, however, is currently shelved as we are dealing with the fall out of the 5.2 bump. Repr got some fixes pushed to it, but this unconvered more issues.

Misc.

July 31, 2025
Patrick Ferris

I spent some time cleaning up container-image, a tool primarily written by Thomas G. to fetch OCI images from repositories. The tidy up went surprisingly well, except for some spurious HTTP errors with AWS. Maybe I'll spend some proper time trying to help the state of HTTP clients in OCaml (of which there are many, but few that work very well out of the box in my experience).

OCaml Weekly 2025 w29 [ocaml-weekly-2025-w29]

July 15, 2025
Patrick Ferris

Ppxlib

July 15, 2025
Patrick Ferris

I met with Nathan this week to discuss future plans for ppxlib. The current state of affairs is that ppxlib is becoming unmaintainable. This is primarily a knock-on effect from changes being made to OCaml's parsetree (e.g. labelled tuples being added in 5.4). Nathan has a plan that will provide two key properties.

Migrations, which allow old compilers to be used with new ppxlib releases, will be more compatible. For example, we will be able to migrate new features downwards and back up without raising an error.
Ppx authors will be able to use new features in an opt-in workflow, rather than ppxlib bumping the internal AST (like we did in ppxlib.0.36.0). This will reduce the maintenance burden significantly whilst still allowing users to write ppxes for new OCaml features.

I also started looking into some older issues in ppxlib related to performance. This is work-in-progress, but I am trying to improve the performance of some passes done by ppxlib. To better understand what was making ppxlib slow, I wanted to use runtime_events_tools but I was dismayed to see it wanting to install over 100 packages! I opened a PR to reduce the number of packages. I think this kind of work goes a little unrecognised as it is not very glamorous. However, I think it really benefits the OCaml community in the long run.

Outreachy

July 15, 2025
Patrick Ferris

In Outreachy news, we had a wonderful mid-internship video call with all the interns and mentors to catch-up on how everyone is getting along. Seeing the progress everyone has made was great! I am very grateful for the work that Michael and Steve have put in so far to make this a very successful Outreachy round for OCaml.

In sadder news, an email was shared with all Outreachy mentors detailing the increasingly critical financial situation the project finds itself in. There are ongoing discussions about how costs can be cut including potentially only running a single round a year.

Graft

July 15, 2025
Patrick Ferris

With the release of Forester.5.0, I made a plan to make a release of Graft.0.1. Unfortunately this is blocked by a new release of hilite, a tool I built for doing build-time syntax highlighting for OCaml code. This powers the syntax highlighting on ocaml.org.

PhD Work and Scientific Programming [rest-of-monthly-2025-06]

June 30, 2025
Patrick Ferris

I have been away on holidays. Some of that was spent hiking in the Massif de Calanques in Marseille, unfortunately this week they were on fire. Anyway, this "weekly" fills in the blanks for the rest of June 2025.

A Calanque just outside of Marseille, a canyon-like structure of calcified stone on the edge of the Mediterranean.

Revisiting hazel_of_ocaml

June 30, 2025
Patrick Ferris

I spent some time thinking and writing about hazel_of_ocaml and the benefits of OCaml as source language and Hazel as a target language. The stability of OCaml with a powerful type-system makes it very expressive.

Hazel, on the other hand, is a hotbed of PL theory research. The typed holes, which are fully supported within the language, are particular powerful targets for a transpiler. They allow transpiler developers to incrementally support the language whilst still getting full feedback of their generated source code. Whilst it is a niche use-case, I thought it was an interesting point to note.

A second year report

June 30, 2025
Patrick Ferris

I wrote my second year report at the end of June. The process was a bit chaotic, but I think ultimately it was useful for clarifying some ideas I had rattling about in my head.

Part of that was working on my publications page and, through a conversation with Anil, realising that my work on Graft was in many ways a part of my own research.

If you are interested, feel free to peruse both my first year and second year reports. Take them with a pinch of salt! I think it is useful to be fairly open with this sort of thing when you can be. Maybe someone will find them useful.

What is scientific programming?

June 30, 2025
Patrick Ferris

This question arose whilst writing my second year report. It got me thinking about how we distinguish scientific programming from other kinds of programming. I came across a paper by Tim Storer: Bridiging the chasm: A survey of Software Engineering Practice in Scientific Programming. I enjoyed a few aspects of this paper.

Relating scientific programming to the scientific method so directly was pretty useful insight for me. Falsifable hypotheses, repeatable experiments and reproducible results when applied as characteristics of the computer stack (e.g. OS, programming language, filesystem etc.) might be a useful lens to argue for better tools to enable this.
The breadth of Storer's case-study analysis on how scientific programming has "gone wrong" is very good, if anyone needs a reference to draw on to help support their tools or research, this seems like a good index to use.

Ppxlib

June 30, 2025
Patrick Ferris

I was reminded again of the pain of helping maintain ppxlib. We are coming to the conclusion, that in its current form, it is becoming unmaintainable. I took a stab at updating bisect_ppx to the latest ppxlib. With OCaml entering a potentially turbulent parsetree era, it might be time to take stock of this and propose some fillers to help. Nathan Rebours and I are meeting next week to discuss ideas he has to make parsetree migrations smooth sailing for all!

Geocaml

June 30, 2025
Patrick Ferris

Work on ocaml-tiff has stalled, which is totally fine. Michael and I independently are working on other things and have obligations to fulfill. Maybe some day we'll carve out enough time together to push forward on this project, but not this week.

Metashells [weekly-2025-05-12]

May 19, 2025
Patrick Ferris

This week, I feel I have been stuck fighting the OCaml ecosystem trying to keep my OxCaml work afloat. Aside from that, Ryan and I made some really nice progress with Shelter, culminating in Ryan describing it as a metashell.

Shelter the metashell

May 19, 2025
Patrick Ferris

The main progress this week with Shelter was composing runc's terminal mode with entering raw terminal input mode on the Shelter side. This is inspired by Ryan's own work on capability interfaces.

Shelter remains mostly intact, acting as an interactive shell. However, just before executing a command we switch to receiving and sending raw terminal inputs and outputs. This means tools like vim now work in Shelter! Not only that, but users can now activate an inner shell (e.g. zsh) and enjoy all the usual features of a fully-fledged shell (tab complete, fuzzy history search etc.) and upon exiting that shell, Shelter will snapshot the session. This lets you alter the granularity of snapshots from the command-line.

Louis Pouzin's "SHELL" [pouzin-shell]

May 19, 2025
Patrick Ferris

I spent some time reading part of the multics design documentation this week. Louis Pouzin coined the term "SHELL" in this document, and I was reminded yet again just how important it is to be a good writer even as a "computer science researcher". For example, this excerpt from the requirements section of the document

The previous definitions imply that a command MUST be designed while keeping in mind the user, sitting at his console, wondering about what might be going on, mistyping or forgetting arguments, even if fully aware of the conventions, and possibly interfering with the command by hasty quits, carriage returns, and other temperamental reactions.

And then later, when defining the "SHELL".

We may envision a common procedure called automatically by the supervisor whenever a user types in some message at his console, at a time when he has no other process in active execution under console control (presently called command level). This procedure acts as an interface between console messages and subroutine. The purpose of such a procedure is to create a medium of exchange into which one could activate any procedure, if it were called inside of another program. Hereafter, for simplification, we shall refer to that procedure as the "SHELL".

It still surprises how little the undergraduate degree in computer science at Cambridge focuses on writing skills.

OxCaml

May 19, 2025
Patrick Ferris

Last week, I got a toplevel with OxCaml working. This required a serious amount of work to understand the changes Janestreet have made to obscure parts of the OCaml compiler and then working those into tools like js_of_ocaml.

This week, Janestreet pushed their latest rounds of changes and of course everything broke! I spent some more time fixing it all back up. I'm not entirely sure how maintainable this is. The problem is that, whilst things compile, the programs do not work together! Only when someone uses the program do the bugs surface.

Other OCaml Work

May 19, 2025
Patrick Ferris

I worked on some other parts of the ecosystem this week.

Ppxlib

May 19, 2025
Patrick Ferris

I helped review some changes to enable Janestreet to have ppx rewriters via attributes (usually they are via extension points). It is a bit of a controversial change to ppxlib as we try to keep the API predictable for users:

Extension points are rewritten: this means the part that is rewritten is nicely delimited by the extension points start and end.
Attributes extend: attributes do not rewrite the code they are attached to but rather extend the code with new AST nodes.

We will see what we decided to do (most likely provide the functionality behind some kind of "expert" interface).

Tiff

May 19, 2025
Patrick Ferris

See Michael's notes.

I spent some time trying to speedup the LZW decompression of TIFF files in the pure OCaml tiff library this week(end). The two big changes to help with this are pretty common when speeding up these parts OCaml programs:

Allocate less
Does less work

In terms of allocating less, the original implementation was using a char list to represent LZW strings. Manipulating these becomes quite costly, particularly since the most common operation is appending a single character to the end of a list. Converting this to use OCaml's immutable string saved a ton of allocations.

In terms of doing less work, I opted to bypass Cstructs sane (but slow) bounds checks in some of the "hotter" parts of the code. In particular, LZW ends up reading potentially huge arrays full of bytes one-by-one. So reading each byte needs to be quite snappy. This is a bit of a trade-off in terms of "safety" but we are in control of this code so I'm not too worried about that.

Here are some results decompressing a fairly large array of some elevation data.

before: | tiff/lzw/cea │ 523851.2259 mjw/run │ 3289761.5414 mnw/run │ 9806796.7121 ns/run│
after:  │ tiff/lzw/cea │  27846.2408 mjw/run │  587928.7527 mnw/run │ 8457161.3761 ns/run│

Opentrace [open-trace]

May 19, 2025
Patrick Ferris
https://tangled.sh/@patrick.sirref.org/opentrace

Thanks to Koonwen's excellent libbpf bindings in OCaml, I have been building a little tool called opentrace to make it easier to track an executable's inputs and outputs.

This work was inspired my Michael's self-proclaimed "gross hack": pyshark. Whilst pyshark achieves its goals by injecting code into commonly used python objects and methods, opentrace uses eBPF. By using a lower-level API (hooks in the kernel), opentrace can remain programming language agnostic. However, less information is none about the user's intent compared to something like pyshark.

Monitoring the System

May 19, 2025
Patrick Ferris

opentrace has an all command that will trace the entire system.

$ sudo opentrace all --flags=O_WRONLY
pid,cgid,comm,kind,flags,mode,filename,return
16324,4417,".nheko-wrapped",openat,577,438,"/home/patrick/.cache/nheko/nheko/curl_alt_svc_cache.txt",47
16324,4417,".nheko-wrapped",openat,193,33188,"/home/patrick/.cache/nheko/nheko/cr3ZUHqBErIOe3PlwJ1SuU8zFKKxL12VzrRoHYMH.tmp",47
16324,4417,".nheko-wrapped",openat,577,438,"/home/patrick/.cache/nheko/nheko/curl_alt_svc_cache.txt",47
16324,4417,".nheko-wrapped",openat,193,33188,"/home/patrick/.cache/nheko/nheko/QfZTdZSuC56NdcDQ3aMxJc3BhMhAj8PmtYW1zFDP.tmp",47
2530,4235,"systemd",openat,524865,438,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/cgroup.subtree_control",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.min",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.low",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.high",41
2530,4235,"systemd",openat,524545,0,"/sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/gammastep.service/memory.max",41

The --flags=O_WRONLY argument filters the events where the O_WRONLY flag was set in the call to open.

We also get the name of the current executable linked to the task (comm). The -wrapped is an artefact of using Nix.

Tracing an Executable

May 19, 2025
Patrick Ferris

The primary use case for this tool is to inspect what files your program might be reading and writing.

$ sudo opentrace exec --format=json --flags=O_CREAT -- opam list
$ cat trace.json | jq ".[] | .fname"
"/home/patrick/.opam/cshell/.opam-switch/packages/cache"
"/home/patrick/.opam/log/log-118747-29da3d.out"
"/home/patrick/.opam/log/log-118747-29da3d.err"
"/home/patrick/.opam/log/log-118747-29da3d.env"
"/home/patrick/.opam/log/log-118747-29da3d.info"

The "flags" argument can specify a small boolean formula for checking the open flags of a particular event with | (or), & (and), and ~ (not). Parentheses can be used for precedence.

$ sudo opentrace exec --flags="O_WRONLY|O_RDONLY" -- ocaml --version

Spawning Subprocesses

May 19, 2025
Patrick Ferris

One feature opentrace needs (in this proof-of-concept phase) is the ability to also trace subprocesses.

opentrace is primarily an eBPF program that is loaded into the kernel and communicates with an OCaml program. Events are communicated via a ring buffer and most of the post-processing happens in OCaml. To capture subprocesses, opentrace creates a new control group (cgroup) and places the new process into that group. This gives opentrace a new identifier to track, namely the cgroup.

So consider the following program.

let () =
  Eio_posix.run @@ fun env ->
  Eio.Path.(save ~create:(`Or_truncate 0o664) (env#fs / "hello.txt") "hello");
  Eio.Process.run env#process_mgr
    [ "/bin/bash"; "-c"; "echo 'heya' > heya.txt" ]

It first creates a file using direct calls to functions like openat. Then it spawns a process which creates a new file called heya.txt. This happens in a separate process. However, with the --cgroups flag we can capture both interactions with the operating system.

$ sudo opentrace exec --cgroups --flags="O_CREAT|O_TRUNC" ./main.exe
pid,cgid,comm,kind,flags,mode,filename,return
153187,530807,"main.exe",openat,526914,436,"hello.txt",5
153192,530807,"bash",openat,577,438,"heya.txt",3

Eio's Process API

May 19, 2025
Patrick Ferris

I have used the Eio_unix fork action process API to be able to extend what happens in the child process. Loading most eBPF programs into the kernel requires special privileges hence the need for sudo. When a user requests for a particular program to be executed and traced, opentrace spawns a process via the Eio Process API. Opentrace defines a few new so-called "fork actions", little fragments of C code that are run after the call to fork (clone). Most likely this ends with a call to execve, but other calls are possible for example setuid allowing opentrace to change the user of the child process so it does not run as root. Similarly, this is where (if used) we create the cgroup and place the process into that group.

Limitations: Io_uring

May 19, 2025
Patrick Ferris

Whilst testing opentrace against some of the tools I use nearly daily, I noticed some events were being missed. I tried tracing forester, and only the initial read of forest.toml was logged. It dawned on me that the reason for this was that forester (via eio) was using io_uring to perform most of the IO. Most attempts to open files were bypassing the open system calls, and instead they were being performed by the kernel after reading a submission request for an openat2-style call!

This is not news to seasoned, Linux systems programmers. Io_uring bypasses SECCOMP filters for exactly the same reasons.

$ sudo opentrace exec -- forester build
$ cat trace.csv
pid,cgid,comm,kind,flags,mode,filename,return
155007,535570,"forester",openat,524288,0,"forest.toml",5
155007,535570,"forester",Uring,2621440,0,"",0
155021,535570,"cp",openat,131072,0,"/home/patrick/documents/forest/theme/favicon-32x32.png",4
155007,535570,"forester",Uring,2686976,0,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0
155007,535570,"iou-wrk-155007",Uring,557634,420,"",0

It is interesting to note two things here:

We can tell that forester reads the configuration file probably using something like In_channel in OCaml (it does).
It appears that Uring is performing IO in both worker threads and directly.

The file paths are empty at the moment as I cannot find a clean way to trace openat submission requests into Uring without it sometimes going very wrong. I have tried quite a few methods (e.g. tracing do_filp_open) and at the moment I am tracing io_openat2, but this seems quite brittle, and often the filename is completely garbled, so I do not set it. If anyone has any ideas to trace Io_uring more reliably, I am all ears!

Shelter in the Forest [weekly-2025-04-21]

April 21, 2025
Patrick Ferris

I spent much of this week working on Shelter and things related to it. Some of that time was also spent on Forester.

Forester

April 21, 2025
Patrick Ferris

I added two important quality-of-life features to my frontend to Forester this week.

Bibtex Support

April 21, 2025
Patrick Ferris

I had previously mentioned adding support to Forester for Markdown. This week I added support for Bibtex too. From any *.bib file in your forest, Forester will now dutifully recognise it as a Bibtex file and convert, as best it can, all the entries into Reference trees.

I'm becoming quite convinced of this model at the moment. I'm using Forester's Code.t as a target representation. In fact, to ease the process, I really shouldn't spend all my time on my website, I have reused my Yaml.t -> Code.t and Markdown.t -> Code.t functions in the Bibtex parser.

To see it in action, you could have a look at the Build Systems à la Carte paper which is generated completely from Bibtex.

Full Heading Support

April 21, 2025
Patrick Ferris

The eagled-eyed viewer may have noticed that the table of contents for this page has more than one level. I finally caved and spent an evening rejigging my Cmarkit.Doc.t -> Tree code which was hacky and broken and is now less hacky and less broken.

In addition, headings support links and emphasis etc.

Lunch with Jon Sterling

April 21, 2025
Patrick Ferris

I had a delightful lunch with Jon Sterling discussing the future of Forester, the nature of the Web (old and new) and the success posting weekly updates for our colleagues. Thanks Jon.

Shelter Fixes

April 21, 2025
Patrick Ferris

I spent a good chunk of my week fixing bugs in Shelter with the aim to perhaps setup a VM somewhere and let people kick the tyres of what we've got so far.

The first bug is pretty annoying. At the moment, our filsystem backend is ZFS and we make heavy use of snapshots and cloning in order to provide time-travelling capabilities. Unfortunately, ZFS will take a snapshot before data has fully made it to disk (or whatever is the equivalent point it should reach in ZFS). Commands that generated lots of disk activity would be snapshotted in a half finished state and this would cause all sorts of problems. Thanks to Mark Elvers for the pointer to how OBuilder uses ZFS for the OCaml macOS builders which unmount datasets immediately therefore inducing a flush of sorts. Shelter now follows a similar model with all of the slow downs that create. Anil and I discussed some amalgamation of overlayfs, tmpfs and ZFS to alleviate some of this but for now that's a premature optimisation.

A small eDSL for Shelter

April 21, 2025
Patrick Ferris

Whilst testing Shelter, I ended up wanting a way to programmatically invoke the different run commands. This is similar to say a Dockerfile, but maybe with a little more expressivity.

This lead me to revisit the Build systems à la Carte paper and rediscover selective applicative functors.

I started playing around with a selective applicative interface to Shelter, this would allow you to express your dependencies statically but select them dynamically (as the paper says).

module D = Shl (Identity)

let shelterfile =
  let open D in
  let base_image = from "alpine" in
  let is_node_lst img = String.equal "v22.15.0" (stdout img) in
  let cmds base =
    let node_version = run "node --version" base in
    Select.if'
      (Select.map ~f:is_node_lst node_version)
      (run "node -e 'console.log('success!')")
      (run "node -e 'console.log('failure!')")
      base
  in
  with_session "node" (cmds base_image)

From this, we get a slightly more expressive way to describe images.

Geocaml TIFF Library

April 21, 2025
Patrick Ferris

I was pleasantly surprised to receive a pull request from Michael adding support to ocaml-tiff for reading TIFF files compressed using LZW. I was also surprised to hear the TIFF LZW is a little different to others.

In trying to get this PR merged, I moved the initialisation of the Eio eventloop to outside each individual test case. This one change then completely broke the entire test suite. After a period of debugging and help from Thomas Leonard the root cause was OCaml's OUnit2 library using process-level parallelism (via Unix.fork), sharing the ring between the parent and the child lead to the issues.

Part II Students

April 21, 2025
Patrick Ferris

As the new term begins, it signals that there are only just over two weeks for the final year undegrads at Cambridge to submit their dissertations.

The four students that I help supervise have been sending me drafts of their work (and reminder you can read about their projects) and I'm very impressed. I'm sure the next two weeks will be stressful, but I'm proud of what they have accomplished over the past academic year.

Shelter, Hazel and More! [weekly-2025-03-31]

March 31, 2025
Patrick Ferris

Last week I focused on Shelter -- our idea that shells should have the same ability as reproducible build tools like Nix or Docker. To this end I now have a fairly fleshed out prototype.

Shelter Prototype

March 31, 2025
Patrick Ferris

Shelter is a spin-off from the work Michael and I started with Shark. It takes the same ideas but applies them directly to a shell-like interface.

We're still in the middle of working all of this, but you can read more about it at Shelter.

Forester Hacking

March 31, 2025
Patrick Ferris

As you can probably tell, my website is still using Forester. I rebased my Markdown branch to include the new Atom syndication feature.

Alongside that I added support for arbitrary HTML injection into Forester via codeblocks in Markdown. This was actually very straightforward thanks to Markup and being able to re-parse Forester syntax in the middle of converting a Markdown document. The HTML for the shell in Shelter uses this feature.

If you are interested in taking this custom Forester for a spin, there's a branch on Github. In fact, nearly the only change beyond letting the core engine know about markdown files, is adding a new parser frontend.

Hazel

March 31, 2025
Patrick Ferris

For one of my Part II students, I've been prototyping a transpiler from OCaml to Hazel. This has gone pretty well and now supports type annotations as well as straight-forward implementation translation.

Consider the following OCaml map function.

let rec map f = function
  | [] -> []
  | x :: xs -> (f x) :: map f xs

The tool, hazel_of_ocaml can translate this to Hazel code, including making the polymorphism explicit.

let map : forall a -> forall b -> (a -> b) -> [a] -> [b] 
  = typfun a -> typfun b -> fun f -> fun x1 -> case x1
  | [] => []
  | x :: xs => f(x) :: map(f)(xs)
end in ?

You can copy and paste that codeblock into the hazel playground. But do note that it still needs some manual editing to add the type applications in directly (map@<a>@<b>). With the right amount of type-inferencing and scoping I actually think that you could place those type applications in yourself. This could make a nice Part II project I think.

Ppxlib

March 31, 2025
Patrick Ferris

I recently wrote about the painful experience of miragrating ppxlib to the 5.2 OCaml AST. This week, Nathan Rebours and I merged a PR to add the 5.3 AST! Ppxlib has been playing catch-up with the compiler and we decided it was best to try to catch up quickly and deal with the ecosystem fallout all at once rather than incrementally. With this new AST merged, ppx authors can now use the new Pexp_effect parsetree node. I'll write a little more about this in a separate post soon.

Teaching, teaching, teaching... [weekly-2025-02-10]

February 10, 2025

On paper, I don't have that many students. I teach four undergraduates (first year students at Pembroke College) Discrete Maths. I supervise three third year students for their final year project and another one I co-supervise with Michael Dales. However, I do end up spending at least two full days a week on teaching. Something I really enjoy and take seriously. The time it takes is also quite unpredictable; last week for instance all the third year students had their mid-project demonstrations (a five-minute presentation in front of their peers and a few professor-types). My first year students also found two slides particularly challenging to understand from their lectures and asked if I could help explain what was going on, so I produced some materials for that.

OCaml

Within the OCaml universe I spent a good bit of time trying to maintain a couple of different packages:

Mirage Crypto: this library provides cryptographic primitives for OCaml programs. Unfortunately, the maintainers removed direct support for Eio replacing it with a "Unix" alternative. This is not a fair swap as now Eio programs must make a dependency to Unix! Speaking to Anil, the best approach might be to make Eio programs handle this directly. I think this highlighted how fragmented open-source maintenance can be as no user of Eio seems to have bumped into this yet and the upstream maintainers did not communicate a rather large breaking change.
Ppxlib: In addition to the 5.2 AST bump (which is nearly ready to be merged), I queued up a 5.3 AST bump right behind it. I plan to write up a more detailed post about the challenges of maintaining this part of ppxlib.

Paris

I spent some time in Paris at the end of the week. I enjoyed visiting the Musée d'Orsay and in particular their collection of impressionist paintings. Here's my favourite from that visit by Camille Pissarro.

Woman in an Orchard (Spring Sunshine in the Meadow at Eragny)

Effective Geospatial Code in OCaml [part-ii-geocaml]

This project is co-supervised with Michael Dales.

Geospatial processing is critical in environmental science and is used in a range of environmental scenarios, both in areas of carbon accounting like digital Monitoring, Reporting and Verification in Reforestation Carbon Removal, and also biodiversity analysis like calculating worldwide extinction rates.
However there are problems with the current state of affairs, Python is one of the most popular data science libraries, and when doing geospatial data science with Python you use the GDAL geospatial data library. This workflow is not ideal [2]. For environmental scientists, interacting with the geospatial data in the imperative way that Python requires can be very difficult, and it is also not designed for the scale of computation needed for large geospatial projects. Furthermore, Python’s dynamic type system means that tricky type errors due to heterogeneity of data points can go unchecked. GDAL also only allows synchronous IO for loading GeoTIFF files, which can hurt performance.
I will write a non-blocking library for reading GeoTIFF files in OCaml (built on top of the WIP OCaml-TIFF library1 written by one of my supervisors, Patrick Ferris). I will then use OCaml to write an embedded Domain Specific Language (eDSL) which will be used to interact with geospatial data in a declarative way.

Weeknotes 2026-w38 › OCaml Maintenance › `rtree.0.3.0` release [rtree-0-3-0]

September 18, 2026
Patrick Ferris

I cut a release of ocaml-rtree that removes the repr dependency. See the relevant discuss post.

This was a feature that Michael had asked for when building laserver. I took the time to open a PR to laserver to use rtree.0.3.0 which does simplify the code.

I wanted to ensure the library was still working, so I grabbed some of the LAS data for Lough Neagh, provided by OpenDataNI for 2017. After converting it to cloud-optimised point cloud (COPC) data via pdal (yes, two different "cloud" meanings) I managed to view the south-east portion of Lough Neagh.

This is overlooking the Lough Neagh Discovery Centre and Croaghan Island.