Patrick Ferris https://patrick.sirref.org/index/ index /index/ Patrick Ferris false false false footer{ display: none } email - mastodon - github I'm a PhD student in the Energy and Environment Group at the Department of Computer Science, University of Cambridge. I'm particularly interested in: geocaml NI Forests Outreachy https://patrick.sirref.org/posts/ posts /posts/ Posts Occasionally I write posts, mostly about OCaml or something related. An RSS feed is available. Patrick Ferris 2026 3 9 https://patrick.sirref.org/vibecoding-etiquette/ vibecoding-etiquette /vibecoding-etiquette/ Vibecoding Etiquette I am yet to jump head-first into using LLM-based tools like Claude Code or even ChatGPT to help with my programming in any serious way. After using some "free"-tier tools to try to better understand some tricky eBPF problems and to make sense of the semantics of POSIX shells, I was left unimpressed and ultimately felt my time was wasted. I feel unwilling to send my money to the corporations behind these tools. This is for a variety of reasons including the deeply worrying relationship with the "opertational planning" vs. "operational decision-making" given the leaders of the US DoW environmental concerns, exploitation of human labour and a fast-and-loose approach to copyright laws. All of which leaves the entire industry very unattractive to me. Everybody can draw their line where they want, and I am, in good faith, trying to find where mine should be. Unfortunately, my own decision to not use these tools is not completely in my control. What do I do about pull requests to open-source libraries I maintain that include agentic code? I am, after all, participating in the idea of open-source and inclusive collaboration. Agentic code also comes in many flavours, from blatant LLM-generated pull requests to heavily edited, human-in-the-loop contributions. An immediate first step for me is to develop some policy for contributions to projects I maintain regarding code where LLMs have had a hand. This can help set expectations for contributors. Eio's "AI-generated Code" subsection is succinct and prohibits contributions that solely use AI. It obfuscates how you think. Purely AI-generated code tells us little about how you think and the problems you might be having. This makes it harder to provide good feedback on PRs and issues. It is often more work to review. Particularly for the OCaml ecosystem and libraries like Eio, it seems that these tools are not very good and generate a lot of believable code that is in actual fact completely wrong. PR comments and the code submitted with them can say completely different things. It is a grey area for licensing. Models like ChatGPT have been trained on lots of code with different licenses and has been known to simply copy code as an answer to a prompt. We would like to avoid this headache as best we can. Additionally, I plead to other developers to own their use of these tools. Please, make Claude a co-author of the commits where Claude has generated any code at the very least. This is the bare minimum. I have strong feelings about the etiquette of sending AI summarised information or LLM-generated code to other people, but before any of that, the decent thing is to remove any uncertainty in the receiving party's mind of how the information they are receiving came to be. In this way, I appreciate the OCaml compiler's AI notice which includes: If a significant portion of your code, PR description, review comments or messages has been AI-generated, this must be disclosed, stating which tool was used and for what. Reviewing AI-produced code can require a different approach from human-written code. You should have an acknowledgement anyway even if another human contributor did some of the work. Patrick Ferris 2025 10 24 https://patrick.sirref.org/icfp-2025/ icfp-2025 /icfp-2025/ ICFP 2025 Two weeks ago I was fortunate enough to attend the International Conference on Functional Programming in Singapore. My first time in Asia and my second time at the conference, what follows are some thoughts and presentations I enjoyed whilst I was there. Ryan Gibb for his camaraderie and sage advice: "the best track is hallway track". Not to mention all of his photos too. Thanks Ryan! From right to left, top to bottom: Anil Madhavapeddy, Sadiq Jaffer, Patrick Ferris, Ryan Gibb and Roy Ang. Patrick Ferris 2025 10 24 Random Hacking Below are just some of the pieces of random hacking I got up to whilst in Singapore. Patrick Ferris https://patrick.sirref.org/oxcaml-and-uring/ oxcaml-and-uring /oxcaml-and-uring/ OxCaml and Uring Jane Street were a big presence at ICFP 2025, carting along with them a shiny new OCaml compiler: OxCaml. If have been playing around with OxCaml recently but nothing outside toplevels in Javascript. Until now! After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change: module Heap = struct type ptr = int32# ... end type cqe = user_data_id : Heap.ptr; res : int32#; unboxed values You can see how this impacted the library! to realise OxCaml gives you this kind of control. However, I am worried about the ergonomics of manipulating values like int32# and the temptation to case it into an int Patrick Ferris 2025 10 24 Bibtex Parser On the flight over to Singapore, I started to write a post for this website. Some readers may remember that I use forester after a preprocessing step via graft to build this site. Part of that preprocessing consumes bibtex files and outputs forester trees. For the initial graft release I used the excellent Talaria Bibtex library. Unfortunately, it presented a pretty confusing API that, whilst nicely typed, felt a little cumbersome to use. Additionally, it was too specific in what it would accept as valid Bibtex. I wanted something that was a little more And so I built bib. This was my first real attempt at using the Bytesrw library. I was happy with the results, although I did have to copy a substantial amount of code from other bytesrw-compatible libraries. I'd be happy to shift to this bytes-first world if it wasn't for the ever-present-in-my-mind issue of relocatable values in the OCaml heap! Patrick Ferris 2025 10 24 Geocaml Libraries There has been some excellent outreachy work happening on the geocaml/ocaml-tiff library which is a potential Outreachy internship project for December 2025. More on this in a future post. counterpart to geotessera, a library for working with Tessera. The library is not ready for people to use directly, but I was working on various constituent parts including: Modern OCaml bindings to PROJ Further work on a WKT library for OCaml Coming soon... ocaml-geotessera Patrick Ferris 2025 10 24 Talks Patrick Ferris 2025 10 24 Hazel of OCaml at TyDe The type-driven development NUS School of Computing. I find the workshop name to be a little misleading, but perhaps this comes from my engineering background. The talks focused a lot on type theory, presenting the formal mathematics of interesting type systems with less focus on why it might be useful to have such an exotic type system... but then again, that was the nature of the entire conference. I presented a talk on hazel_of_ocaml. The slides are here, along with the extended abstract. My talk ended up focusing a lot on the pedagogical benefits of having a type system that helped explain the most important thing a type system does... catch type errors! This work represents a small piece of work inside Max Carroll's excellent part II project, which he presented at the HATRA workshop: Decomposable Type Highlighting for Bidirectional Type and Cast Systems. Patrick Ferris 2025 10 24 Scientific Programming at PROPL I think it is important to be honest whenever you get a talk accepted that you otherwise feel should not have been. My talk "about scientific programming what we talk about when we talk" is one such talk. The slides are here. The talk itself was nothing to write home about, but I think there were a few nice ideas that came out of it and the conversations I had with fellow PROPL attenders. Patrick Ferris 2025 10 24 TMF Data Discrepancies Redux On the eve of my talk, I decided to re-run some analysis I did a few years ago on the tropical moist forest dataset, knowing that since then a few more years had become available. After fighting with some bit-rot in the Google Earth Engine script I had used, the new results were not very uplifting. Land Use Class Percent Change Undisturbed  -2.9 Degraded  -5.5 Deforested  8.6 Regrowth  0.5 Water  0.0 Other -0.6 The data in the table corresponds to the land use class values in Indonesia in the year 2008. The analysis computed the difference across each of the classes between the dataset released in 2021 and the dataset released in 2024. Most notably, the amount of deforested land has gone up by 8.6 percent! As a somewhat tangential, though related, aside, I had the pleasure of talking to Sam Westrick, Seong-Heon Jung and Sundara Vishnu Satish of the ParCour research group. They were telling me about MaPLe, a programming language from the MLs built for parallelism. We discussed some ideas of using real-world, geospatial algorithms as benchmarks for MaPLe. Michael, if you got this far remind me to chat to you about this! Patrick Ferris 2025 10 24 Shelter Reactions At the end of my talk, I briefly mentioned shelter. By far, this was the topic I got the most engagement on post-talk. During the conference, this was also true when people asked what I was working on. The tag line "a shell with undo" seemed to capture the attention of most people. I'm feeling a renewed sense of interest in my own work thanks to that, even if the "research" contributions are not very apparent. However, I also had a good discussion with KC about the underlying MRDT in Shelter and how we might think about collaborating in the future on MRDT-related work. Related to that, I managed to spend the evening in Chinatown with Roly Perera. Roly's work has focused on Fluid, a transparency-based programming language that allows authors to reveal their computation to their user. Excitingly, Roly presented a new text-based part of the language at PROPL. Our work is in the same universe, loosely orbiting some notion of reproducibility and explainability, but quite different in engineering work and how users might actually interact with our systems. I thoroughly enjoyed talking about solutions or research ideas that preserve existing workflows. Roly was great at pulling that out of the current work I have been doing. Patrick Ferris 2025 10 24 Other Talks and Highlights I was a part of, often in a very small way, some other talks that were presented at ICFP including Functional Networking for Docker, I encourage you to read Anil's notes on the topic. It is also worth mentioning Anil's post-posix talk Michael gave an excellent, very live, talk about Yirgacheffe at PROPL which I thoroughly enjoyed. I'm also excited about his recent developments which I'm sure you can clue into over on his blog. Below is a non-exhaustive list of talks I enjoyed whilst at the conference. Some relevant, most of them not! By far, one of the best talks was Polynomial-time Program Equivalence for Machine Knitting. I think the talk really nailed the "here's the idea, the details are in the paper"-presenting style as they described an "...algorithm that canonicalizes the algebraic representations of the topological semantics of machine knitting programs". I attended most of the OxCaml tutorial presented by the incredibly friendly Gavin Gray. Ryan's talk Spatial Programming for Environmental Monitoring at PROPL was excellent. There's plenty more to talk about, and I'm sure my future posts will reference other parts of my experience at ICFP 2025. Thanks for reading, and if anybody is interested in any aspects of this do reach out! Patrick Ferris 2025 8 7 https://patrick.sirref.org/irmin-retro/ irmin-retro /irmin-retro/ Irmin Retrospective Irmin is an OCaml library for building branchable and mergeable data stores. The data is mergeable in the sense of mergeable replicated data types. I have been using Irmin for over five years to build different kinds of interesting data stores including: A simple markdown-based note-taking web application. A content-addressed database. Mentoring an intern who worked on Irmin in the browser. Most recently, an Irmin-backed shell session manager. I was asked to provide some feedback recently on Irmin, so I thought writing a little retrospective here would be a good way to do that. The remit for the retrospective was about improving Irmin, so the content is focussed on pain points and areas of improvement. An in-memory Irmin store module Store = Irmin_mem.KV.Make Irmin.Contents.String Patrick Ferris 2025 8 7 What is Irmin? Irmin, at its simplest, is a key-value database. Users associate keys with values and can query and update these bindings. Additionally, this database supports versioning. This means independent snapshots of the database can coexist and users can switch between them and update them without fear of interfering with other versions. Different versions of the database can be combined by merging. When you set up your instance of an Irmin database, you also provide it with a merge function. # #show_type Irmin.Merge.f;; type nonrec 'a f = old:'a Irmin.Merge.promise -> 'a -> 'a -> 'a, Irmin.Merge.conflict result Lwt.t Patrick Ferris 2025 8 7 Design and API Irmin's API is very git-inspired. There is a large overlap of shared vocabulary and concepts: repositories, branches, commits, heads etc. Probably the most confusing aspect of this is the notion of a Store. When I was describing Irmin above, I used the term database to help distinguish between some of these concepts. In Irmin's documentation, it is used for multiple Creating a Store" stores refer to the entire database, whereas in the Irmin API docs we have that: There are two kinds of store in Irmin: the ones based on persistent named branches and the ones based temporary detached heads. For Irmin library users, a Store.t can be thought of as a checkout of the database at a particular revision. This may be from a branch or a specific commit. # Store.of_branch;; - : Store.repo -> string -> Store.t Lwt.t = <fun> # Store.of_commit;; - : Store.commit -> Store.t Lwt.t = <fun> The overloading of the term is confusing. I think it leaves users unsure about what other people might mean when they say "store". Being careful with these terms, and the contexts in which they are used, would help avoid this confusion. Patrick Ferris 2025 8 7 Module and Functor Soup Undoubtedly for a majority of use-cases and users, Irmin is over-functorised. Nearly every module requires that you must apply some functor to access any useful code. In general, a user will have to interact with the Schema module when describing the types they want to instantiate their store with. To counteract this, Irmin has plenty of KV modules that provide a Make functor that only requires a user to provide a suitable content module # #show_module Irmin_mem.KV;; module KV : sig type endpoint = unit type metadata = unit type hash type info = Store.info type contents_key = 'h type 'h node_key = 'h type 'h commit_key = 'h module]]> sig ... end end I think it is fair to say the documentation is hard to follow as it is a module and functor soup. For example, looking at irmin.3.11.0, starting at the toplevel documentation page our path to finding this paricular module and functor is as follows: We jump into Irmin_mem from the nicely written landing page. We scroll down to find the KV module. We now make sense of the module's signature by following the KV_maker link. We see a at the end of the module, and we navigate through that. Finally we have come to our journey's end, and find the schema module with type constraints like type Branch.t = string. There is a big assumption there that you know what the Schema module is telling you and how it relates to your "store". There is a counter-argument here, in that Irmin is incredibly flexible in terms of what you can use as types for your keys, branches, hashes etc. I was quite easily able to produce for example. I think there is a middle ground. For Irmin to be usable it needs to reduce the complexity of the API. The complexity is due to, in large part, over-functorisation. If the functors cannot be removed then perhaps better documentation or more introductory libraries like irmin-containers would be helpful. Perhaps a standalone library that acts as an interface to Irmin stores would be helpful. I find myself time and again implementing something like that. Patrick Ferris 2025 8 7 Backends Galore Irmin has plenty of backends including a git-compatible one, a MirageOS backend, an in-browser IndexedDB backend and even a FoundationDB backend. Irmin needs to have fewer, better tested backends. Of course, some of these are likely driven by funding and use-case specific details. However, I think by focusing on a few key backends and striving for solid performance but also strong consitency guarantees, Irmin would become a more viable candidate for potential users. I also believe the browser backend plays a key part to making this work in order to facilate local-first applications. Patrick Ferris 2025 8 7 Syncing Remote Stores The synchronisation mechanisms in Irmin are powerful, but often the API is very confusing. For example, with the git-compatible Unix backend, nested deep in the documentation is the function required to create a remote endpoint. Once they have found this function, using it is not easy as it requires them to learn about the Mimic.ctx, which is an abstraction of the networking stack! Patrick Ferris 2025 8 7 Do Fewer Things and Do Them Well Irmin is simultaneously a database that supports: content-addressing, merging, key-value lookup, branches, transactions, complete OS portability etc. I think, in short, it tries to do too many things and comes up short on some of them in ways that really matter. When building the content-addressed database I needed strong guarantees about some atomic actions to perform on the underlying store. For example, setting the value and accessing the commit associated with it which was not possible with the API at that time in an atomic way. This kind of missing functionality is probably only understood after users stumble across it, but it also makes the library feel less focused and ready for production use-cases. Patrick Ferris 2025 8 7 Documentation It will probably come as no surprise that one of the main limitations of using Irmin is the lack of documentation and tutorials. This is a particular pain point as the API is not very straightforward. A while back, I started an Irmin book intending to help document the kinds of things real-world users would need in order to use Irmin in earnest. For example: ways to deal with type migrations or primers on runtime types. I still believe this sort of work would be invaluable, but there is a large cost to completing it, and it is not clear if there is enough interest in the project to warrant such an effort. The tutorials on the irmin website are still good starting points for most new users. But they quickly lack the depth for real-world scenarios. Patrick Ferris 2025 8 7 Feature Wishlist What follows are additional ideas for improving Irmin. Patrick Ferris 2025 8 7 Heterogeneous Stores missing feature for lots of first-time users of the library: you can only store values of a single type. This leads to a few common workarounds: Storing a serealised version of your values, essentially escaping the type-system and implementing a form of dynamic typing. Growing your value to hold lots of different types via some large variant type. Both of these options are feasible and have been used in practice. However, they are workarounds. A long time ago CraigFe created an RFC for heterogeneous stores, which I think about a lot. The main idea is to augment Irmin to be GADTs that carry type information about the kinds of values they access heterogeneuous variants of other data structures API would make Irmin more appealing as a library for persistent data storage. Patrick Ferris 2025 8 7 Real-world Retrospectives This echos some thoughts from the Documentation section. There are some very real-world Irmin Tezos blockchain were using Irmin which uses the irmin-pack backend. Does this make irmin-pack the best tested backend and perhaps should be the default backend for new users? Or is it hyper-specific to the Tezos use case? There are blogs on tarides.com about the irmin-pack backend, but they might be outdated and should not be the first place to find advice on which Irmin backend to use. Patrick Ferris 2025 8 7 Active Development and Engagement As far as I know, Irmin was at the forefront of technologies that have come to be described as local-first. There is a growing interest Ink & Switch's essay on the matter. And it would be great to see more research via Irmin at things like the lofi unconference and the PLF workshop at SPLASH 2023! Thank you for reading! And thank you to all the Irmin contributors. I hope this might be useful in the future for building the next-generation Irmin! 2025 7 23 https://patrick.sirref.org/slipshow-x-xocaml/ slipshow-x-xocaml /slipshow-x-xocaml/ Slipshow x x-ocaml A short, explanatory post about combining two very fun pieces of work in OCaml. Paul-Elliot has been building Slipshow for some time now where slides are slips and your presentations run vertically. More recently, Arthur has built x-ocaml, a web component library for executable OCaml cells embedded into OCaml. Using xocmd, a small tool I built for translating markdown codeblocks to x-ocaml components, your Slipshow's can now be executable! learn-effects.html]]> Take a look at an example! (or the source markdown). I really like this light-weight approach to building interactive presentations Patrick Ferris 2025 7 18 https://patrick.sirref.org/ocaml-quarterly-q2/ ocaml-quarterly-q2 /ocaml-quarterly-q2/ Quarterly OCaml Q2 Thanks to Tarides sponsorship, I get to work on open-source OCaml. This quarterly is a companion to my weeklies, summarising the last three months of development, peppered with ideas and thoughts about OCaml, its community and its future. Patrick Ferris 2025 7 18 What I wanted to work on? There were two main things I hoped to continue working on: ppxlib and outreachy. These are projects that I was previously working on, and in the case of Outreachy I have now been involved for many years. In addition to this, all of my Part II projects this year used OCaml in some regard. In general, I want to see more adoption of OCaml. Over the years this has taken many forms including my initial work on developing workflows for OCaml that just turned five years old. This directly fed into the rebranding and rethinking of ocaml.org itself. Patrick Ferris 2025 7 18 What I worked on? Patrick Ferris 2025 7 18 Ppxlib Ppxlib is the de facto standard library for building OCaml preprocessors. At the time of writing, opam list --depends-on=ppxlib informs me that there are 267 reverse dependencies. Janestreet is a heavy user of ppxes and has authored many. One of the main accomplishments this quarter was bumping the internal AST to 5.2. This allows ppx authors to use new OCaml language features in their ppxes. In bumping the AST, we knowingly broke compatability for pretty much every single reverse dependency. As best we can, we have been sending patches to ppx libraries and helping users migrate to the latest ppxlib. There is an interesting dicussion to had about the nature of open-source, and its interaction with industrial monorepos and community tended package repositories. Package ecosystems thrive whenever there is a dedicated community creating, maintaining and publishing packages. The idea is that the published world should be healthy. The publishing medium can act as natural limiting PRs to the opam-repository have come to notice, reacts poorly to changes coming from internally consistent monorepos where introducing breaking changes is easily fixed by applying patches there and then. Whatsmore, OCaml is often stated as an incredibly safe language to perform large refactorings thanks to its type system. Ppxlib sits awkwardly in the space of possible breaking changes. Tied to OCaml's parsetree, impacts of changes there ripple down to ppxlib. The compiler itself can remain internally consistent, and is protected as it need only parse source code. Ppxlib, on the other hand, exposes the parsetree to users and thus any changes to the parsetree will likely be felt by ppx authors. Since I started working on ppxlib, it feels as though the number of syntax changes support for these, maintainers of ppxlib will forever be stuck doing busy work! Thankfully, Nathan has thoughts on how to improve this. There are a slew of other features I have added to ppxlib including: Support for deriving from classes. Support for deriving from module bindings and signatures. Fixing compiler and ppxlib dummy locations. Bumping to 5.3. Migrations for 5.4. Patrick Ferris 2025 7 18 Outreachy Our two projects this year, one on claudius and one on dune, are going extremely well. At the time of writing, we just had a mid-internship call to catch up, and I was blown away by the progress each intern has made. Unfortunately, Outreachy is struggling with funding and the next round is perhaps not going to happen. This is a real shame and I am hoping that it will not be the case. Outreachy has been a wonderful source of new, committed OCaml developers. If you haven't already, do peruse the webpage on OCaml.org to see past internships or watch the demo day presentations. For the mentors involved, We are always looking for new mentors and project ideas, please do reach out to me if you are interested. Patrick Ferris 2025 7 18 Hazel Hazel is a research programming language with typed holes written completely in OCaml reason dialect Relating this back to the original intent of my work, to improve OCaml adoption, I believe this also means keeping existing users happy. In the last quarter I developed a compiler from OCaml to Hazel. Whilst new features are still being added to Hazel, I hope this could serve as a tool to help develop test-suites and standard library functions using existing Typed Debugging for Hazel Hazel programs to great effect. OCaml should continue to be a world-class programming language for building other programming languages. I hope to upstream some of this work to Hazel and provide some low effort maintenance to help keep their compiler in good shape. Patrick Ferris 2025 7 18 Systems Programming in OCaml In a cross-over with my own research, I have been developing many tools related to systems programming in OCaml including: An eBPF-based open syscall tracing tool. A library in OCaml for spawning void processes. A shell session manager that uses Irmin to manage sessions. It is nice to see a new push to finally land the direct-style Irmin PR! This work is means to develop the underlying libraries that support it. For example, I have opened a few PRs to Eio to add new "fork actions" to the spawn API. I also investigated the feasibility of changing the underlying mechanisms in Eio to use Picos. In the future, I think this could be important avoid further splitting the OCaml ecosystem. Patrick Ferris 2025 7 18 OxCaml I dabbled a little with OxCaml and build try-oxcaml to let people take it for a spin without having to perform opam repository gymnastics. It turned into a lot of work to track down some pretty inane bugs This unblocked a few of my colleagues to get OxCaml working on tools like odoc_notebooks and x-ocaml. Patrick Ferris 2025 7 18 Forester A good proportion of my work this quarter has been focused on how to present the very work that I am doing. Jon Sterling has been developing a tool for scientific thought called Forester which seemed like a possible candidate for writing and sharing my work. Porting my existing blog posts and website content from markdown to Forester's LaTeX-inspired syntax didn't seem like an option. In particular, many of my posts ocaml-mdx This lead to the development of Graft: a preprocessor for Forester forests, converting markdown and bibtex to trees. Patrick Ferris 2025 7 18 What's next? So, I worked on most of what I wanted to work on and then some! Going forward I hope to keep maintaining ppxlib in some capacity and coordinating the OCaml community's Outreachy efforts. research makes heavy use of Irmin, and I would be interested to help with that too. More recently, a new library has been released for working with numerical data in OCaml called Raven: I am interested to use this library in Geocaml, a suite of geospatial tools written in OCaml that I maintain. I feel conflicted about the OxCaml efforts. I admit, I am not fully aware of the full benefits of the features, but I worry about some proliferation of modes that make the type system in OCaml unbearable to use. On top of this, with my ppxlib hat on, I worry about the impact of changing the compiler so frequently, placing strain on an already small community. That being said, by releasing OxCaml separately I do believe running it as an experimental set of packages will help understand the tool better. But I would not be building anything I intend to maintain or research with right now as that ecosystem is far too volatile. Finally, over the past two years I have done a lot of teaching. From Part II projects to supervisions intend to reduce my in-person teaching to focus on my research. However, I am interested in producing more materials for learning, maybe some of this in OCaml, perhaps something similar to Beautiful Racket. I have tried in the past to do these sorts of things, for example The Irmin Book. Patrick Ferris 2025 7 18 Thank you Thank you for reading this wrap up! And thank you again to Tarides for letting me work so freely on things that I think are good for the OCaml community. Patrick Ferris 2025 5 19 https://patrick.sirref.org/open-trace/ open-trace /open-trace/ Opentrace https://tangled.sh/@patrick.sirref.org/opentrace Thanks to Koonwen's excellent libbpf bindings in OCaml, I have been building a little tool called opentrace to make it easier to track an executable's inputs and outputs. This work was inspired my Michael's self-proclaimed "gross hack": pyshark. Whilst pyshark achieves its goals by injecting code into commonly used python objects and methods, opentrace uses eBPF opentrace can remain programming language agnostic. However, less information is none about the user's intent compared to something like pyshark. Patrick Ferris 2025 5 19 Monitoring the System opentrace has an all command that will trace the entire system. The --flags=O_WRONLY argument filters the events where the O_WRONLY flag was set in the call to open. comm -wrapped is an artefact of using Nix. Patrick Ferris 2025 5 19 Tracing an Executable The primary use case for this tool is to inspect what files your program might be reading and writing. The "flags" argument can specify a small boolean formula for checking the open flags of a particular event with | & ~ Parentheses can be used for precedence. Patrick Ferris 2025 5 19 Spawning Subprocesses One feature opentrace opentrace is primarily an eBPF program that is loaded into the kernel and communicates with an OCaml program. Events are communicated via a ring buffer and most of the post-processing happens in OCaml. To capture subprocesses, opentrace This gives opentrace a new identifier to track, namely the cgroup. So consider the following program. let = Eio_posix.run @@ fun env -> Eio.Path. save ~create: `Or_truncate 0o664 env#fs / "hello.txt" "hello" ; Eio.Process.run env#process_mgr "/bin/bash"; "-c"; "echo 'heya' > heya.txt" It first creates a file using direct calls to functions like openat. Then it spawns a process which creates a new file called heya.txt. This happens in a separate process. However, with the --cgroups flag we can capture both interactions with the operating system. Patrick Ferris 2025 5 19 Eio's Process API I have used the Eio_unix fork action process API to be able to extend what happens in the child process. Loading most eBPF programs into the kernel requires special privileges hence the need for sudo. When a user requests for a particular program to be executed and traced, opentrace spawns a process via the Eio Process API. Opentrace defines a few new so-called "fork actions", little fragments of C code that are run after the call to fork clone Most likely this ends with a call to execve, but other calls are possible for example setuid allowing opentrace to change the user of the child process so it does not run as root into that group. Patrick Ferris 2025 5 19 Limitations: Io_uring Whilst testing opentrace against some of the tools I use nearly daily, I noticed some events were being missed. I tried tracing forester, and only the initial read of forest.toml was logged. It dawned on me that the reason for this was that forester eio io_uring to perform most of the IO. Most attempts to open files were bypassing the open system calls, and instead they were being performed by the kernel after reading a submission request for an openat2-style call! This is not news to seasoned, Linux systems programmers. Io_uring bypasses SECCOMP filters for exactly the same reasons. It is interesting to note two things here: We can tell that forester reads the configuration file probably using something like In_channel it does It appears that Uring is performing IO in both worker threads and directly. The file paths are empty at the moment as I cannot find a clean way to trace openat submission requests into Uring without it sometimes going very wrong. do_filp_open io_openat2, but this seems quite brittle, and often the filename is completely garbled, so I do not set it. If anyone has any ideas to trace Io_uring more reliably, I am all ears! Patrick Ferris 2025 5 9 https://patrick.sirref.org/try-oxcaml/ try-oxcaml /try-oxcaml/ Try OxCaml https://patrick.sirref.org/oxcaml This week, I have been trying out Janestreet's Oxidised OCaml ICFP paper This adds a system of modes to OCaml for expressing things like locality. ...we’re introducing a system of modes, which track properties like the locality and uniqueness of OCaml values. Modes allow the compiler to emit better, lower-allocation code, empower users to write safer APIs, and with the advent of multicore, statically guarantee data race freedom—all in a lightweight way that only affects those in need. For example, you might say: let is_empty s : string @@ local = String.length s = 0 To get a feel for how this changes the language, you can follow their instructions on their custom opam-repository. Alternatively, you can give it a go in your browser! This is a full toplevel running with Base Happy OxCamling! Patrick Ferris 2025 5 2 https://patrick.sirref.org/hazel-of-ocaml/ hazel-of-ocaml /hazel-of-ocaml/ A Transpiler from OCaml to Hazel Over the past few months, I have been piecing together a transpiler from Hazel to OCaml. This is, in part, to help one of my third-year undergraduate students who is working on type error debugging in Hazel. Patrick Ferris 2025 5 2 Typed Holes Hazel is a functional programming language with typed holes. Holes are pieces of your program that have not yet been filled in. Holes can appear anywhere in your program both as expression or types. Hazel can still evaluate your program in the presence of holes. To get a flavour of Hazel, take a regular map function for lists. fun xs -> case xs | [] => [] | x :: xs => f (x) :: map(f)(xs) end in map(fun x -> ?)([1, 2, 3])]]> ? familiar with OCaml types ? list Hazel supports local type inference but nothing involving unification variables. For example, a simple add_one function in Hazel fun x -> x + 1 ? -> Int. Patrick Ferris 2025 5 2 From OCaml to Hazel The ability to transpile OCaml programs to Hazel programs is motivated by one simple thought: there are more OCaml programs than there are Hazel programs. This could help bootstrap projects by alleviating the need to rewrite Patrick Ferris 2025 5 2 A Transformation of Syntax Hazel markets itself as an "Elm/ML-like functional programming language". From the previous example of map, it should be apparent just how close to OCaml the language is. It turns out that a majority of the transpiler is a transformation of syntax. Take a simple ADT for an arithmetic programming language. type expr = | Float of float | Add of expr * expr | Sub of expr * expr | Mul of expr * expr | Div of expr * expr And when we run hazel_of_ocaml over this OCaml type declaration. Not much has changed expect some syntax. Hazel does not have a notion of top-level expression so hazel_of_ocaml wraps the program into one set of value bindings. For the most part, Hazel acts as a subset of the pure, functional part of OCaml. At the time of writing, this subset is fairly limited If we try out the same map function but written in OCaml and transpiled to Hazel we get. let rec map f = function | -> | x :: xs -> f x :: map f xs Which becomes the following hazel program. fun x1 -> case x1 | [] => [] | x :: xs => f(x) :: map(f)(xs) end in ?]]> We could have a field day discussing the syntax of OCaml and Hazel taking a look at how to handle polymorphism in Hazel. Patrick Ferris 2025 5 2 Explicit Polymorphism Hazel has explicit polymorphism. So far, we have not seen it as we have let the types have holes in them. The map function in OCaml has the following type. val map : 'a -> 'b -> 'a list -> 'b list Craig's excellent blogpost on the matter ... type variables in signatures are implicitly universally-quantified So in reality, we have that map has the following type. b) -> a list -> b list]]> In Hazel, we have to explicitly type our map function to be polymorphic. Not only does this mean the type annotation requires universally quantified type variables, but we must also perform type application wherever we choose to apply the map forall b -> (a -> b) -> [a] -> [b] = typfun a -> typfun b -> fun f -> fun xs -> case xs | [] => [] | x :: xs => f (x) :: map@@(f)(xs) end in map@@(fun x -> ?)([1, 2, 3])]]> forall introduces a universally quantified type variable into our type annotation, and typfun requires @<T> where T is some type. This allows hazel to quite easily support higher rank polymorphism, but we will not worry too much about that. Patrick Ferris 2025 5 2 Propagating OCaml Types into Hazel Most often, OCaml users interact with prenex at the front of the type. OCaml does support quantifiers inside certain types like records. What this means for the transpiler is that we can reuse OCaml's type inference to safely instantiate the correct type annotations and type applications in Hazel! To do this, hazel_of_ocaml uses Merlin to inspect the type of the function in either a value binding or at the point of a function application. Take a simple, polymorphic length function. let rec length = function | -> 0 | _ :: xs -> 1 + length xs let int_len = length 1; 2; 3 let str_len = length "only"; "two" When we run this through hazel_of_ocaml with the -type flag we get. [a] -> Int = typfun a -> fun x1 -> case x1 | [] => 0 | _ :: xs => 1 + length@(xs) end in let int_len : Int = length@(1 :: 2 :: [3]) in let str_len : Int = length@("only" :: ["two"]) in ?]]> hazel_of_ocaml has correctly instantiated the type for length inside the recursive function and then in each case with the integer list and the string list. Patrick Ferris 2025 5 2 A Corpus of Hazel Programs The impetus for this work was to derive a corpus of ill-typed Hazel programs. Luckily, such a corpus exists for OCaml! Seidel et al. created a corpus of OCaml programs from their undergraduate students at UC San Diego. Some of these programs have been transpiled to Hazel. Patrick Ferris 2025 5 2 Future Work Hazel is a fun, research programming language. Potential third-year students may find it interesting to take this work further. For example, how would this look in terms of a module system? From a purely engineering perspective, plenty of work would be needed to convert a multi-library OCaml project to Hazel cmi Another line of research would be to have Hazel target one of the intermediate representations in OCaml which would give Hazel a fully functioning compiler to "native" code? 2025 2 18 https://patrick.sirref.org/ppxlib-5-2/ ppxlib-5-2 /ppxlib-5-2/ Bumping Ppxlib's AST to 5.2 Ppxlib is a libary for building OCaml preprocessors. Users can generate OCaml code from At the core of ppxlib is the OCaml parsetree; a data structure in the compiler that represents OCaml source code. Ppxlib ppxlib some In ppxlib.0.36.0, the internal AST was bumped from the AST of OCaml 4.14.0, to the AST of OCaml 5.2.0. This post discusses some of the internal mechanisms that allows ppxlib to support multiple compilers and how this bump broke a lot of backwards compatibility. 2025 2 18 Manual Migrations in Ppxlib It is possible to perform migrations between parsetrees manually using the ppxlib.ast package. open Ppxlib open Ast_builder.Make struct let loc = Location.none end let pp_expr = Format.printf " " Pp_ast. expression ?config:None let loc = Location.none let str txt = txt; loc module To_408 = Ppxlib_ast.Convert Ppxlib_ast.Js Ppxlib_ast__.Versions.OCaml_408 The Ppxlib_ast.Js module is ppxlib ppxlib.0.36.0 provided by the Convert functor to copy parts of the AST from one version to another. Usually there is very little difference between the versioned parsetrees. let func arg body = pexp_fun Nolabel None ppat_var str arg body let add_expr = func "x" func "y" eapply evar "Int.add" evar "x"; evar "y" The above excerpt of OCaml code produces an expression in the language. More accurately, a function that has two arguments. If we print the simplified AST we can see that the arguments are stored together in a list. # pp_expr add_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" ; pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "add" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit = This is a relatively new feature of the parsetree, if we migrate the expression down to 4.08 we see it is represented differently. The output is a little too verbose to be readable, and there's no good way to show the AST, so you will just have to take my word. let expr_408 = To_408.copy_expression add_expr 2025 2 18 Function Arity in the OCaml Parsetree OCaml 5.2 introduced a new way to represent functions. Before 5.2, all functions were represented with single arity. Functions with multiple arguments would return functions as their body. For example: let add x y = x + y A two-argument add function would be represented in the parsetree as fun x -> fun y -> x + y . Now, since OCaml 5.2, functions can be expressed with their syntactic arity intact. The function add would look something like . Both arguments are stored as a list. The constructor Pexp_function was not a new addition to the parsetree. It used to represent pattern-matching expressions like function A -> 1. This can now be expressed as a function body of Pfunction_cases. Whilst these additions and modifications to the parsetree are arguably a better design, it has caused chaos in terms of ppxlib's reverse dependencies! 2025 2 18 Coalescing Function Arguments In ppxlib 0.36.0, functions from Ast_builder will help ppx authors produce maximum arity functions. However, functions from Ast_helper will not do this. let ast_helper_func arg body = Ast_helper.Exp.fun_ Nolabel None ppat_var str arg body let ast_helper_expr = ast_helper_func "x" ast_helper_func "y" eapply evar "Int.equal" evar "x"; evar "y" Here we recreate our helper function to create function expressions using Ast_helper.Exp.fun_ instead. We can now show the difference between the expr expression and the ast_helper_expr expression. # pp_expr add_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" ; pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "add" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit = Above, we see a function expression with arity two. Below, we see nested function expressions each with arity one. # pp_expr ast_helper_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" , None , Pfunction_body Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "equal" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit = 2025 2 18 Ramifications for Ppxlib Users Many, many, many ppxes broke after migrating the ppxlib AST to 5.2. Mostly, this is due to pattern-matching. Downstream users of ppxlib frequently pattern-match directly against the parsetree. However, nodes such as Pexp_fun no longer exist and Pexp_function has completely different constructor arguments! Pexp_fun x function Here is a non-exhaustive list of some of the ppxes that have been patched. It may useful for other ppx authors who wish to upgrade to ppxlib.0.36.0: https://github.com/patricoferris/ppx_deriving_yaml/pull/58 https://github.com/mirage/ocaml-rpc/pull/181 https://gitlab.com/o-labs/ppx_deriving_jsoo/-/merge_requests/1 https://github.com/ocaml-community/sedlex/pull/160 https://github.com/ocaml-sys/config.ml/pull/22 https://github.com/vogler/ppx_distr_guards/pull/2 https://github.com/ocaml-gospel/gospel/pull/424 https://github.com/teamwalnut/graphql-ppx/pull/299 https://github.com/ocsigen/lwt/pull/1033 https://gitlab.com/gopiandcode/ppx-inline-alcotest/-/merge_requests/3 https://github.com/dariusf/ppx_interact/pull/3 https://github.com/ocaml-ppx/ppx_deriving_yojson/pull/160 https://github.com/LPCIC/elpi/pull/276 https://github.com/mirage/repr/pull/110 https://github.com/ProgramingIsTheFuture/ppx_default/pull/3 https://github.com/codinuum/vlt/pull/3 https://github.com/sim642/ppx_deriving_hash/pull/6 https://github.com/ocsigen/tyxml/pull/340 https://github.com/sim642/ppx_viewpattern/pull/2 https://github.com/little-arhat/ppx_jsobject_conv/pull/11 https://github.com/robur-coop/lun/pull/1 https://gitlab.com/o-labs/ppx_deriving_encoding/-/merge_requests/4 https://github.com/francoisthire/bam/pull/12 https://github.com/xguerin/bitstring/pull/36 tension is not new. 2025 2 18 The Future of Ppxlib Ppxlib is not going anywhere anytime soon. We have committed to trying to keep the internal AST up to date with the latest OCaml release. In fact, there is already a PR for the 5.3 bump! There have been many discussions about simplifying this process, but so far the time and effort to build a ppxlib an attempt using views Thank you to Tarides and Jane Street for funding my time to work on this release and all the chaos it has caused. Happy preprocessing! 2025 2 28 https://patrick.sirref.org/vpnkit-upgrade/ vpnkit-upgrade /vpnkit-upgrade/ Defunctorising VPNKit VPNKit is a core part of the Docker for Mac/Windows stack. It is tasked with translating network activity from the host to the Linux container. This short post discusses some of the recent changes to MirageOS and how they impact VPNKit. https://patrick.sirref.org/dune-virt-libs/ dune-virt-libs /dune-virt-libs/ Dune Virtual Libraries Dune, for quite some time, has supported virtual libraries module type S = struct ... end This feature has been quite widely used in OCaml. For example, the excellent digestif library of hashing algorithms provides both C and OCaml implementations individually packaged into digestif.c and digestif.ocaml Irmin in the browser js_of_ocaml will happily compile for us. There are two distinct limitations to this approach. Applications can only link exactly one implementation. dune#5597 2025 2 28 Functors in MirageOS MirageOS unix, hvt Recently, there has been a push to use dune virtual libraries instead. 2025 2 28 VPNKit's Fake Clock & Time For VPNkit, we need this to port some of the DNS forwarding tests which use a fake clock and sleep module. Previously, they could be provided directly as functor arguments to the various DNS modules e.g. let module R = Dns_forward.Resolver.Make Proto_client Fake.Time Fake.Clock In this new world of dune virtual libraries, our Resolver is significantly more simple. let module R = Dns_forward.Resolver.Make Proto_client The first thing I had to do was make the mirage-mtime and mirage-sleep virtual libraries unwrapped. This allowed me to make a new private dune library called fake_time with two public modules: the normal interface required by mirage-mtime and another module called Fake_time_state containing: let timeofday = ref 0L let c = Lwt_condition.create let advance nsecs = timeofday := Int64.add !timeofday nsecs; Lwt_condition.broadcast c let reset = timeofday := 0L; Lwt_condition.broadcast c I did the same for fake_sleep which depended on the state from the fake_time Whereas before, all the tests could live in the same test suite, now they had to be split into those that needed the fake implementation and those that did not. This is another slight limitation of this virtual library approach. Thus, the dune rules for running the dns_forward test suite became: ; Uses the default implementations provided by virtual libraries executable name test libraries dns_forward alcotest ; Override the defaults and select the private fake libraries executable name test_fake libraries dns_forward alcotest fake_sleep fake_time With all of this machinery in place, I could now update test_fake to use the new modules that would provide the custom functionality for the clock and time implementations. The bad server should be marked offline and no-one will wait for it Fake_time_state.reset ; avoid the timeouts winning the race with the actual result Fake_time_state.advance Duration. of_ms 500 ; 2025 2 28 Conclusion The switch to dune virtual libraries seems to be a good one. A majority of applications only require a single clock or time implementation. However, it has become harder to inject custom implementations of various resources into your MirageOS-based applications. The pull request to upgrade VPNKit is open. Patrick Ferris 2023 8 9 https://patrick.sirref.org/ocaml-rtree/ ocaml-rtree /ocaml-rtree/ Reviving OCaml R-Tree Fast-forwarding a decade-old OCaml implementation of the R-Tree geospatial data-structure. Antonin Guttman first proposed R-Trees in 1984. In order to handle spatial data efficiently, as required in computer aided design and geo-data applications, a database system needs an index mechanism that will help it retrieve data items quickly according to their spatial locations... R*-trees blog post seems to be the most popular introduction. Patrick Ferris https://patrick.sirref.org/ocaml-rtree-001/ ocaml-rtree-001 /ocaml-rtree-001/ An OCaml Rtree Implementation In the Energy and Environment Group we are working on various geospatial tools, one of which is written in OCaml and needed a geospatial index to speedup spatial queries over a dataset. Searching Github revealed an implementation in pure OCaml. After reaching out to the author, they were more than happy for their work to be maintained in the Geocaml organisation, updated to the latest OCaml tools and hopefully released on opam. The dunification Patrick Ferris https://patrick.sirref.org/ocaml-rtree-002/ ocaml-rtree-002 /ocaml-rtree-002/ OCaml Rtree Examples Rectangle envelope. module Line = struct type t = p0 : float * float; p1 : float * float let t = let open Repr in record "line" fun p0 p1 -> p0; p1 |+ field "p0" pair float float fun t -> t.p0 |+ field "p1" pair float float fun t -> t.p1 |> sealr type envelope = Rtree.Rectangle.t let envelope p0 = x1, y1 ; p1 = x2, y2 = let x0 = Float.min x1 x2 in let x1 = Float.max x1 x2 in let y0 = Float.min y1 y2 in let y1 = Float.max y1 y2 in Rtree.Rectangle.v ~x0 ~y0 ~x1 ~y1 end In addition to the envelope function, we also need to provide a runtime representation using Repr. In the example, we've constructed it by hand but I recommend using ppx_repr to derive it for you. From here we can create our index. module R = Rtree.Make Rtree.Rectangle Line Patrick Ferris https://patrick.sirref.org/ocaml-rtree-003/ ocaml-rtree-003 /ocaml-rtree-003/ Inserting Values There are two ways to insert values into the index -- either by inserting them one-by-one or by bulk loading them. The latter is the recommended approach where possible as the resulting layout of the R-Tree is much better. # let r = let i = R.empty 4 in R.insert i Line. p0 = 1., 1. ; p1 = 2., 2. ;; val r : R.t = <abstr> As you might expect, bulk loading takes a list of values instead. # let r = let lines = Line. p0 = 0., 0. ; p1 = 1., 1. ; p0 = 1., 1. ; p1 = 2., 2. ; p0 = 2., 2. ; p1 = 3., 3. ; p0 = 3., 3. ; p1 = 4., 4. ; in R.load ~max_node_load:4 lines;; val r : R.t = <abstr> Patrick Ferris https://patrick.sirref.org/ocaml-rtree-004/ ocaml-rtree-004 /ocaml-rtree-004/ Spatial Queries The lines we loaded into the last index are really just four segments of the line y = x. Let's spatially query for the first two segments. # R.find r Rtree.Rectangle.v ~x0:0. ~y0:0. ~x1:2. ~y1:2. ;; - : Line.t list = Line.p0 = 0., 0. ; p1 = 1., 1. ; Line.p0 = 1., 1. ; p1 = 2., 2. ; Line.p0 = 2., 2. ; p1 = 3., 3. You can see the query actually returned the first three, that's because all bound checks are inclusive. The source repository includes a fairly simple example of rendering an R-Tree using Vg. This can be done outside of the repository using the tree representation. # #show R.tree;; val tree : R.t -> R.tree type nonrec tree = R.tree = Node of Line.envelope * R.tree list | Leaf of Line.envelope * Line.t list | Empty Here is an example rendering from the repository showing some random lines and the envelopes of the R-tree. Patrick Ferris https://patrick.sirref.org/ocaml-blog/ ocaml-blog /ocaml-blog/ Patrick's OCaml Blog Welcome to the OCaml corner of my website. Here you will find long-form posts and shorter, more sporadic updates about my work in the OCaml community. This might include things about: Outreachy where I am the current OCaml community co-ordinator. Ppxlib where I help Nathan maintain the core library to build OCaml preprocessors. Any of the open-source tools I help maintain. An RSS feed is available. Posts Patrick Ferris 2025 8 7 https://patrick.sirref.org/irmin-retro/ irmin-retro /irmin-retro/ Irmin Retrospective Irmin is an OCaml library for building branchable and mergeable data stores. The data is mergeable in the sense of mergeable replicated data types. I have been using Irmin for over five years to build different kinds of interesting data stores including: A simple markdown-based note-taking web application. A content-addressed database. Mentoring an intern who worked on Irmin in the browser. Most recently, an Irmin-backed shell session manager. I was asked to provide some feedback recently on Irmin, so I thought writing a little retrospective here would be a good way to do that. The remit for the retrospective was about improving Irmin, so the content is focussed on pain points and areas of improvement. An in-memory Irmin store module Store = Irmin_mem.KV.Make Irmin.Contents.String Patrick Ferris 2025 8 7 What is Irmin? Irmin, at its simplest, is a key-value database. Users associate keys with values and can query and update these bindings. Additionally, this database supports versioning. This means independent snapshots of the database can coexist and users can switch between them and update them without fear of interfering with other versions. Different versions of the database can be combined by merging. When you set up your instance of an Irmin database, you also provide it with a merge function. # #show_type Irmin.Merge.f;; type nonrec 'a f = old:'a Irmin.Merge.promise -> 'a -> 'a -> 'a, Irmin.Merge.conflict result Lwt.t Patrick Ferris 2025 8 7 Design and API Irmin's API is very git-inspired. There is a large overlap of shared vocabulary and concepts: repositories, branches, commits, heads etc. Probably the most confusing aspect of this is the notion of a Store. When I was describing Irmin above, I used the term database to help distinguish between some of these concepts. In Irmin's documentation, it is used for multiple Creating a Store" stores refer to the entire database, whereas in the Irmin API docs we have that: There are two kinds of store in Irmin: the ones based on persistent named branches and the ones based temporary detached heads. For Irmin library users, a Store.t can be thought of as a checkout of the database at a particular revision. This may be from a branch or a specific commit. # Store.of_branch;; - : Store.repo -> string -> Store.t Lwt.t = <fun> # Store.of_commit;; - : Store.commit -> Store.t Lwt.t = <fun> The overloading of the term is confusing. I think it leaves users unsure about what other people might mean when they say "store". Being careful with these terms, and the contexts in which they are used, would help avoid this confusion. Patrick Ferris 2025 8 7 Module and Functor Soup Undoubtedly for a majority of use-cases and users, Irmin is over-functorised. Nearly every module requires that you must apply some functor to access any useful code. In general, a user will have to interact with the Schema module when describing the types they want to instantiate their store with. To counteract this, Irmin has plenty of KV modules that provide a Make functor that only requires a user to provide a suitable content module # #show_module Irmin_mem.KV;; module KV : sig type endpoint = unit type metadata = unit type hash type info = Store.info type contents_key = 'h type 'h node_key = 'h type 'h commit_key = 'h module]]> sig ... end end I think it is fair to say the documentation is hard to follow as it is a module and functor soup. For example, looking at irmin.3.11.0, starting at the toplevel documentation page our path to finding this paricular module and functor is as follows: We jump into Irmin_mem from the nicely written landing page. We scroll down to find the KV module. We now make sense of the module's signature by following the KV_maker link. We see a at the end of the module, and we navigate through that. Finally we have come to our journey's end, and find the schema module with type constraints like type Branch.t = string. There is a big assumption there that you know what the Schema module is telling you and how it relates to your "store". There is a counter-argument here, in that Irmin is incredibly flexible in terms of what you can use as types for your keys, branches, hashes etc. I was quite easily able to produce for example. I think there is a middle ground. For Irmin to be usable it needs to reduce the complexity of the API. The complexity is due to, in large part, over-functorisation. If the functors cannot be removed then perhaps better documentation or more introductory libraries like irmin-containers would be helpful. Perhaps a standalone library that acts as an interface to Irmin stores would be helpful. I find myself time and again implementing something like that. Patrick Ferris 2025 8 7 Backends Galore Irmin has plenty of backends including a git-compatible one, a MirageOS backend, an in-browser IndexedDB backend and even a FoundationDB backend. Irmin needs to have fewer, better tested backends. Of course, some of these are likely driven by funding and use-case specific details. However, I think by focusing on a few key backends and striving for solid performance but also strong consitency guarantees, Irmin would become a more viable candidate for potential users. I also believe the browser backend plays a key part to making this work in order to facilate local-first applications. Patrick Ferris 2025 8 7 Syncing Remote Stores The synchronisation mechanisms in Irmin are powerful, but often the API is very confusing. For example, with the git-compatible Unix backend, nested deep in the documentation is the function required to create a remote endpoint. Once they have found this function, using it is not easy as it requires them to learn about the Mimic.ctx, which is an abstraction of the networking stack! Patrick Ferris 2025 8 7 Do Fewer Things and Do Them Well Irmin is simultaneously a database that supports: content-addressing, merging, key-value lookup, branches, transactions, complete OS portability etc. I think, in short, it tries to do too many things and comes up short on some of them in ways that really matter. When building the content-addressed database I needed strong guarantees about some atomic actions to perform on the underlying store. For example, setting the value and accessing the commit associated with it which was not possible with the API at that time in an atomic way. This kind of missing functionality is probably only understood after users stumble across it, but it also makes the library feel less focused and ready for production use-cases. Patrick Ferris 2025 8 7 Documentation It will probably come as no surprise that one of the main limitations of using Irmin is the lack of documentation and tutorials. This is a particular pain point as the API is not very straightforward. A while back, I started an Irmin book intending to help document the kinds of things real-world users would need in order to use Irmin in earnest. For example: ways to deal with type migrations or primers on runtime types. I still believe this sort of work would be invaluable, but there is a large cost to completing it, and it is not clear if there is enough interest in the project to warrant such an effort. The tutorials on the irmin website are still good starting points for most new users. But they quickly lack the depth for real-world scenarios. Patrick Ferris 2025 8 7 Feature Wishlist What follows are additional ideas for improving Irmin. Patrick Ferris 2025 8 7 Heterogeneous Stores missing feature for lots of first-time users of the library: you can only store values of a single type. This leads to a few common workarounds: Storing a serealised version of your values, essentially escaping the type-system and implementing a form of dynamic typing. Growing your value to hold lots of different types via some large variant type. Both of these options are feasible and have been used in practice. However, they are workarounds. A long time ago CraigFe created an RFC for heterogeneous stores, which I think about a lot. The main idea is to augment Irmin to be GADTs that carry type information about the kinds of values they access heterogeneuous variants of other data structures API would make Irmin more appealing as a library for persistent data storage. Patrick Ferris 2025 8 7 Real-world Retrospectives This echos some thoughts from the Documentation section. There are some very real-world Irmin Tezos blockchain were using Irmin which uses the irmin-pack backend. Does this make irmin-pack the best tested backend and perhaps should be the default backend for new users? Or is it hyper-specific to the Tezos use case? There are blogs on tarides.com about the irmin-pack backend, but they might be outdated and should not be the first place to find advice on which Irmin backend to use. Patrick Ferris 2025 8 7 Active Development and Engagement As far as I know, Irmin was at the forefront of technologies that have come to be described as local-first. There is a growing interest Ink & Switch's essay on the matter. And it would be great to see more research via Irmin at things like the lofi unconference and the PLF workshop at SPLASH 2023! Thank you for reading! And thank you to all the Irmin contributors. I hope this might be useful in the future for building the next-generation Irmin! 2025 7 23 https://patrick.sirref.org/slipshow-x-xocaml/ slipshow-x-xocaml /slipshow-x-xocaml/ Slipshow x x-ocaml A short, explanatory post about combining two very fun pieces of work in OCaml. Paul-Elliot has been building Slipshow for some time now where slides are slips and your presentations run vertically. More recently, Arthur has built x-ocaml, a web component library for executable OCaml cells embedded into OCaml. Using xocmd, a small tool I built for translating markdown codeblocks to x-ocaml components, your Slipshow's can now be executable! learn-effects.html]]> Take a look at an example! (or the source markdown). I really like this light-weight approach to building interactive presentations Patrick Ferris 2025 7 18 https://patrick.sirref.org/ocaml-quarterly-q2/ ocaml-quarterly-q2 /ocaml-quarterly-q2/ Quarterly OCaml Q2 Thanks to Tarides sponsorship, I get to work on open-source OCaml. This quarterly is a companion to my weeklies, summarising the last three months of development, peppered with ideas and thoughts about OCaml, its community and its future. Patrick Ferris 2025 7 18 What I wanted to work on? There were two main things I hoped to continue working on: ppxlib and outreachy. These are projects that I was previously working on, and in the case of Outreachy I have now been involved for many years. In addition to this, all of my Part II projects this year used OCaml in some regard. In general, I want to see more adoption of OCaml. Over the years this has taken many forms including my initial work on developing workflows for OCaml that just turned five years old. This directly fed into the rebranding and rethinking of ocaml.org itself. Patrick Ferris 2025 7 18 What I worked on? Patrick Ferris 2025 7 18 Ppxlib Ppxlib is the de facto standard library for building OCaml preprocessors. At the time of writing, opam list --depends-on=ppxlib informs me that there are 267 reverse dependencies. Janestreet is a heavy user of ppxes and has authored many. One of the main accomplishments this quarter was bumping the internal AST to 5.2. This allows ppx authors to use new OCaml language features in their ppxes. In bumping the AST, we knowingly broke compatability for pretty much every single reverse dependency. As best we can, we have been sending patches to ppx libraries and helping users migrate to the latest ppxlib. There is an interesting dicussion to had about the nature of open-source, and its interaction with industrial monorepos and community tended package repositories. Package ecosystems thrive whenever there is a dedicated community creating, maintaining and publishing packages. The idea is that the published world should be healthy. The publishing medium can act as natural limiting PRs to the opam-repository have come to notice, reacts poorly to changes coming from internally consistent monorepos where introducing breaking changes is easily fixed by applying patches there and then. Whatsmore, OCaml is often stated as an incredibly safe language to perform large refactorings thanks to its type system. Ppxlib sits awkwardly in the space of possible breaking changes. Tied to OCaml's parsetree, impacts of changes there ripple down to ppxlib. The compiler itself can remain internally consistent, and is protected as it need only parse source code. Ppxlib, on the other hand, exposes the parsetree to users and thus any changes to the parsetree will likely be felt by ppx authors. Since I started working on ppxlib, it feels as though the number of syntax changes support for these, maintainers of ppxlib will forever be stuck doing busy work! Thankfully, Nathan has thoughts on how to improve this. There are a slew of other features I have added to ppxlib including: Support for deriving from classes. Support for deriving from module bindings and signatures. Fixing compiler and ppxlib dummy locations. Bumping to 5.3. Migrations for 5.4. Patrick Ferris 2025 7 18 Outreachy Our two projects this year, one on claudius and one on dune, are going extremely well. At the time of writing, we just had a mid-internship call to catch up, and I was blown away by the progress each intern has made. Unfortunately, Outreachy is struggling with funding and the next round is perhaps not going to happen. This is a real shame and I am hoping that it will not be the case. Outreachy has been a wonderful source of new, committed OCaml developers. If you haven't already, do peruse the webpage on OCaml.org to see past internships or watch the demo day presentations. For the mentors involved, We are always looking for new mentors and project ideas, please do reach out to me if you are interested. Patrick Ferris 2025 7 18 Hazel Hazel is a research programming language with typed holes written completely in OCaml reason dialect Relating this back to the original intent of my work, to improve OCaml adoption, I believe this also means keeping existing users happy. In the last quarter I developed a compiler from OCaml to Hazel. Whilst new features are still being added to Hazel, I hope this could serve as a tool to help develop test-suites and standard library functions using existing Typed Debugging for Hazel Hazel programs to great effect. OCaml should continue to be a world-class programming language for building other programming languages. I hope to upstream some of this work to Hazel and provide some low effort maintenance to help keep their compiler in good shape. Patrick Ferris 2025 7 18 Systems Programming in OCaml In a cross-over with my own research, I have been developing many tools related to systems programming in OCaml including: An eBPF-based open syscall tracing tool. A library in OCaml for spawning void processes. A shell session manager that uses Irmin to manage sessions. It is nice to see a new push to finally land the direct-style Irmin PR! This work is means to develop the underlying libraries that support it. For example, I have opened a few PRs to Eio to add new "fork actions" to the spawn API. I also investigated the feasibility of changing the underlying mechanisms in Eio to use Picos. In the future, I think this could be important avoid further splitting the OCaml ecosystem. Patrick Ferris 2025 7 18 OxCaml I dabbled a little with OxCaml and build try-oxcaml to let people take it for a spin without having to perform opam repository gymnastics. It turned into a lot of work to track down some pretty inane bugs This unblocked a few of my colleagues to get OxCaml working on tools like odoc_notebooks and x-ocaml. Patrick Ferris 2025 7 18 Forester A good proportion of my work this quarter has been focused on how to present the very work that I am doing. Jon Sterling has been developing a tool for scientific thought called Forester which seemed like a possible candidate for writing and sharing my work. Porting my existing blog posts and website content from markdown to Forester's LaTeX-inspired syntax didn't seem like an option. In particular, many of my posts ocaml-mdx This lead to the development of Graft: a preprocessor for Forester forests, converting markdown and bibtex to trees. Patrick Ferris 2025 7 18 What's next? So, I worked on most of what I wanted to work on and then some! Going forward I hope to keep maintaining ppxlib in some capacity and coordinating the OCaml community's Outreachy efforts. research makes heavy use of Irmin, and I would be interested to help with that too. More recently, a new library has been released for working with numerical data in OCaml called Raven: I am interested to use this library in Geocaml, a suite of geospatial tools written in OCaml that I maintain. I feel conflicted about the OxCaml efforts. I admit, I am not fully aware of the full benefits of the features, but I worry about some proliferation of modes that make the type system in OCaml unbearable to use. On top of this, with my ppxlib hat on, I worry about the impact of changing the compiler so frequently, placing strain on an already small community. That being said, by releasing OxCaml separately I do believe running it as an experimental set of packages will help understand the tool better. But I would not be building anything I intend to maintain or research with right now as that ecosystem is far too volatile. Finally, over the past two years I have done a lot of teaching. From Part II projects to supervisions intend to reduce my in-person teaching to focus on my research. However, I am interested in producing more materials for learning, maybe some of this in OCaml, perhaps something similar to Beautiful Racket. I have tried in the past to do these sorts of things, for example The Irmin Book. Patrick Ferris 2025 7 18 Thank you Thank you for reading this wrap up! And thank you again to Tarides for letting me work so freely on things that I think are good for the OCaml community. Patrick Ferris 2025 5 19 https://patrick.sirref.org/open-trace/ open-trace /open-trace/ Opentrace https://tangled.sh/@patrick.sirref.org/opentrace Thanks to Koonwen's excellent libbpf bindings in OCaml, I have been building a little tool called opentrace to make it easier to track an executable's inputs and outputs. This work was inspired my Michael's self-proclaimed "gross hack": pyshark. Whilst pyshark achieves its goals by injecting code into commonly used python objects and methods, opentrace uses eBPF opentrace can remain programming language agnostic. However, less information is none about the user's intent compared to something like pyshark. Patrick Ferris 2025 5 19 Monitoring the System opentrace has an all command that will trace the entire system. The --flags=O_WRONLY argument filters the events where the O_WRONLY flag was set in the call to open. comm -wrapped is an artefact of using Nix. Patrick Ferris 2025 5 19 Tracing an Executable The primary use case for this tool is to inspect what files your program might be reading and writing. The "flags" argument can specify a small boolean formula for checking the open flags of a particular event with | & ~ Parentheses can be used for precedence. Patrick Ferris 2025 5 19 Spawning Subprocesses One feature opentrace opentrace is primarily an eBPF program that is loaded into the kernel and communicates with an OCaml program. Events are communicated via a ring buffer and most of the post-processing happens in OCaml. To capture subprocesses, opentrace This gives opentrace a new identifier to track, namely the cgroup. So consider the following program. let = Eio_posix.run @@ fun env -> Eio.Path. save ~create: `Or_truncate 0o664 env#fs / "hello.txt" "hello" ; Eio.Process.run env#process_mgr "/bin/bash"; "-c"; "echo 'heya' > heya.txt" It first creates a file using direct calls to functions like openat. Then it spawns a process which creates a new file called heya.txt. This happens in a separate process. However, with the --cgroups flag we can capture both interactions with the operating system. Patrick Ferris 2025 5 19 Eio's Process API I have used the Eio_unix fork action process API to be able to extend what happens in the child process. Loading most eBPF programs into the kernel requires special privileges hence the need for sudo. When a user requests for a particular program to be executed and traced, opentrace spawns a process via the Eio Process API. Opentrace defines a few new so-called "fork actions", little fragments of C code that are run after the call to fork clone Most likely this ends with a call to execve, but other calls are possible for example setuid allowing opentrace to change the user of the child process so it does not run as root into that group. Patrick Ferris 2025 5 19 Limitations: Io_uring Whilst testing opentrace against some of the tools I use nearly daily, I noticed some events were being missed. I tried tracing forester, and only the initial read of forest.toml was logged. It dawned on me that the reason for this was that forester eio io_uring to perform most of the IO. Most attempts to open files were bypassing the open system calls, and instead they were being performed by the kernel after reading a submission request for an openat2-style call! This is not news to seasoned, Linux systems programmers. Io_uring bypasses SECCOMP filters for exactly the same reasons. It is interesting to note two things here: We can tell that forester reads the configuration file probably using something like In_channel it does It appears that Uring is performing IO in both worker threads and directly. The file paths are empty at the moment as I cannot find a clean way to trace openat submission requests into Uring without it sometimes going very wrong. do_filp_open io_openat2, but this seems quite brittle, and often the filename is completely garbled, so I do not set it. If anyone has any ideas to trace Io_uring more reliably, I am all ears! Patrick Ferris 2025 5 9 https://patrick.sirref.org/try-oxcaml/ try-oxcaml /try-oxcaml/ Try OxCaml https://patrick.sirref.org/oxcaml This week, I have been trying out Janestreet's Oxidised OCaml ICFP paper This adds a system of modes to OCaml for expressing things like locality. ...we’re introducing a system of modes, which track properties like the locality and uniqueness of OCaml values. Modes allow the compiler to emit better, lower-allocation code, empower users to write safer APIs, and with the advent of multicore, statically guarantee data race freedom—all in a lightweight way that only affects those in need. For example, you might say: let is_empty s : string @@ local = String.length s = 0 To get a feel for how this changes the language, you can follow their instructions on their custom opam-repository. Alternatively, you can give it a go in your browser! This is a full toplevel running with Base Happy OxCamling! Patrick Ferris 2025 5 2 https://patrick.sirref.org/hazel-of-ocaml/ hazel-of-ocaml /hazel-of-ocaml/ A Transpiler from OCaml to Hazel Over the past few months, I have been piecing together a transpiler from Hazel to OCaml. This is, in part, to help one of my third-year undergraduate students who is working on type error debugging in Hazel. Patrick Ferris 2025 5 2 Typed Holes Hazel is a functional programming language with typed holes. Holes are pieces of your program that have not yet been filled in. Holes can appear anywhere in your program both as expression or types. Hazel can still evaluate your program in the presence of holes. To get a flavour of Hazel, take a regular map function for lists. fun xs -> case xs | [] => [] | x :: xs => f (x) :: map(f)(xs) end in map(fun x -> ?)([1, 2, 3])]]> ? familiar with OCaml types ? list Hazel supports local type inference but nothing involving unification variables. For example, a simple add_one function in Hazel fun x -> x + 1 ? -> Int. Patrick Ferris 2025 5 2 From OCaml to Hazel The ability to transpile OCaml programs to Hazel programs is motivated by one simple thought: there are more OCaml programs than there are Hazel programs. This could help bootstrap projects by alleviating the need to rewrite Patrick Ferris 2025 5 2 A Transformation of Syntax Hazel markets itself as an "Elm/ML-like functional programming language". From the previous example of map, it should be apparent just how close to OCaml the language is. It turns out that a majority of the transpiler is a transformation of syntax. Take a simple ADT for an arithmetic programming language. type expr = | Float of float | Add of expr * expr | Sub of expr * expr | Mul of expr * expr | Div of expr * expr And when we run hazel_of_ocaml over this OCaml type declaration. Not much has changed expect some syntax. Hazel does not have a notion of top-level expression so hazel_of_ocaml wraps the program into one set of value bindings. For the most part, Hazel acts as a subset of the pure, functional part of OCaml. At the time of writing, this subset is fairly limited If we try out the same map function but written in OCaml and transpiled to Hazel we get. let rec map f = function | -> | x :: xs -> f x :: map f xs Which becomes the following hazel program. fun x1 -> case x1 | [] => [] | x :: xs => f(x) :: map(f)(xs) end in ?]]> We could have a field day discussing the syntax of OCaml and Hazel taking a look at how to handle polymorphism in Hazel. Patrick Ferris 2025 5 2 Explicit Polymorphism Hazel has explicit polymorphism. So far, we have not seen it as we have let the types have holes in them. The map function in OCaml has the following type. val map : 'a -> 'b -> 'a list -> 'b list Craig's excellent blogpost on the matter ... type variables in signatures are implicitly universally-quantified So in reality, we have that map has the following type. b) -> a list -> b list]]> In Hazel, we have to explicitly type our map function to be polymorphic. Not only does this mean the type annotation requires universally quantified type variables, but we must also perform type application wherever we choose to apply the map forall b -> (a -> b) -> [a] -> [b] = typfun a -> typfun b -> fun f -> fun xs -> case xs | [] => [] | x :: xs => f (x) :: map@@(f)(xs) end in map@@(fun x -> ?)([1, 2, 3])]]> forall introduces a universally quantified type variable into our type annotation, and typfun requires @<T> where T is some type. This allows hazel to quite easily support higher rank polymorphism, but we will not worry too much about that. Patrick Ferris 2025 5 2 Propagating OCaml Types into Hazel Most often, OCaml users interact with prenex at the front of the type. OCaml does support quantifiers inside certain types like records. What this means for the transpiler is that we can reuse OCaml's type inference to safely instantiate the correct type annotations and type applications in Hazel! To do this, hazel_of_ocaml uses Merlin to inspect the type of the function in either a value binding or at the point of a function application. Take a simple, polymorphic length function. let rec length = function | -> 0 | _ :: xs -> 1 + length xs let int_len = length 1; 2; 3 let str_len = length "only"; "two" When we run this through hazel_of_ocaml with the -type flag we get. [a] -> Int = typfun a -> fun x1 -> case x1 | [] => 0 | _ :: xs => 1 + length@(xs) end in let int_len : Int = length@(1 :: 2 :: [3]) in let str_len : Int = length@("only" :: ["two"]) in ?]]> hazel_of_ocaml has correctly instantiated the type for length inside the recursive function and then in each case with the integer list and the string list. Patrick Ferris 2025 5 2 A Corpus of Hazel Programs The impetus for this work was to derive a corpus of ill-typed Hazel programs. Luckily, such a corpus exists for OCaml! Seidel et al. created a corpus of OCaml programs from their undergraduate students at UC San Diego. Some of these programs have been transpiled to Hazel. Patrick Ferris 2025 5 2 Future Work Hazel is a fun, research programming language. Potential third-year students may find it interesting to take this work further. For example, how would this look in terms of a module system? From a purely engineering perspective, plenty of work would be needed to convert a multi-library OCaml project to Hazel cmi Another line of research would be to have Hazel target one of the intermediate representations in OCaml which would give Hazel a fully functioning compiler to "native" code? 2025 2 18 https://patrick.sirref.org/ppxlib-5-2/ ppxlib-5-2 /ppxlib-5-2/ Bumping Ppxlib's AST to 5.2 Ppxlib is a libary for building OCaml preprocessors. Users can generate OCaml code from At the core of ppxlib is the OCaml parsetree; a data structure in the compiler that represents OCaml source code. Ppxlib ppxlib some In ppxlib.0.36.0, the internal AST was bumped from the AST of OCaml 4.14.0, to the AST of OCaml 5.2.0. This post discusses some of the internal mechanisms that allows ppxlib to support multiple compilers and how this bump broke a lot of backwards compatibility. 2025 2 18 Manual Migrations in Ppxlib It is possible to perform migrations between parsetrees manually using the ppxlib.ast package. open Ppxlib open Ast_builder.Make struct let loc = Location.none end let pp_expr = Format.printf " " Pp_ast. expression ?config:None let loc = Location.none let str txt = txt; loc module To_408 = Ppxlib_ast.Convert Ppxlib_ast.Js Ppxlib_ast__.Versions.OCaml_408 The Ppxlib_ast.Js module is ppxlib ppxlib.0.36.0 provided by the Convert functor to copy parts of the AST from one version to another. Usually there is very little difference between the versioned parsetrees. let func arg body = pexp_fun Nolabel None ppat_var str arg body let add_expr = func "x" func "y" eapply evar "Int.add" evar "x"; evar "y" The above excerpt of OCaml code produces an expression in the language. More accurately, a function that has two arguments. If we print the simplified AST we can see that the arguments are stored together in a list. # pp_expr add_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" ; pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "add" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit = This is a relatively new feature of the parsetree, if we migrate the expression down to 4.08 we see it is represented differently. The output is a little too verbose to be readable, and there's no good way to show the AST, so you will just have to take my word. let expr_408 = To_408.copy_expression add_expr 2025 2 18 Function Arity in the OCaml Parsetree OCaml 5.2 introduced a new way to represent functions. Before 5.2, all functions were represented with single arity. Functions with multiple arguments would return functions as their body. For example: let add x y = x + y A two-argument add function would be represented in the parsetree as fun x -> fun y -> x + y . Now, since OCaml 5.2, functions can be expressed with their syntactic arity intact. The function add would look something like . Both arguments are stored as a list. The constructor Pexp_function was not a new addition to the parsetree. It used to represent pattern-matching expressions like function A -> 1. This can now be expressed as a function body of Pfunction_cases. Whilst these additions and modifications to the parsetree are arguably a better design, it has caused chaos in terms of ppxlib's reverse dependencies! 2025 2 18 Coalescing Function Arguments In ppxlib 0.36.0, functions from Ast_builder will help ppx authors produce maximum arity functions. However, functions from Ast_helper will not do this. let ast_helper_func arg body = Ast_helper.Exp.fun_ Nolabel None ppat_var str arg body let ast_helper_expr = ast_helper_func "x" ast_helper_func "y" eapply evar "Int.equal" evar "x"; evar "y" Here we recreate our helper function to create function expressions using Ast_helper.Exp.fun_ instead. We can now show the difference between the expr expression and the ast_helper_expr expression. # pp_expr add_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" ; pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "add" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit = Above, we see a function expression with arity two. Below, we see nested function expressions each with arity one. # pp_expr ast_helper_expr;; Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "x" , None , Pfunction_body Pexp_function pparam_loc = __loc ; pparam_desc = Pparam_val Nolabel, None, Ppat_var "y" , None , Pfunction_body Pexp_apply Pexp_ident Ldot Lident "Int", "equal" , Nolabel, Pexp_ident Lident "x" ; Nolabel, Pexp_ident Lident "y" - : unit = 2025 2 18 Ramifications for Ppxlib Users Many, many, many ppxes broke after migrating the ppxlib AST to 5.2. Mostly, this is due to pattern-matching. Downstream users of ppxlib frequently pattern-match directly against the parsetree. However, nodes such as Pexp_fun no longer exist and Pexp_function has completely different constructor arguments! Pexp_fun x function Here is a non-exhaustive list of some of the ppxes that have been patched. It may useful for other ppx authors who wish to upgrade to ppxlib.0.36.0: https://github.com/patricoferris/ppx_deriving_yaml/pull/58 https://github.com/mirage/ocaml-rpc/pull/181 https://gitlab.com/o-labs/ppx_deriving_jsoo/-/merge_requests/1 https://github.com/ocaml-community/sedlex/pull/160 https://github.com/ocaml-sys/config.ml/pull/22 https://github.com/vogler/ppx_distr_guards/pull/2 https://github.com/ocaml-gospel/gospel/pull/424 https://github.com/teamwalnut/graphql-ppx/pull/299 https://github.com/ocsigen/lwt/pull/1033 https://gitlab.com/gopiandcode/ppx-inline-alcotest/-/merge_requests/3 https://github.com/dariusf/ppx_interact/pull/3 https://github.com/ocaml-ppx/ppx_deriving_yojson/pull/160 https://github.com/LPCIC/elpi/pull/276 https://github.com/mirage/repr/pull/110 https://github.com/ProgramingIsTheFuture/ppx_default/pull/3 https://github.com/codinuum/vlt/pull/3 https://github.com/sim642/ppx_deriving_hash/pull/6 https://github.com/ocsigen/tyxml/pull/340 https://github.com/sim642/ppx_viewpattern/pull/2 https://github.com/little-arhat/ppx_jsobject_conv/pull/11 https://github.com/robur-coop/lun/pull/1 https://gitlab.com/o-labs/ppx_deriving_encoding/-/merge_requests/4 https://github.com/francoisthire/bam/pull/12 https://github.com/xguerin/bitstring/pull/36 tension is not new. 2025 2 18 The Future of Ppxlib Ppxlib is not going anywhere anytime soon. We have committed to trying to keep the internal AST up to date with the latest OCaml release. In fact, there is already a PR for the 5.3 bump! There have been many discussions about simplifying this process, but so far the time and effort to build a ppxlib an attempt using views Thank you to Tarides and Jane Street for funding my time to work on this release and all the chaos it has caused. Happy preprocessing! 2025 2 28 https://patrick.sirref.org/vpnkit-upgrade/ vpnkit-upgrade /vpnkit-upgrade/ Defunctorising VPNKit VPNKit is a core part of the Docker for Mac/Windows stack. It is tasked with translating network activity from the host to the Linux container. This short post discusses some of the recent changes to MirageOS and how they impact VPNKit. https://patrick.sirref.org/dune-virt-libs/ dune-virt-libs /dune-virt-libs/ Dune Virtual Libraries Dune, for quite some time, has supported virtual libraries module type S = struct ... end This feature has been quite widely used in OCaml. For example, the excellent digestif library of hashing algorithms provides both C and OCaml implementations individually packaged into digestif.c and digestif.ocaml Irmin in the browser js_of_ocaml will happily compile for us. There are two distinct limitations to this approach. Applications can only link exactly one implementation. dune#5597 2025 2 28 Functors in MirageOS MirageOS unix, hvt Recently, there has been a push to use dune virtual libraries instead. 2025 2 28 VPNKit's Fake Clock & Time For VPNkit, we need this to port some of the DNS forwarding tests which use a fake clock and sleep module. Previously, they could be provided directly as functor arguments to the various DNS modules e.g. let module R = Dns_forward.Resolver.Make Proto_client Fake.Time Fake.Clock In this new world of dune virtual libraries, our Resolver is significantly more simple. let module R = Dns_forward.Resolver.Make Proto_client The first thing I had to do was make the mirage-mtime and mirage-sleep virtual libraries unwrapped. This allowed me to make a new private dune library called fake_time with two public modules: the normal interface required by mirage-mtime and another module called Fake_time_state containing: let timeofday = ref 0L let c = Lwt_condition.create let advance nsecs = timeofday := Int64.add !timeofday nsecs; Lwt_condition.broadcast c let reset = timeofday := 0L; Lwt_condition.broadcast c I did the same for fake_sleep which depended on the state from the fake_time Whereas before, all the tests could live in the same test suite, now they had to be split into those that needed the fake implementation and those that did not. This is another slight limitation of this virtual library approach. Thus, the dune rules for running the dns_forward test suite became: ; Uses the default implementations provided by virtual libraries executable name test libraries dns_forward alcotest ; Override the defaults and select the private fake libraries executable name test_fake libraries dns_forward alcotest fake_sleep fake_time With all of this machinery in place, I could now update test_fake to use the new modules that would provide the custom functionality for the clock and time implementations. The bad server should be marked offline and no-one will wait for it Fake_time_state.reset ; avoid the timeouts winning the race with the actual result Fake_time_state.advance Duration. of_ms 500 ; 2025 2 28 Conclusion The switch to dune virtual libraries seems to be a good one. A majority of applications only require a single clock or time implementation. However, it has become harder to inject custom implementations of various resources into your MirageOS-based applications. The pull request to upgrade VPNKit is open. Patrick Ferris 2023 8 9 https://patrick.sirref.org/ocaml-rtree/ ocaml-rtree /ocaml-rtree/ Reviving OCaml R-Tree Fast-forwarding a decade-old OCaml implementation of the R-Tree geospatial data-structure. Antonin Guttman first proposed R-Trees in 1984. In order to handle spatial data efficiently, as required in computer aided design and geo-data applications, a database system needs an index mechanism that will help it retrieve data items quickly according to their spatial locations... R*-trees blog post seems to be the most popular introduction. Patrick Ferris https://patrick.sirref.org/ocaml-rtree-001/ ocaml-rtree-001 /ocaml-rtree-001/ An OCaml Rtree Implementation In the Energy and Environment Group we are working on various geospatial tools, one of which is written in OCaml and needed a geospatial index to speedup spatial queries over a dataset. Searching Github revealed an implementation in pure OCaml. After reaching out to the author, they were more than happy for their work to be maintained in the Geocaml organisation, updated to the latest OCaml tools and hopefully released on opam. The dunification Patrick Ferris https://patrick.sirref.org/ocaml-rtree-002/ ocaml-rtree-002 /ocaml-rtree-002/ OCaml Rtree Examples Rectangle envelope. module Line = struct type t = p0 : float * float; p1 : float * float let t = let open Repr in record "line" fun p0 p1 -> p0; p1 |+ field "p0" pair float float fun t -> t.p0 |+ field "p1" pair float float fun t -> t.p1 |> sealr type envelope = Rtree.Rectangle.t let envelope p0 = x1, y1 ; p1 = x2, y2 = let x0 = Float.min x1 x2 in let x1 = Float.max x1 x2 in let y0 = Float.min y1 y2 in let y1 = Float.max y1 y2 in Rtree.Rectangle.v ~x0 ~y0 ~x1 ~y1 end In addition to the envelope function, we also need to provide a runtime representation using Repr. In the example, we've constructed it by hand but I recommend using ppx_repr to derive it for you. From here we can create our index. module R = Rtree.Make Rtree.Rectangle Line Patrick Ferris https://patrick.sirref.org/ocaml-rtree-003/ ocaml-rtree-003 /ocaml-rtree-003/ Inserting Values There are two ways to insert values into the index -- either by inserting them one-by-one or by bulk loading them. The latter is the recommended approach where possible as the resulting layout of the R-Tree is much better. # let r = let i = R.empty 4 in R.insert i Line. p0 = 1., 1. ; p1 = 2., 2. ;; val r : R.t = <abstr> As you might expect, bulk loading takes a list of values instead. # let r = let lines = Line. p0 = 0., 0. ; p1 = 1., 1. ; p0 = 1., 1. ; p1 = 2., 2. ; p0 = 2., 2. ; p1 = 3., 3. ; p0 = 3., 3. ; p1 = 4., 4. ; in R.load ~max_node_load:4 lines;; val r : R.t = <abstr> Patrick Ferris https://patrick.sirref.org/ocaml-rtree-004/ ocaml-rtree-004 /ocaml-rtree-004/ Spatial Queries The lines we loaded into the last index are really just four segments of the line y = x. Let's spatially query for the first two segments. # R.find r Rtree.Rectangle.v ~x0:0. ~y0:0. ~x1:2. ~y1:2. ;; - : Line.t list = Line.p0 = 0., 0. ; p1 = 1., 1. ; Line.p0 = 1., 1. ; p1 = 2., 2. ; Line.p0 = 2., 2. ; p1 = 3., 3. You can see the query actually returned the first three, that's because all bound checks are inclusive. The source repository includes a fairly simple example of rendering an R-Tree using Vg. This can be done outside of the repository using the tree representation. # #show R.tree;; val tree : R.t -> R.tree type nonrec tree = R.tree = Node of Line.envelope * R.tree list | Leaf of Line.envelope * Line.t list | Empty Here is an example rendering from the repository showing some random lines and the envelopes of the R-tree. Weeklies A collection of less thoughtfully crafted reports, focused on OCaml-related activities. Patrick Ferris 2026 3 10 https://patrick.sirref.org/ocaml-roundup-february-2026/ ocaml-roundup-february-2026 /ocaml-roundup-february-2026/ OCaml Roundup: February 2026 Much like last month, this month has been busy with lots of work on ocaml-tiff thanks to Tambe Salome and ppxlib. Patrick Ferris 2026 3 10 Outreachy Tambe Salome, at the time of writing, has completed her internship! We will be hosting the biannual Demo Day celebration for this round's interns, so please do come along. Since the last update Tambe Salome and I have been slowly making progress on making the write-functionality more feature-complete. As I eluded last month, TIFF files are hard to write. Their internal structure is all over the place: directories, immediate and non-immediate values, 32-bit or 64-bit, little or big endian etc. We're tackling this bit by bit, evolving the library to be able to support writing to files. In the early days, when we only read TIFF files there was no need to something like in memory before writing them to the file, so we will need some kind of API change internally to support this. This month, I also bumped up against well-meaning contributions to ocaml-tiff that made use of AI. This left me with some thoughts that I am adding below. Patrick Ferris 2026 3 9 https://patrick.sirref.org/vibecoding-etiquette/ vibecoding-etiquette /vibecoding-etiquette/ Vibecoding Etiquette I am yet to jump head-first into using LLM-based tools like Claude Code or even ChatGPT to help with my programming in any serious way. After using some "free"-tier tools to try to better understand some tricky eBPF problems and to make sense of the semantics of POSIX shells, I was left unimpressed and ultimately felt my time was wasted. I feel unwilling to send my money to the corporations behind these tools. This is for a variety of reasons including the deeply worrying relationship with the "opertational planning" vs. "operational decision-making" given the leaders of the US DoW environmental concerns, exploitation of human labour and a fast-and-loose approach to copyright laws. All of which leaves the entire industry very unattractive to me. Everybody can draw their line where they want, and I am, in good faith, trying to find where mine should be. Unfortunately, my own decision to not use these tools is not completely in my control. What do I do about pull requests to open-source libraries I maintain that include agentic code? I am, after all, participating in the idea of open-source and inclusive collaboration. Agentic code also comes in many flavours, from blatant LLM-generated pull requests to heavily edited, human-in-the-loop contributions. An immediate first step for me is to develop some policy for contributions to projects I maintain regarding code where LLMs have had a hand. This can help set expectations for contributors. Eio's "AI-generated Code" subsection is succinct and prohibits contributions that solely use AI. It obfuscates how you think. Purely AI-generated code tells us little about how you think and the problems you might be having. This makes it harder to provide good feedback on PRs and issues. It is often more work to review. Particularly for the OCaml ecosystem and libraries like Eio, it seems that these tools are not very good and generate a lot of believable code that is in actual fact completely wrong. PR comments and the code submitted with them can say completely different things. It is a grey area for licensing. Models like ChatGPT have been trained on lots of code with different licenses and has been known to simply copy code as an answer to a prompt. We would like to avoid this headache as best we can. Additionally, I plead to other developers to own their use of these tools. Please, make Claude a co-author of the commits where Claude has generated any code at the very least. This is the bare minimum. I have strong feelings about the etiquette of sending AI summarised information or LLM-generated code to other people, but before any of that, the decent thing is to remove any uncertainty in the receiving party's mind of how the information they are receiving came to be. In this way, I appreciate the OCaml compiler's AI notice which includes: If a significant portion of your code, PR description, review comments or messages has been AI-generated, this must be disclosed, stating which tool was used and for what. Reviewing AI-produced code can require a different approach from human-written code. You should have an acknowledgement anyway even if another human contributor did some of the work. Patrick Ferris 2026 3 10 Ppxlib We are still in the throes of supporting OCaml 5.5 in ppxlib. Some of the features have proved more complicated to encode into migrations that "roundtrip". Roundtripping, in the context of ppxlib, is when a modern feature of OCaml must be preserved in older abstract syntax trees which may have no specific node to encode them. Patrick Ferris 2026 3 10 Bugs in Ptyp_open Unfortunately, Jon Sterling was bitten by this recently. module M = struct type t = int end type t = M. t list Here, the is an example of a new AST node called Ptyp_open. I had hastily not migrated Ptyp_open to older compilers. So what happened? Jon had used this new feature in his project with a 5.3 compiler with ppxlib 0.35. This is before our internal bump to the 5.2 AST so the code migrated all the way down to OCaml 4.14! Whilst passing from 5.2 to 5.1 we raised an error saying: Error: migration error: module open in types is not supported before OCaml 5.02 patched this issue and have planned to release a patched 0.35 in the not too distant future. If you can bump to ppxlib>=0.36 then you will not face this issue. Patrick Ferris 2026 3 10 Modular Explicits We ran into a fairly interesting corner of the OCaml language whilst trying to migrate modular explicits in ppxlib. As a recap, in OCaml 5.5 you can now right function types that depend on modules. For example: module type Add = sig type t val add : t -> t -> t end let add : module A : Add -> A.t -> A.t -> A.t = fun module A : Add a b -> A.add a b We have to tell the compiler the module type of A by adding : Add. However, there is another very similar syntax for these kinds of constraints, . And in fact, the former syntax was syntactic sugar for the latter. In OCaml 5.5, these two were distinguished from one another and is the required syntax for modular explicits. As you might imagine, this is a bit of a headache in terms of ppxlib migrations. In the end, we try to preserve the original intent of the author where possible, and you can read more about it on the PR for distinguishing the two. Patrick Ferris 2026 3 10 Six Month Retrospective This month marks the end of six months of work with Tarides. I wrote a small retrospective and plan for my next six months below. Patrick Ferris 2026 2 9 https://patrick.sirref.org/fellowship-roundup/ fellowship-roundup /fellowship-roundup/ Fellowship Roundup Patrick Ferris 2026 2 9 Overview February is the final month of my current six-month fellowship work with Tarides. A lot of the work can be summarised by reading the roundups October, November, December and January However, they do not quite capture all the work I have been doing. Below, I pick individual projects and expand on them more holistically rather than pointing at individual PRs or issues. Patrick Ferris 2026 2 9 Ppxlib Most of my time over the past six months has been devoted to ppxlib maintenance. A large proportion of this time has been solo development work. Ppxlib Moreover, it is completely invaluable to Jane Street too. Like other AST-based ppxlib must follow a regular release schedule to support newer compilers. During the years of multicore OCaml development, ppxlib maintenance was a little easier as the Parsetree rarely changed. However, since then, the same cannot be said. Since OCaml 5.2, here are just some of the Parsetree changes: Functions are represented according to their arity. Local module opens on types. Effect syntax. Modular explicits. External types. Locations for all parts of long identifiers. Labeled tuples. In a perfect world, each of these require a codec for serialising the feature into ASTs that do not support the feature, plenty of tests and new Ast_builder/Ast_pattern functions for using the feature. Nathan and I have been managing pretty well I would say, though we are not getting any technical debt work done. Patrick Ferris 2026 2 9 Outreachy As the OCaml coordinator for Outreachy I have been running the latest December 2025 round. We are lucky to have four projects on the go! Thibaut Mattio is mentoring two projects: an ML dashboard for Raven and an OxCaml backend for Raven. Xvw is mentoring a Yocaml project. I am mentoring a Geocaml project. Patrick Ferris 2026 2 9 Eio During my work on tools like Shelter I have made a few improvements to Eio including: A draft PR for incremental reading of directories. New fork actions for processes: setgid, setuid and process groups. Utility functions for overriding standard environments. Fixing documentation. Patrick Ferris 2026 2 9 OxCaml I have enjoyed my work on OxCaml went it has happend. I think a few tools have been useful including the try-oxcaml work. David and I also had fun at ICFP discussing some quick experiments related to io_uring and OxCaml. One road-blocker here is the comparative amount of time spent trying to keep the OxCaml ecosystem working. Given my limited time working on fellowship most of the time was spent making it just work, not exploring or experimenting with the features themselves. I think this has got better in recent months and perhaps jumping back in I would be surprised at the progress that has been made there. Patrick Ferris 2026 2 9 Forester Tarides are funding some work on forester. This site uses forester by way of graft and I have spent some time working on these tools during the fellowship. This includes a bytesrw-based bibtex library in OCaml that could be released soon. Patrick Ferris 2026 2 9 Papers I had the great fortune of attending ICFP where I presented two talks and was co-author on a few others too. Anil Madhavapeddy David J. Scott Patrick Ferris Ryan T. Gibb Thomas Gazagnaire 2025 https://patrick.sirref.org/madhavapeddy2025docker/ madhavapeddy2025docker /madhavapeddy2025docker/ Functional Networking for Millions of Docker Desktops (Experience Report) Reference https://dl.acm.org/doi/10.1145/3747525 Proc. ACM Program. Lang. 10.1145/3747525 Anil Madhavapeddy David J. Scott Patrick Ferris Ryan T. Gibb Thomas Gazagnaire 2025 Abstract Docker is a developer tool used by millions of developers to build, share and run software stacks. The Docker Desktop clients for Mac and Windows have long used a novel combination of virtualisation and OCaml unikernels to seamlessly run Linux containers on these non-Linux hosts. We reflect on a decade of shipping this functional OCaml code into production across hundreds of millions of developer desktops, and discuss the lessons learnt from our experiences in integrating OCaml deeply into the container architecture that now drives much of the global cloud. We conclude by observing just how good a fit for systems programming that the unikernel approach has been, particularly when combined with the OCaml module and type system. Patrick Ferris 2025 https://patrick.sirref.org/ferris2025scientific_programming/ ferris2025scientific_programming /ferris2025scientific_programming/ What we talk about when we talk about scientific programming Reference https://watch.eeg.cl.cam.ac.uk/w/aYXqXLtgQawYMjVXjSQtjx Patrick Ferris 2025 Abstract Patrick Ferris Anil Madhavapeddy 2025 https://patrick.sirref.org/ferris2025hazel_of_ocaml/ ferris2025hazel_of_ocaml /ferris2025hazel_of_ocaml/ Generating a corpus of Hazel programs from ill-typed OCaml programs Reference https://patrick.sirref.org/var/tyde2025.pdf Patrick Ferris Anil Madhavapeddy 2025 Abstract When developing a new programming language, having a large corpus of both correct and incorrect programs allows language designers to test and explore the capabilities of their new language. However, bootstrapping such a corpus of incorrect programs is time-consuming and arduous. We therefore explore how to reuse code from more mature languages to generate a corpus of ill-typed code for newer ones. We have developed a compiler to Hazel, an emerging language with typed holes, from the more mature OCaml ecosystem. We find it practical to generate a comprehensive corpus of ill-typed programs for Hazel development, and discuss future larger scale efforts towards bridging ecosystems. See also the online hazel of ocaml compiler Max Carroll Anil Madhavapeddy Patrick Ferris 2025 https://patrick.sirref.org/carroll2025decomposable_type_highlighting/ carroll2025decomposable_type_highlighting /carroll2025decomposable_type_highlighting/ Decomposable Type Highlighting for Bidirectional Type and Cast Systems Reference https://maxcarroll0.github.io/papers/workshops/HATRA-decomposable-type-highlighting/ Max Carroll Anil Madhavapeddy Patrick Ferris 2025 Abstract We explore how to provide programmers with an interactive interface for explaining the process by which static types and dynamic casts are derived, with the goal of improving the debugging of static and dynamic type errors. To this end, we define mathematical foundations for a decomposable highlighting system within a bidirectional system, and show how these can be propagated through dynamic types in a cast system. Our prototype implementation in the gradually typed Hazel language includes a web-based user interface, through which we highlight the importance of type level debugging. Michael Winston Dales Alison Eyres Patrick Ferris Francesca A. Ridley Simon Tarr Anil Madhavapeddy 2025 https://patrick.sirref.org/dales2025yirgacheffe/ dales2025yirgacheffe /dales2025yirgacheffe/ Yirgacheffe: A Declarative Approach to Geospatial Data Reference https://dl.acm.org/doi/10.1145/3759536.3763806 10.1145/3759536.3763806 Michael Winston Dales Alison Eyres Patrick Ferris Francesca A. Ridley Simon Tarr Anil Madhavapeddy 2025 Abstract I had some great conversations with KC too with respect to my current research. Patrick Ferris 2026 2 9 Future Work As I enter my final year of the PhD, my goal would be to try and align more of my fellowship work with my own research wherever it makes sense. This research focuses on the following hypothesis: Embedding deep provenance tracking, reversible execution and mergeable reproducible, publishable results; all whilst leaving existing workflows intact and being programming language agnostic. I want to focus on building these tools using Irmin, Eio, OxCaml etc. There are other projects I wish to continue regardless, like Outreachy Patrick Ferris 2026 2 9 Overlapping PhD Work There are multiple projects that overlap with my work. For example, Irmin, which I wrote a retrospective for. This is at the core of Shelter and suggests an extremely valid use case for a brancheable, mergeable database. In addition to this, continuing to work on Eio makes sense to me too. Eio currently has no maintainer and no active development, but the issues are piling up. In particular, I am making heavy use of Eio in Merry OCaml. Aside from research angles on brancheable and mergeable shells, I am also quite interested in how this might look in terms of OxCaml. I met with Anil and Thomas recently to discuss the future of Eio and there was some agreement that we are all heavy users of it and hope to keep up its maintenance and set of features. One large piece of work that is sorely needed here is moving Eio to bytes and not bigarray-backed Cstructs. Any libraries using bytesrw jsont Anil has been working on some of the pieces recently to make this better! Patrick Ferris 2026 2 9 Maintaining and Releasing my own libraries I have published quite a few OCaml libraries in my time, below is just a few Some of these are used in the community including hilite, ppx_deriving_yaml and the multi-codecs. I would like to do some general maintenance of some of these tools and release a few iterations of graft which now makes use of bib. Patrick Ferris 2026 2 9 Ppxlib Maintenance By far, maintenance of ppxlib takes up a majority of my time. I think, given the other work I need to focus on, it would be great to try to minimise the amount of time working on ppxlib. In particular, perhaps only maintaining ppxlib as far as OCaml release-readiness is a possibility and the OOM bug reported by Jane Street Patrick Ferris 2026 2 9 Forester, Graft and Writing With all of this work, I am eager to communicate what I am up to and my thoughts on open source, OCaml, OxCaml etc. I am glad to be using Forester my own work on Eio I hope to continue working on Graft and related tools like bib. But I would also like to start writing some long-form content similar to the Irmin retrospective. Patrick Ferris 2026 2 5 https://patrick.sirref.org/ocaml-roundup-january-2026/ ocaml-roundup-january-2026 /ocaml-roundup-january-2026/ OCaml Roundup: January 2026 A new year, another roundup in my open-source, OCaml activities. This month has been busy with lots of work on ocaml-tiff thanks to Tambe Salome and ppxlib. Patrick Ferris 2026 2 5 Outreachy Tambe Salome has been working to add write support to ocaml-tiff; and we are well on the way to having good support. Writing TIFF files is no easy business. Unlike many formats, TIFF files have an internal metadata-data split, with values sometimes being place nowhere near the metadata describing the values in the file. In practice, it means if you do not wish to read the whole file into memory, then your IO layer must support seeking to file offsets. This means when you come to write the file, you might need to write at different offsets too. This becomes particularly complicated when you do not know ahead of time how copying a TIFF file we have access to but pushing all the copying via the OCaml values we read from our source TIFF. And Salome has made excellent progress: input.exif $ dune exec -- example/copy.exe test/data/uniform.tiff example.tiff $ exiftool example.tiff > output.exif $ patdiff input.exif output.exif ------ input.exif ++++++ output.exif @|-1,23 +1,23 ============================================================ |ExifTool Version Number : 13.39 -|File Size : 66 kB +|File Size : 118 bytes |File Permissions : -rw-r--r-- |File Type : TIFF |File Type Extension : tif |MIME Type : image/tiff |Exif Byte Order : Little-endian (Intel, II) |Image Width : 256 |Image Height : 256 |Bits Per Sample : 8 |Compression : Uncompressed |Photometric Interpretation : BlackIsZero |Strip Offsets : 122 |Rows Per Strip : 256 |Strip Byte Counts : 65536 |Planar Configuration : Chunky |Image Size : 256x256 |Megapixels : 0.066]]> From this slightly edited output, we can see that the OCaml libary is doing a good job at preserving the metadata. The file size discrepancy is because we have not tackled writing the raw data to the file yet! This is next on our TODOs, write data to a file given a strip-offset plan. Once we do that we can then start to think how might we come up with such a plan if we did not have one. Patrick Ferris 2026 2 5 Ppxlib The first cut of OCaml 5.5 is not too far away, at which point the ppxlib of new features sneaked into this release including: modular explicits, external type declarations and the previously mentioned local structure items. All of these features require migrations to support newer compilers. In this migration process, ppxlib lowers modern ASTs to the AST of OCaml 5.2. New migration process. We have a few PRs to support 5.5 fully: The initial PR to support 5.5. Supporting all the new local structure items. Full support for modular explicits and external type declarations. Additionally, Nathan has been adding better feature encoding support for the 5.3 and 5.4. Unlike the migration code above, these also include Ast_builder and Ast_pattern functions for building and matching on the new AST nodes. This allows ppx authors to use those nodes in their ppxes. OCaml 5.3 effect syntax. OCaml 5.4 labeled tuples. Finally, we have an important bug fix where long identifiers were given a Location.none. Unfortunately, it is not possible to fully recover a long Read more about that fix here. Patrick Ferris 2026 1 11 https://patrick.sirref.org/ocaml-roundup-december-2025/ ocaml-roundup-december-2025 /ocaml-roundup-december-2025/ OCaml Roundup: December 2025 December was busy, both personally and in my OCaml-related work. If you haven't been following along have a read of my October and November roundups. Patrick Ferris 2026 1 11 https://patrick.sirref.org/outreachy-december-round/ outreachy-december-round /outreachy-december-round/ Outreachy December Round December marks the beginning of a new Outreachy round, and I am very excited to be working with Tambe Salome on improving the ocaml-tiff library. In particular, we will be adding the ability to write TIFF files from raw data. I imagine an API something like: val write : 'repr, 'kind Tiff.Data.t -> Tiff.File.wo -> unit So far Salome has been making excellent progress churning through some issues that are blocking us from starting the main project, including: Adding a sublibrary for Eio. Using the new Nx library. Uncovering bugs in our decoding of the number of bits per pixel Preliminary work on write support! As a community, the mentors and mentees also had a community call to introduce ourselves to eachother. I am very excited about the Yocaml project and the two Raven projects. Patrick Ferris 2026 1 11 Ppxlib If you recall, I discussed the new representation for local constructs in the AST. Patrick Ferris 2025 12 9 https://patrick.sirref.org/ppxlib-5.5-support ppxlib-5.5-support /ppxlib-5.5-support Initial OCaml 5.5 Support We recently merged 5.5 support into our main some new features coming to OCaml including the ability to define any structure locally. Most OCaml programmers will be familiar with things like let open M in External types have also landed, bringing with them a new "type kind" in the AST. You can read about them in the manual. It transpires that since then, changes have landed in the compiler that mean just about any OCaml structure item can now be declared locally. I have documented an issue on ppxlib about the problems this will cause for us: Currently, we only migrate the previously supported structure items that let open allows anything to appear in a local let-binding except additional let let ... let include change had a 1:1 mapping from a 5.4 AST node to a 5.5 AST node, now we do not and will need to think of a suitable encoding. Patrick Ferris 2026 1 11 Migration Woes gasche for the report Longidents are used in OCaml to identify values like Foo.bar or . In OCaml 5.4, each segment of the identifier was given a precise location. This means that in the migration from 5.3 to 5.4 we have to reconstruct those locations. However, our code didn't do this, instead it simply defaulted to Location.none causing some imprecise error codes in preprocessed OCaml files. For example: let = NonExistingModule.run Has the following error: I have proposed a fix which partially brings back locations during the migration. Part of the problem, I think, is that the exact location of long identifiers containing applications Patrick Ferris 2025 12 9 https://patrick.sirref.org/ocaml-roundup-november-2025/ ocaml-roundup-november-2025 /ocaml-roundup-november-2025/ OCaml Roundup: November 2025 A roundup of some of my OCaml activity this month! Patrick Ferris 2025 12 9 Ppxlib Updates I managed to get a substantial amount of ppxlib work completed in the last month. Some of those bits are described in detail below. Patrick Ferris 2025 12 9 https://patrick.sirref.org/ppxlib-5.5-support ppxlib-5.5-support /ppxlib-5.5-support Initial OCaml 5.5 Support We recently merged 5.5 support into our main some new features coming to OCaml including the ability to define any structure locally. Most OCaml programmers will be familiar with things like let open M in External types have also landed, bringing with them a new "type kind" in the AST. You can read about them in the manual. Patrick Ferris 2025 12 9 Bug Fixes for Attribute Rewriters There was a bug in the error handling for attribute rewriters -- a relatively new feature to have been added to ppxlib. In some places, ppxlib makes use of a With_error.t monad. Unlike the Result.t monad, With_error.t allows you to collect errors rather than find the first error and thread it through your program. This can make sense if your program can meaningfully do work after encountering an error. Unfortunately, in the logic for replacing attributes, the code could not recover from an error where there were duplicate attributes. Patrick Ferris 2025 12 9 Value binding constraints in ppxlib I spent a good few hours understanding why the CI tests in my PR to mirage/repr were still failing. It turns out, the bug, is fairly simple to reproduce. Consider the following OCaml code: let g : int = 1 We have a let-binding that contains a type constraint. Where, in the abstract syntax tree sense, do we attach the constraint? Since OCaml 5.1, we have quite a few options for how this can be represented. The first is using the Ppat_constraint AST node which allows you to attach a type constraint to any pattern. | Ppat_constraint of pattern * core_type x + 1]]> So it doesn't quite capture what we want for this binding constraint. On older OCaml compilers, let g : int = 1 also includes a Pexp_constraint on the body of the value binding. This makes sense if we consider a more general case of the syntax where we allow functions. let g x : int = x + 1 In this example, we no longer parse a Ppat_constraint and instead this is treated as syntactic sugar for: let g x = x + 1 : int In OCaml 5.1 a new field for value bindings was added allowing type constraints to be tracked directly in the bindings themselves. and value_binding = pvb_pat: pattern; pvb_expr: expression; pvb_constraint: poly_constraint option; pvb_attributes: attributes; pvb_loc: Location.t; : int pvb_constraint directly. This is a nice change, bringing the concrete syntax of the language closer to the parsetree. For ppxlib, however, it does create a bit of a headache as we now have to encode the different versions of expressing type constraints into the AST of the compiler before 5.1! So what was the bug? The author's of repr had added some value bindings to circumvent some "unused value warnings" in OCaml. They choose to write these bindings like: let _ : int = 1 Here, I have simplified the types and the body of the binding. The significant thing to notice is the parentheses surrounding the value binding's pattern. This forces the AST representation to be a single Ppat_constraint node regardless of the current compiler version. No Pexp_constraint and no pvb_constraint. However, using ppxlib with a compiler less than 5.1 meant that this did not roundtrip cleanly. Instead, the value after passing through ppxlib's migrations was: let _ : int = 1 After bottoming out at the issue, I discovered we already had a test case for this in ppxlib! Whatsomore, ocamlformat.0.20.0 repr transforms instances of let x : t = ... into ! This limitation does not seem inherent to the parsetree, the printer or the migrations. But I have Patrick Ferris 2025 12 9 Outreachy December 8th 2025 marks the start of the next round of Outreachy. I have been preparing some issues for geocaml/ocaml-tiff and I am excited to get started with Tambe Salome! Patrick Ferris 2025 12 9 https://patrick.sirref.org/graft-and-bib-update/ graft-and-bib-update /graft-and-bib-update/ Graft and Bib I spent some time at the start of the month updating and fixing Graft and Bib. I added named subtrees to the markdown syntax. This allows users to create subtrees with headings that are also externally linkable. With some more bug fixing, I hope to release both libraries before the new year! Patrick Ferris 2025 11 5 https://patrick.sirref.org/ocaml-roundup-october-2025/ ocaml-roundup-october-2025 /ocaml-roundup-october-2025/ OCaml Roundup: October 2025 Welcome to a monthly roundup of OCaml-related open-source work I have been involved with. If you haven't already, a quick look at my ICFP roundup might be a good preface to what follows. Patrick Ferris 2025 11 5 Outreachy December 2025 The contribution period for this year's Outreachy round took place for most of the month of October. This year I am excited to be going back to a Geocaml project, working primarily to add writing capabilities to ocaml-tiff. As per usual, I have been extremely pleased with the level of interest in the various OCaml projects being offered. Whilst we are still deliberating on our choices for interns this year, I won't say much more on the contributions, but if you want to get involved please do reach out! OCaml is a great language for geospatial work, striking a balance between reasonable performance for numerical code and also a rich type system for expressing complicated geospatial data-structures. Patrick Ferris 2025 11 5 Ppxlib and the Future of Compiler Support Nathan has put a tremendous amount of effort into laying a roadmap this month for how ppxlib, going forward, will support future compilers. This is in response to the calamity that was bumping the internal AST to 5.2, and whilst I think we are adding a good deal more complexity, it will hopefully keep the ppx universe on an even keel for longer periods of time. The first substantial PR for this modern world of ppxlib is in review! Aside from that I also debugged an issue that lead to ppxlib OOM-ing machines... of course, any recursive fold-like loop where the accumulator keeps growing is likely to do that! It was nice to be able to crack out memtrace to be able to hunt it down: Patrick Ferris 2025 11 5 Incremental Directory Reading for Eio A common tasks for many a CLI tool is to scan a directory of files, filter out the ones you are looking for and apply some transformation to said files. In fact, that is what Graft and Forester both do! A standard approach to this involves a recursive dance of readdir and stat, the former lists the entries in a directory whilst the latter lets you know what kind of file is at a particular path We're making two If a directory contains thousands of files, we end up allocating a big list of strings after returning from readdir. On Linux, we can get around most of that with which allows us to incrementally read a directory. Whatsomore, this system call also returns the file type so there's no need to do an extra stat! I prototyped an implementation of this called Eio.Path.walk which looks like: val walk : _ t -> File.Stat.kind * string Seq.t -> 'a -> 'a We effectively halve the number of system calls. In a benchmark that traverse deeply-nested directories with plenty of files, a full traversal whent from 1.46s to 110ms. I have since rejigged the implementation to automatically which does make it tricky to control the number of open file descriptors compared to if the user were in control. Patrick Ferris 2025 11 5 https://patrick.sirref.org/icfp-oxcaml-uring/ icfp-oxcaml-uring /icfp-oxcaml-uring/ OxCaml Experiments I have included here a transclusion from my ICFP post about an experiment using unboxed 32-bit integers in OCaml's Uring library. Patrick Ferris https://patrick.sirref.org/oxcaml-and-uring/ oxcaml-and-uring /oxcaml-and-uring/ OxCaml and Uring Jane Street were a big presence at ICFP 2025, carting along with them a shiny new OCaml compiler: OxCaml. If have been playing around with OxCaml recently but nothing outside toplevels in Javascript. Until now! After talking to David, I spent some time converting a small corner of the ocaml-uring library to use a part of OxCaml. In particular making the following change: module Heap = struct type ptr = int32# ... end type cqe = user_data_id : Heap.ptr; res : int32#; unboxed values You can see how this impacted the library! to realise OxCaml gives you this kind of control. However, I am worried about the ergonomics of manipulating values like int32# and the temptation to case it into an int Patrick Ferris 2025 11 5 x-ocaml with OxCaml I sent a few PRs to the OxCaml repository to get the toplevel working again and lucky the underlying culprit for why patches were disappearing was found. Watch this space for a new toplevel in the browser soon. Patrick Ferris 2025 11 5 Geospatial OCaml In addition to the Outreachy work I mentioned above, I also found the time to keep the Geocaml project ticking along. A few projects saw some much-needed love including: Patrick Ferris 2025 11 5 PROJ in OCaml A modernisation of the bindings to PROJ. PROJ is probably the most used and tested code for working with geospatial projections. Any time you get a sateillite image or a blob of GeoJSON, the it detailing exactly what the numbers that tell you where something is mean. One of the most common being WGS84 will need to convert between them. That is what PROJ does. Patrick Ferris 2025 11 5 WKT in OCaml For example, a polygon might look like: I used bytesrw to build a simple codec for the WKT format. It is probably about time to write the basic geospatial object library too, something similar to Rust's geo library. Patrick Ferris 2025 11 5 Geotessera A lot of people in my group are excited about Tessera. They have made it easy to used the Tessera embeddings via a python library called geotessera. In true OCaml-at-heart spirit, I tried my hand at porting this library. I had a good deal of success, converting the data into Nx array, extracting the CRS and transform out of the GeoTIFF landmasks using ocaml-tiff, reading datasets using ocaml-geojson etc. This made me excited but also keen to improve the APIs of most of these libraries too. I feel geocaml is at a critical mass where we can start to have this useful feedback loop for the libraries because there is enough functionality to actually use them! Patrick Ferris 2025 9 2 https://patrick.sirref.org/ocaml-weekly-2025-w32-w35/ ocaml-weekly-2025-w32-w35 /ocaml-weekly-2025-w32-w35/ OCaml Weekly 2025 w32 to w35 I have been working on a few different OCaml-related projects over the last few weeks. This is also coinciding with my partner and I moving across the UK, which has made finding time to write weeklies and posts a little tricky. Nevertheless, here are some of the things I have been thinking about and working on! I managed to publish one signficant post this month: a retrospective on Irmin. Patrick Ferris 2025 9 2 Eio Increasingly, I'm feeling the dream of a unified framework for asynchronous IO slipping through the OCaml community's fingers. It is perhaps not such a bad thing, and I think with the right library authoring we can at least get to a place where it isn't so bad, for example providing read functions as opposed to using an opinionated IO library directly. That being said, I am a very happy user of Eio when those choices do not matter, Shelter I have spent a good bit of time upstreaming support for various missing pieces in Eio's API including: Setuid and setgid fork action's for the process API. Set process group support for job control in the process API. Responding to Buf_write.of_flow request, and tinkering with the example there. I think this does highlight the awkwardness of making code portable across concurrency mechanisms, particularly with Eio's structured concurrency. I did some investigating into EINTR bug which seems to be stemming from a known-issue on Uring in that writes are not buffered which usually does not matter except perhaps when there are parallel writes to stdout. Spent some time thinking about the fiber local storage across domains issue, I've passed on some thoughts to folks working on this. Patrick Ferris 2025 9 2 Vpnkit You might recall I was interested in using vpnkit. Hannes has done an amazing amount of work much better and could be considered soon for merging. This defunctorisation is actually very useful for the Eio port I wrote a long time ago. Patrick Ferris 2025 9 2 Papers and Talks at ICFP Somehow, I have ended up on lots of papers and talks at ICFP and the co-located events in October. The vaguely OCaml-related ones include: Essentially a Vpnkit Experience Report. An extended abstract on generating a corpus of ill-typed Hazel programs was accepted into TyDe workshop. Relatedly, the work that project supported was accepted into HATRA which was the Part II project I supervised: Decomposable Type Highlighting for Bidirectional Type and Cast Systems. And two PROPL talks! Patrick Ferris 2025 9 2 Outreachy We have come to the end of another Outreachy round! I will write more on this soon in its own separate post. But for now I am very grateful to this round's mentors gridbugs and mdales, and also our fantastic interns. If you are interested, please do watch our demo day presentations. The next round is fast approaching and we still need to work out the logistics. But I had a good conversation with mdales about possible Geocaml projects that I intend to submit! Patrick Ferris 2025 7 31 https://patrick.sirref.org/ocaml-weekly-2025-w30-w31/ ocaml-weekly-2025-w30-w31 /ocaml-weekly-2025-w30-w31/ OCaml Weekly 2025 w30 and w31 A bumper edition today, a fortnightly. Patrick Ferris 2025 7 31 Opam Releases I did a little personal spring-cleaning of packages I maintain and pushed a few releases to opam. This included a 0.1.0 release of graft! There is some light documentation online at https://graft.sirref.org. Elsewhere, Hilite got a 0.5.0 release. This release makes the core syntax highlighting separate from the markdown part of the library. You can now generate pairs of tokens and identifiers graft Patrick Ferris 2025 7 31 Interactive OCaml Presentations I had fun trying to fuse slipshow and x-ocaml! 2025 7 23 https://patrick.sirref.org/slipshow-x-xocaml/ slipshow-x-xocaml /slipshow-x-xocaml/ Slipshow x x-ocaml A short, explanatory post about combining two very fun pieces of work in OCaml. Paul-Elliot has been building Slipshow for some time now where slides are slips and your presentations run vertically. More recently, Arthur has built x-ocaml, a web component library for executable OCaml cells embedded into OCaml. Using xocmd, a small tool I built for translating markdown codeblocks to x-ocaml components, your Slipshow's can now be executable! learn-effects.html]]> Take a look at an example! (or the source markdown). I really like this light-weight approach to building interactive presentations I tried using x-ocaml with Irmin and something is not see the issue in case you can help intended to be used alongside a longer-form retrospective I am writing on my use of Irmin over the years. Patrick Ferris 2025 7 31 Outreachy I have been helping Michael's intern whilst he has been away on their project Claudius. It has reminded me, again, how inpenetretable some of OCaml's tooling is. In this case the generation of opam files from dune-project files, and in particular, the required manual the dune-project file, 2. run dune build to update the opam file, 3. remove all occurences of the library from dune Patrick Ferris 2025 7 31 Ppxlib Work on Ppxlib has been fairly varied this past two weeks. I mentioned before about an exiting plan Nathan has planned to help ease the burden on ppxlib maintainers and ppx authors whenever there is an OCaml parsetree bump. That, however, is currently shelved as we are dealing with the fall out of the 5.2 bump. Repr got some fixes pushed to it, but this unconvered more issues. Patrick Ferris 2025 7 31 Misc. I spent some time cleaning up container-image, a tool primarily written by Thomas G. to fetch OCI images from repositories. The tidy up went surprisingly well, except for some spurious HTTP errors with AWS. Maybe I'll spend some Patrick Ferris 2025 7 15 https://patrick.sirref.org/ocaml-weekly-2025-w29/ ocaml-weekly-2025-w29 /ocaml-weekly-2025-w29/ OCaml Weekly 2025 w29 Patrick Ferris 2025 7 15 Ppxlib I met with Nathan this week to discuss future plans for ppxlib. The current state of affairs is that ppxlib is becoming unmaintainable. This is primarily a knock-on effect from changes being made to Nathan has a plan that will provide two key properties. Migrations, which allow old compilers to be used with new ppxlib releases, will be more compatible. For example, we will be able to migrate new features downwards and back up without raising an error. Ppx authors will be able to use new features in an opt-in workflow, rather than ppxlib in ppxlib.0.36.0 significantly whilst still allowing users to write ppxes for new OCaml features. I also started looking into some older issues in ppxlib related to performance. This is work-in-progress, but I am trying to improve the performance of some passes done by ppxlib. To better understand what was making ppxlib slow, I wanted to use runtime_events_tools but I was dismayed to see it wanting to install over 100 packages! I opened a PR to reduce the number of packages. I think this kind of work goes a little unrecognised as it is not very glamorous. However, I think it really benefits the OCaml community in the long run. Patrick Ferris 2025 7 15 Outreachy In Outreachy news, we had a wonderful mid-internship video call with all the interns and mentors to catch-up on how everyone is getting along. Seeing the progress everyone has made was great! I am very grateful for the work that Michael and Steve have put in so far to make this a very successful Outreachy round for OCaml. In sadder news, an email was shared with all Outreachy mentors detailing the increasingly critical financial situation the project finds itself in. There are ongoing discussions about how costs can be cut including potentially only running a single round a year. Patrick Ferris 2025 7 15 Graft With the release of Forester.5.0, I made a plan to make a release of Graft.0.1. Unfortunately this is blocked by a new release of hilite, a tool I built for doing build-time syntax highlighting for OCaml code. This powers the syntax highlighting on ocaml.org. https://patrick.sirref.org/teaching/ teaching /teaching/ Teaching I supervise multiple courses in the Department of Computer Science including: https://patrick.sirref.org/focs/ focs /focs/ Foundations of Computer Science https://www.cl.cam.ac.uk/teaching/2425/FoundsCS/ Foundations of Computer Science is a first-year course taught in the Department of Computer Science. It is currently lectured by Anil Madhavapeddy and taught in OCaml. https://patrick.sirref.org/discrete-maths/ discrete-maths /discrete-maths/ Discrete Mathematics https://www.cl.cam.ac.uk/teaching/2425/DiscMath/ Discrete Mathematics is a first-year course taught in the Department of Computer Science. It is currently lectured by Jon Sterling. https://patrick.sirref.org/part-ii/ part-ii /part-ii/ Part II Projects Part II is the name for third year students studying Computer Science at Cambridge. Over the course of the year, a student is tasked with building a substantial piece of software culminating in a dissertation-style write up at the end. I did this back in 2020, you can read about that here. 2023 9 10 https://patrick.sirref.org/part-ii-2023/ part-ii-2023 /part-ii-2023/ Part II Students 2023 In the final year of the Cambridge undergraduate degree in computer science, students undertake a coding project with an accompanying dissertation. https://patrick.sirref.org/part-ii-dslid/ part-ii-dslid /part-ii-dslid/ A DSL for Decentralised Identity in OCaml https://github.com/majkimge/DecentralisedDigitalIdentity This project was completed by Michał Mgeładze-Arciuch and supervised by Anil Madhavapeddy and Patrick Ferris. 2024 9 10 https://patrick.sirref.org/part-ii-2024/ part-ii-2024 /part-ii-2024/ Part II Students 2024 In the final year of the Cambridge undergraduate degree in computer science, students undertake a coding project with an accompanying dissertation. https://patrick.sirref.org/part-ii-hazel/ part-ii-hazel /part-ii-hazel/ Type Error Debugging in Hazel This project will add some features to the Hazel language. Hazel https://patrick.sirref.org/part-ii-udsl/ part-ii-udsl /part-ii-udsl/ A Universal Data Access DSL https://patrick.sirref.org/part-ii-paraeff/ part-ii-paraeff /part-ii-paraeff/ Implementing Parallel Algebraic Effect Handlers in OCaml https://patrick.sirref.org/part-ii-geocaml/ part-ii-geocaml /part-ii-geocaml/ Effective Geospatial Code in OCaml This project is co-supervised with Michael Dales. Geospatial processing is critical in environmental science and is used in a range of environmental scenarios, both in areas of carbon accounting like digital Monitoring, Reporting and Verification in Reforestation Carbon Removal, and also biodiversity analysis like calculating worldwide extinction rates. https://patrick.sirref.org/publications/ publications /publications/ Publications Various talks, papers and preprints. Published Conference Papers Anil Madhavapeddy David J. Scott Patrick Ferris Ryan T. Gibb Thomas Gazagnaire 2025 https://patrick.sirref.org/madhavapeddy2025docker/ madhavapeddy2025docker /madhavapeddy2025docker/ Functional Networking for Millions of Docker Desktops (Experience Report) Reference https://dl.acm.org/doi/10.1145/3747525 Proc. ACM Program. Lang. 10.1145/3747525 Anil Madhavapeddy David J. Scott Patrick Ferris Ryan T. Gibb Thomas Gazagnaire 2025 Abstract Docker is a developer tool used by millions of developers to build, share and run software stacks. The Docker Desktop clients for Mac and Windows have long used a novel combination of virtualisation and OCaml unikernels to seamlessly run Linux containers on these non-Linux hosts. We reflect on a decade of shipping this functional OCaml code into production across hundreds of millions of developer desktops, and discuss the lessons learnt from our experiences in integrating OCaml deeply into the container architecture that now drives much of the global cloud. We conclude by observing just how good a fit for systems programming that the unikernel approach has been, particularly when combined with the OCaml module and type system. Workshop Papers Patrick Ferris Anil Madhavapeddy 2025 https://patrick.sirref.org/ferris2025hazel_of_ocaml/ ferris2025hazel_of_ocaml /ferris2025hazel_of_ocaml/ Generating a corpus of Hazel programs from ill-typed OCaml programs Reference https://patrick.sirref.org/var/tyde2025.pdf Patrick Ferris Anil Madhavapeddy 2025 Abstract When developing a new programming language, having a large corpus of both correct and incorrect programs allows language designers to test and explore the capabilities of their new language. However, bootstrapping such a corpus of incorrect programs is time-consuming and arduous. We therefore explore how to reuse code from more mature languages to generate a corpus of ill-typed code for newer ones. We have developed a compiler to Hazel, an emerging language with typed holes, from the more mature OCaml ecosystem. We find it practical to generate a comprehensive corpus of ill-typed programs for Hazel development, and discuss future larger scale efforts towards bridging ecosystems. Max Carroll Anil Madhavapeddy Patrick Ferris 2025 https://patrick.sirref.org/carroll2025decomposable_type_highlighting/ carroll2025decomposable_type_highlighting /carroll2025decomposable_type_highlighting/ Decomposable Type Highlighting for Bidirectional Type and Cast Systems Reference https://maxcarroll0.github.io/papers/workshops/HATRA-decomposable-type-highlighting/ Max Carroll Anil Madhavapeddy Patrick Ferris 2025 Abstract We explore how to provide programmers with an interactive interface for explaining the process by which static types and dynamic casts are derived, with the goal of improving the debugging of static and dynamic type errors. To this end, we define mathematical foundations for a decomposable highlighting system within a bidirectional system, and show how these can be propagated through dynamic types in a cast system. Our prototype implementation in the gradually typed Hazel language includes a web-based user interface, through which we highlight the importance of type level debugging. Michael Winston Dales Alison Eyres Patrick Ferris Francesca A. Ridley Simon Tarr Anil Madhavapeddy 2025 https://patrick.sirref.org/dales2025yirgacheffe/ dales2025yirgacheffe /dales2025yirgacheffe/ Yirgacheffe: A Declarative Approach to Geospatial Data Reference https://dl.acm.org/doi/10.1145/3759536.3763806 10.1145/3759536.3763806 Michael Winston Dales Alison Eyres Patrick Ferris Francesca A. Ridley Simon Tarr Anil Madhavapeddy 2025 Abstract Cyrus Omar Patrick Ferris Anil Madhavapeddy 2024 https://patrick.sirref.org/omar2024bastion/ omar2024bastion /omar2024bastion/ Modularizing Reasoning about AI Capabilities via Abstract Dijkstra Monads Reference https://icfp24.sigplan.org/home/hope-2024 Cyrus Omar Patrick Ferris Anil Madhavapeddy 2024 Abstract Ryan Gibb Patrick Ferris Anil Madhavapeddy 2024 https://patrick.sirref.org/gibb2024carbonaware/ gibb2024carbonaware /gibb2024carbonaware/ Carbon-Aware Name Resolution Reference https://www.sicsa.ac.uk/wp-content/uploads/2024/11/LOCO2024_paper_28.pdf 1st International Workshop on Low Carbon Computing Ryan Gibb Patrick Ferris Anil Madhavapeddy 2024 Abstract The current Internet architecture fails to treat the carbon emissions associated with networked services as a first-class metric. We propose extending the DNS with load balancing techniques to consider the carbon cost of scheduling decisions, and further to actively wake machines running networked services as a side effect of name resolution. By extending the DNS, we maintain compatibility with existing Internet infrastructure, unlocking the ability for existing applications to be carbon-aware. Michael Dales Patrick Ferris Anil Madhavapeddy 2024 https://patrick.sirref.org/dales2024frugallinux/ dales2024frugallinux /dales2024frugallinux/ Lineage first computing: towards a frugal userspace for Linux Reference https://www.sicsa.ac.uk/wp-content/uploads/2024/11/LOCO2024_paper_30.pdf 1st International Workshop on Low Carbon Computing Michael Dales Patrick Ferris Anil Madhavapeddy 2024 Abstract Data-science is a vital tool in tackling the ongoing climate crisis, but it is one that has a significant cost to it also, in terms of energy used during execution and in terms of significant hardware investment required to run it. These are then amplified considerably in the exploratory nature of scientific research.We argue this is because the affordances of the operating system, particularly filesystems, make little effort to support reuse of computed artifacts directly or promote reuse through trusted lineage data. However, it turns out that many of the userspace interfaces exposed by the Linux kernel entirely support a radically more efficient -- a more frugal -- model of computation than is currently the default. In this talk, we explore a new OS architecture which defaults to deterministic, reusable computation with the careful recording of side effects. This in turn allows the OS to guide complex computations towards previously acquired intermediate results, but still allowing for recomputation when required. We use these interfaces to build a new Linux userspace where we put the workflow graph-containing relationships between tools, provenance and labelling-as the core of a system that drives how processes, data, and users interact. Our prototype shell, dubbed Shark, is a data-science first operating environment that is designed to both ensure efficient use of computation and storage resources, and to make it easy for non-experts to create pipeline descriptions from end results post-hoc. It does this by making the lineage graph of a data-pipeline the key concept that ties everything else together: by tracking how the data-pipeline evolves from experimental practice, and tracking what data has already been built and what hasn't we can prevent re-execution both at development and after publication. Jessica Man Sadiq Jaffer Patrick Ferris Martin Kleppmann Anil Madhavapeddy 2024 https://patrick.sirref.org/man2024emissionimpossible/ man2024emissionimpossible /man2024emissionimpossible/ Emission Impossible: privacy-preserving carbon emissions claims Reference https://www.sicsa.ac.uk/wp-content/uploads/2024/11/LOCO2024_paper_35.pdf 1st International Workshop on Low Carbon Computing Jessica Man Sadiq Jaffer Patrick Ferris Martin Kleppmann Anil Madhavapeddy 2024 Abstract Thomas Leonard Patrick Ferris Christiano Haesbaert Lucas Pluvinage Vesa Karvonen Sudha Parimala K Sivaramakrishnan Vincent Balat Anil Madhavapeddy 2023 https://patrick.sirref.org/leonard2023eio/ leonard2023eio /leonard2023eio/ Eio 1.0-effects-based IO for OCaml 5 Reference Thomas Leonard Patrick Ferris Christiano Haesbaert Lucas Pluvinage Vesa Karvonen Sudha Parimala K Sivaramakrishnan Vincent Balat Anil Madhavapeddy 2023 Abstract Eio provides an effects-based direct-style IO stack for OCaml 5. This talk introduces Eio's main features, such as use of effects, multicore support and lock-free data-structures, support for modular programming, interoperability with other concurrency libraries such as Lwt, Async and Domainslib, and interactive monitoring support enabled by the custom runtime events in OCaml 5.1. We will report on our experiences porting existing applications to Eio. Sadiq Jaffer Patrick Ferris 2022 https://patrick.sirref.org/jaffer2022runtimeevents/ jaffer2022runtimeevents /jaffer2022runtimeevents/ Continuous Monitoring of OCaml Applications using Runtime Events Reference https://icfp22.sigplan.org/details/ocaml-2022-papers/7/Continuous-Monitoring-of-OCaml-Applications-using-Runtime-Events Sadiq Jaffer Patrick Ferris 2022 Abstract The upcoming 5.0 release of OCaml includes a new runtime tracing system designed for continuous monitoring of OCaml applications called Runtime Events. It enables very low overhead, programmatic access to performance data emitted by the OCaml runtime and garbage collector. This talk focuses on the implementation of Runtime Events and the user experience of writing applications exploiting this new feature. Misc. Patrick Ferris 2025 https://patrick.sirref.org/ferris2025scientific_programming/ ferris2025scientific_programming /ferris2025scientific_programming/ What we talk about when we talk about scientific programming Reference https://watch.eeg.cl.cam.ac.uk/w/aYXqXLtgQawYMjVXjSQtjx Patrick Ferris 2025 Abstract Sadiq Jaffer Michael Dales Patrick Ferris Derek Sorensen Thomas Swinfield Robin Message Srinivasan Keshav Anil Madhavapeddy 2024 https://patrick.sirref.org/jaffer2024global/ jaffer2024global /jaffer2024global/ Global, robust and comparable digital carbon assets Reference 10.1109/ICBC59979.2024.10634343 Sadiq Jaffer Michael Dales Patrick Ferris Derek Sorensen Thomas Swinfield Robin Message Srinivasan Keshav Anil Madhavapeddy 2024 Abstract Preprints Patrick Ferris Michael Dales Sadiq Jaffer Amelia Holcomb Eleanor Scott Thomas Swinfield Alison Eyres Andrew Balmford David Coomes Srinivasan Keshav Anil Madhavapeddy 2023 https://patrick.sirref.org/ferris2023planetary/ ferris2023planetary /ferris2023planetary/ Planetary computing for data-driven environmental policy-making Reference Arxiv 10.48550/arXiv.2303.04501 Patrick Ferris Michael Dales Sadiq Jaffer Amelia Holcomb Eleanor Scott Thomas Swinfield Alison Eyres Andrew Balmford David Coomes Srinivasan Keshav Anil Madhavapeddy 2023 Abstract We make a case for planetary computing -- infrastructure to handle the ingestion, transformation, analysis and publication of global data products for furthering environmental science and enabling better informed policy-making. We draw on our experiences as a team of computer scientists working with environmental scientists on forest carbon and biodiversity preservation, and classify existing solutions by their flexibility in scalably processing geospatial data, and also how well they support building trust in the results via traceability and reproducibility. We identify research gaps in the intersection of computing and environmental science around how to handle continuously changing datasets that are often collected across decades and require careful access control rather than being fully open access. Andrew Balmford David Coomes Michael Dales Patrick Ferris James Hartup Sadiq Jaffer Srinivasan Keshav Miranda Lam Anil Madhavapeddy Robin Message 2023 https://patrick.sirref.org/balmford2023pact/ balmford2023pact /balmford2023pact/ PACT Tropical Moist Forest Accreditation Methodology v2.0 Reference Cambridge Open Engage 10.33774/coe-2023-g584d-v5 Andrew Balmford David Coomes Michael Dales Patrick Ferris James Hartup Sadiq Jaffer Srinivasan Keshav Miranda Lam Anil Madhavapeddy Robin Message 2023 Abstract https://patrick.sirref.org/weeklies/ weeklies /weeklies/ Patrick Ferris' Weeklies feed of OCaml-related work, please see OCaml weeklies. My personal weeklies will include the OCaml specific ones too. An RSS feed is available. Patrick 2025 11 24 https://patrick.sirref.org/weekly-2025-w49/ weekly-2025-w49 /weekly-2025-w49/ Data Provenance in Shelter A late update that stretches more than a week... https://shelter.sirref.org/Patrick Ferris/ 2025 12 9 https://shelter.sirref.org/shelter-log-002/ https://shelter.sirref.org/shelter-log-002/ https://shelter.sirref.org/shelter-log-002/ @ How? A quick update on a new feature that was added recently, the @ how meta-command. provenance of a given file. How that file came to be and who and what processes have interacted with it since. Shelter now has a command for asking exactly that of any given file in the system. hello.txt echo world > world.txt cat world.txt > hello.txt @ how hello.txt]]> The previous snippet of shell script will output: hello.txt Ran cat world.txt > hello.txt (read: "world.txt")]]> There are plenty of optimisations and improvements to make, but this is a nice first step to showing the kinds of feature you get when programming with Shelter. Patrick Ferris 2025 12 9 https://patrick.sirref.org/ocaml-roundup-november-2025/ ocaml-roundup-november-2025 /ocaml-roundup-november-2025/ OCaml Roundup: November 2025 A roundup of some of my OCaml activity this month! Patrick Ferris 2025 12 9 Ppxlib Updates I managed to get a substantial amount of ppxlib work completed in the last month. Some of those bits are described in detail below. Patrick Ferris 2025 12 9 https://patrick.sirref.org/ppxlib-5.5-support ppxlib-5.5-support /ppxlib-5.5-support Initial OCaml 5.5 Support We recently merged 5.5 support into our main some new features coming to OCaml including the ability to define any structure locally. Most OCaml programmers will be familiar with things like let open M in External types have also landed, bringing with them a new "type kind" in the AST. You can read about them in the manual. Patrick Ferris 2025 12 9 Bug Fixes for Attribute Rewriters There was a bug in the error handling for attribute rewriters -- a relatively new feature to have been added to ppxlib. In some places, ppxlib makes use of a With_error.t monad. Unlike the Result.t monad, With_error.t allows you to collect errors rather than find the first error and thread it through your program. This can make sense if your program can meaningfully do work after encountering an error. Unfortunately, in the logic for replacing attributes, the code could not recover from an error where there were duplicate attributes. Patrick Ferris 2025 12 9 Value binding constraints in ppxlib I spent a good few hours understanding why the CI tests in my PR to mirage/repr were still failing. It turns out, the bug, is fairly simple to reproduce. Consider the following OCaml code: let g : int = 1 We have a let-binding that contains a type constraint. Where, in the abstract syntax tree sense, do we attach the constraint? Since OCaml 5.1, we have quite a few options for how this can be represented. The first is using the Ppat_constraint AST node which allows you to attach a type constraint to any pattern. | Ppat_constraint of pattern * core_type x + 1]]> So it doesn't quite capture what we want for this binding constraint. On older OCaml compilers, let g : int = 1 also includes a Pexp_constraint on the body of the value binding. This makes sense if we consider a more general case of the syntax where we allow functions. let g x : int = x + 1 In this example, we no longer parse a Ppat_constraint and instead this is treated as syntactic sugar for: let g x = x + 1 : int In OCaml 5.1 a new field for value bindings was added allowing type constraints to be tracked directly in the bindings themselves. and value_binding = pvb_pat: pattern; pvb_expr: expression; pvb_constraint: poly_constraint option; pvb_attributes: attributes; pvb_loc: Location.t; : int pvb_constraint directly. This is a nice change, bringing the concrete syntax of the language closer to the parsetree. For ppxlib, however, it does create a bit of a headache as we now have to encode the different versions of expressing type constraints into the AST of the compiler before 5.1! So what was the bug? The author's of repr had added some value bindings to circumvent some "unused value warnings" in OCaml. They choose to write these bindings like: let _ : int = 1 Here, I have simplified the types and the body of the binding. The significant thing to notice is the parentheses surrounding the value binding's pattern. This forces the AST representation to be a single Ppat_constraint node regardless of the current compiler version. No Pexp_constraint and no pvb_constraint. However, using ppxlib with a compiler less than 5.1 meant that this did not roundtrip cleanly. Instead, the value after passing through ppxlib's migrations was: let _ : int = 1 After bottoming out at the issue, I discovered we already had a test case for this in ppxlib! Whatsomore, ocamlformat.0.20.0 repr transforms instances of let x : t = ... into ! This limitation does not seem inherent to the parsetree, the printer or the migrations. But I have Patrick Ferris 2025 12 9 Outreachy December 8th 2025 marks the start of the next round of Outreachy. I have been preparing some issues for geocaml/ocaml-tiff and I am excited to get started with Tambe Salome! Patrick Ferris 2025 12 9 https://patrick.sirref.org/graft-and-bib-update/ graft-and-bib-update /graft-and-bib-update/ Graft and Bib I spent some time at the start of the month updating and fixing Graft and Bib. I added named subtrees to the markdown syntax. This allows users to create subtrees with headings that are also externally linkable. With some more bug fixing, I hope to release both libraries before the new year! Patrick Ferris 2025 11 15 https://patrick.sirref.org/weekly-2025-w46/ weekly-2025-w46 /weekly-2025-w46/ Shelter imports Most of this week was spent fixing bugs in Shelter in an attempt to get it ready to run the LIFE pipeline. To do so, Shelter requires a mechanism by which to pull in files from the outside world. That could be configuration files, source trees, data etc. For now, this has taken the form of a crude @ import statement. In true Forester fashion, I have transcluded the foreign tree from my Shelter logs. https://shelter.sirref.org/Shelter Programmers/ 2025 11 16 https://shelter.sirref.org/shelter-log-001/ https://shelter.sirref.org/shelter-log-001/ https://shelter.sirref.org/shelter-log-001/ Import statements Welcome to the first Shelter log! In this log I will explain the work I have been doing to add an @ import meta-command. The syntax thus far is relatively straightforward. ]]> There are a few key principles underlying @ import. It should be at least as expressive as Docker's COPY ... command. It should deal with a wider variety of import sources. In addition to local https:// archives over ssh etc. Imports, where possible, should be catalogued and shared across sessions. https://shelter.sirref.org/Shelter Programmers/ 2025 11 16 URIs as Sources branch already allows users to import local files and directories into their session. For example: By allowing arbitrary URIs in the import command, we hope to force users away from downloading data via curl or python scripts. Lifting these side-effectful imports into Shelter allows us to manage the data in a much cleaner fashion. https://shelter.sirref.org/Shelter Programmers/ 2025 11 16 Scripts as Sources One idea I had was to allow users to invoke scripts as there "import source". Something like: And using the same eBPF tracing we use during normal execution we can capture a fairly good idea of the tools and files needed to perform an import. https://shelter.sirref.org/Shelter Programmers/ 2025 11 16 Sharing Imports One example of better data management is that Shelter imports can be When a user imports some data, they can optionally name that piece of data. For example: Shelter stores that away and makes a link between the name, the URI and the underlying data. A user may then import the same piece of data into a different session by simply writing: intent via the naming scheme, it makes it possible to then version the data and have a means by which to update said data, one could imagine: Which will retry the original URI and see if the data has changed, or if it has not. There is an implicit versioning number that users can specify in the name should they choose to. Patrick Ferris 2025 11 15 https://patrick.sirref.org/eio-cap-debugging/ eio-cap-debugging /eio-cap-debugging/ Eio Debugging One of the promises of Eio's capability model is an improved debugging experience. Your program must be designed in such a way as to make resources explicit. If you have a load_config function, more than likely it will need access to the file-system. val load_config : _ Eio.Path.t -> Config.t Once all of your resources are explicit and threaded through your program, it This week I was trying to get the LIFE pipeline running in shelter. Part of that involves pulling the appropriate geospatial container image, but my code was terrifically broken, failing with what looked to me with a Docker error. With only this clue, I knew that it was probably in the execution of some command in a child process, but which one? To trace this quickly, I could plug in a slightly different instance of the _ Eio.Process.mgr capability. Essentially unchanged from the one provided by the unix backend except that it would also print to stderr the command it was trying to run! let debug_process_mgr mgr : 'a Eio_unix.Process.mgr : 'a Eio_unix.Process.mgr = let module D = struct type t = unit let spawn_unix ~sw ?cwd ?pgid ?uid ?gid ~env ~fds ~executable args = Eio.traceln "Spawning subprocess... " Fmt. list string args; Eio_unix.Process.spawn_unix ~sw ?cwd ?pgid ?uid ?gid mgr ~env ~fds ~executable args end in let module V = Eio_unix.Process.Make_mgr D in Eio.Resource.T , Eio_unix.Process.Pi.mgr_unix module V Whist there is a fair amount of jumping through first-class-module-shaped hoops here, all I am doing is rewrapping an existing "method" to a spawn a unix child process. Then using this helper function, I can push that into my standard environment. let env = Eio_unix.Stdenv.with_env ~process_mgr: debug_process_mgr env#process_mgr env in Shelter.main config env :> _ Shelter.env dir cmd_file And sure enough I was querying for the Docker user before pulling the image! I hadn't run into this, as for the last while I have been using alpine and debian images that already exists locally. Patrick Ferris 2025 11 10 https://patrick.sirref.org/weekly-2025-w45/ weekly-2025-w45 /weekly-2025-w45/ Taking stock & Graft updates I ended up working on a real mix of things last week. I managed to get two large posts uploaded to this site: an ICFP recap and a deeper-dive into the random bits of OCaml hacking. This was pretty useful in terms of taking stock of where I was at, I always find post-conference, I'm a little lost. The conversations and ideas from ICFP are simultaneously exciting but also pretty inactionable. Patrick Ferris 2025 11 10 https://patrick.sirref.org/graft-updates/ graft-updates /graft-updates/ Graft Updates It would seem that there is a potential second user of Graft! A great source of motivation for some tidying and feature implementing. At the end of last week I fixed some little bugs in Graft users to name their subtrees. Upstream Graft also now pins and uses my Bibtex implementation. Patrick Ferris 2025 11 10 OCaml Geotessera As I discussed in my OCaml roundup, I spent a bit of time polishing the OCaml geotessera code. In particular this meant starting to handle low-level geospatial transformations allowing users to project latitudes and longitudes into the rows and columns of their raster data. I feel like I'm stumbling onto a path Michael has carved out with Yirgacheffe. I'm following along with his work excitedly and we had a good catch-up last week discussing how libraries should handle things like automatic, numeric precision conversion. As a fully paid-up member of the static types fan club, I'm often evangelising the benefits of types. However, I hit a little quirk of the Nx sum function: let sum nx = Nx.sum nx |> Nx.item ;; val sum : 'a, 'b Nx.t -> 'a let arr = Nx.create Nx.int8 2 64; 64 ;; val arr : int, Bigarray.int8_signed_elt Nx.t sum arr;; - : int = -128 Even though sum takes an array that stores elements with representation 'b and allows you to read/write to the elements with OCaml values of type 'a, the sum function performs the calculation in the world of the 'b overflow! Perhaps not too surprising but easy to overlook! The moral of the story is that we still need to be careful in the OCaml world too. Patrick Ferris 2025 11 10 Shelter I did a little cleaning up of the Shelter code too. I noticed a few bugs from the demos I did at ICFP. Most were pretty straight-forward to fix, and the test-suite is now checking parallel for-loops and conditional execution are working. Patrick Ferris 2025 11 10 This week I hope to get a few things done this week: Run some part of the LIFE pipeline through Shelter. Deploy the Shelter daemon allowing other people to connect and take it for a spin. Spend some time writing and reading, pulling ideas together rather than always writing code! I have a slightly harebrained scheme that combines Shelter, atproto, Forester pull that together. 2025 7 18 https://patrick.sirref.org/weekly-2025-07-18/ weekly-2025-07-18 /weekly-2025-07-18/ Back to shelter 2025 7 18 Capnproto Interface to Shelter I spent some time this week thinking about decoupling Shelter's builds from the CLI tool. This would allow users to connect to a remote daemon to build and store their sessions, opening up the possibility of using Shelter on Windows and macOS. The design so far keeps the metadata local, in an Irmin database. I will probably then augment the stored information with some kind of unique daemon identifier to know where the actual data is. This is all still quite fresh, and I will have more by the end of next week. 2025 7 18 OCaml-related Things Something I decided to do last week, was to make a clearer delineation between my OCaml work and research. That line is not always very distinct, but I have split of a separate set of weeklies for OCaml related things. Fear not, the OCaml weeklies are a subset of these weeklies and so I will, when I remember, transclude them for you. Patrick Ferris 2025 7 15 https://patrick.sirref.org/ocaml-weekly-2025-w29/ ocaml-weekly-2025-w29 /ocaml-weekly-2025-w29/ OCaml Weekly 2025 w29 Patrick Ferris 2025 7 15 Ppxlib I met with Nathan this week to discuss future plans for ppxlib. The current state of affairs is that ppxlib is becoming unmaintainable. This is primarily a knock-on effect from changes being made to Nathan has a plan that will provide two key properties. Migrations, which allow old compilers to be used with new ppxlib releases, will be more compatible. For example, we will be able to migrate new features downwards and back up without raising an error. Ppx authors will be able to use new features in an opt-in workflow, rather than ppxlib in ppxlib.0.36.0 significantly whilst still allowing users to write ppxes for new OCaml features. I also started looking into some older issues in ppxlib related to performance. This is work-in-progress, but I am trying to improve the performance of some passes done by ppxlib. To better understand what was making ppxlib slow, I wanted to use runtime_events_tools but I was dismayed to see it wanting to install over 100 packages! I opened a PR to reduce the number of packages. I think this kind of work goes a little unrecognised as it is not very glamorous. However, I think it really benefits the OCaml community in the long run. Patrick Ferris 2025 7 15 Outreachy In Outreachy news, we had a wonderful mid-internship video call with all the interns and mentors to catch-up on how everyone is getting along. Seeing the progress everyone has made was great! I am very grateful for the work that Michael and Steve have put in so far to make this a very successful Outreachy round for OCaml. In sadder news, an email was shared with all Outreachy mentors detailing the increasingly critical financial situation the project finds itself in. There are ongoing discussions about how costs can be cut including potentially only running a single round a year. Patrick Ferris 2025 7 15 Graft With the release of Forester.5.0, I made a plan to make a release of Graft.0.1. Unfortunately this is blocked by a new release of hilite, a tool I built for doing build-time syntax highlighting for OCaml code. This powers the syntax highlighting on ocaml.org. Patrick Ferris 2025 6 30 https://patrick.sirref.org/rest-of-monthly-2025-06/ rest-of-monthly-2025-06 /rest-of-monthly-2025-06/ PhD Work and Scientific Programming I have been away on holidays. Some of that was spent hiking in the Massif de Calanques in Marseille, unfortunately this week they were on fire. Anyway, this "weekly" fills in the blanks for the rest of June 2025. Patrick Ferris 2025 6 30 Revisiting hazel_of_ocaml I spent some time thinking and writing about hazel_of_ocaml and the benefits of OCaml as source language and Hazel as a target language. The stability of OCaml with a powerful type-system makes it very expressive. Hazel, on the other hand, is a hotbed of PL theory research. The typed holes, which are fully supported within the language, are particular powerful targets for a transpiler. They allow transpiler developers to incrementally support the language whilst still getting full feedback of their generated source code. Whilst it is a niche use-case, I thought it was an interesting point to note. Patrick Ferris 2025 6 30 A second year report I wrote my second year report at the end of June. The process was a bit chaotic, but I think ultimately it was useful for clarifying some ideas I had rattling about in my head. Part of that was working on my publications page and, through a conversation with Anil, realising that my work on Graft was in many ways a part of my own research. If you are interested, feel free to peruse both my first year and second year reports. Take them with a pinch of salt! I think it is useful to be fairly open with this sort of thing when you can be. Maybe someone will find them useful. Patrick Ferris 2025 6 30 What is scientific programming? This question arose whilst writing my second year report. It got me thinking about how we distinguish scientific programming from other kinds of programming. I came across a paper by Tim Storer: Bridiging the chasm: A survey of Software Engineering Practice in Scientific Programming. I enjoyed a few aspects of this paper. Relating scientific programming to the scientific method so directly was pretty useful insight for me. Falsifable hypotheses, repeatable experiments and reproducible results when applied as characteristics of the computer stack argue for better tools to enable this. The breadth of Storer's case-study analysis on how scientific programming has "gone wrong" is very good, if anyone needs a reference to draw on to help support their tools or research, this seems like a good index to use. Patrick Ferris 2025 6 30 Ppxlib I was reminded again of the pain of helping maintain ppxlib. We are coming to the conclusion, that in its current form, it is becoming unmaintainable. I took a stab at updating bisect_ppx to the latest ppxlib. With OCaml entering a potentially turbulent parsetree era, it might be time to take stock of this and propose some fillers to help. Nathan Rebours and I are meeting next week to discuss ideas he has to make parsetree migrations smooth sailing for all! Patrick Ferris 2025 6 30 Geocaml Work on ocaml-tiff has stalled, which is totally fine. Michael and I independently are working on other things and have obligations to fulfill. Maybe some day we'll carve out enough time together to push forward on this project, but not this week. Patrick Ferris 2025 6 2 https://patrick.sirref.org/weekly-2025-06-02/ weekly-2025-06-02 /weekly-2025-06-02/ Forester as a Target Syntax This week included some time finishing opentrace and subsequently folding it into shelter. I have been writing up some more of the draft paper for shelter which I am excited to share in the near future. I revisited the upgrading vpnkit PR and pushed some more fixes. I have been thinking, again, about the promise of a direct-style world for OCaml that just hasn't quite landed yet. C'est la vie. Patrick Ferris 2025 6 2 Forester and Graft I spent a bit of time finally pulling out my changes to Forester to add markdown and bibtex support into a standalone tool: graft. Patrick Ferris 2025 6 2 https://patrick.sirref.org/graft/ graft /graft/ Graft https://github.com/patricoferris/graft Graft is a Forester preprocessor. Forester syntax and produces a new forest completely written in Forester syntax. Patrick Ferris 2025 6 2 Usage graft simply preprocesses a forest generating Forester trees from .md, .bib and .tree files. It will copy the structure of the input directory in the output directory. $ graft preprocess --output=grafted-trees trees $ forester build This assumes that you have updated your Forester toml file to put the grafted-trees directory as your source of trees. Patrick Ferris 2025 6 2 Example A typical "tree" might look something like A few things to note: The yaml frontmatter allows you to add some of the metadata fields from Forester. At any point in your markdown there is an escape hatch to Forester using a forester codeblock. It is very satisfying to find the separation of concerns works quite well. For a while I had been rebasing my development branch on Forester. I was also worried about trying to get the code upstream as it pulled in many dependencies. It seems that I have a very workable solution. I welcome contributions to graft including extra input formats. I am also considering extending the Forester configuration to contain some graft configuration for how it should generate Patrick Ferris 2025 6 2 Maths Support As part of that process, graft now supports Markdown KaTeX. For example: Patrick Ferris https://patrick.sirref.org/mrdts/ mrdts /mrdts/ Mergeable Replicated Data Type Implementation Definition A for a data type is a tuple where: is the set of all possible states at a branch, is the initial state, implements every data type operation, implements the three-way merge strategy. Definition 2.1 from "Certified Mergeable Replicated Data Types". Patrick Ferris 2025 6 2 Revision Supervisions I have been doing revision supervisions for Discrete Maths and Foundations of Computer Science. I have also been marking Operating Systems past paper questions for the same group of first years. Patrick Ferris 2025 5 26 https://patrick.sirref.org/weekly-2025-05-26/ weekly-2025-05-26 /weekly-2025-05-26/ Opentrace and Supervisions opentrace and doing revision supervisions. Patrick Ferris 2025 5 19 https://patrick.sirref.org/open-trace/ open-trace /open-trace/ Opentrace https://tangled.sh/@patrick.sirref.org/opentrace Thanks to Koonwen's excellent libbpf bindings in OCaml, I have been building a little tool called opentrace to make it easier to track an executable's inputs and outputs. This work was inspired my Michael's self-proclaimed "gross hack": pyshark. Whilst pyshark achieves its goals by injecting code into commonly used python objects and methods, opentrace uses eBPF opentrace can remain programming language agnostic. However, less information is none about the user's intent compared to something like pyshark. Patrick Ferris 2025 5 19 Monitoring the System opentrace has an all command that will trace the entire system. The --flags=O_WRONLY argument filters the events where the O_WRONLY flag was set in the call to open. comm -wrapped is an artefact of using Nix. Patrick Ferris 2025 5 19 Tracing an Executable The primary use case for this tool is to inspect what files your program might be reading and writing. The "flags" argument can specify a small boolean formula for checking the open flags of a particular event with | & ~ Parentheses can be used for precedence. Patrick Ferris 2025 5 19 Spawning Subprocesses One feature opentrace opentrace is primarily an eBPF program that is loaded into the kernel and communicates with an OCaml program. Events are communicated via a ring buffer and most of the post-processing happens in OCaml. To capture subprocesses, opentrace This gives opentrace a new identifier to track, namely the cgroup. So consider the following program. let = Eio_posix.run @@ fun env -> Eio.Path. save ~create: `Or_truncate 0o664 env#fs / "hello.txt" "hello" ; Eio.Process.run env#process_mgr "/bin/bash"; "-c"; "echo 'heya' > heya.txt" It first creates a file using direct calls to functions like openat. Then it spawns a process which creates a new file called heya.txt. This happens in a separate process. However, with the --cgroups flag we can capture both interactions with the operating system. Patrick Ferris 2025 5 19 Eio's Process API I have used the Eio_unix fork action process API to be able to extend what happens in the child process. Loading most eBPF programs into the kernel requires special privileges hence the need for sudo. When a user requests for a particular program to be executed and traced, opentrace spawns a process via the Eio Process API. Opentrace defines a few new so-called "fork actions", little fragments of C code that are run after the call to fork clone Most likely this ends with a call to execve, but other calls are possible for example setuid allowing opentrace to change the user of the child process so it does not run as root into that group. Patrick Ferris 2025 5 19 Limitations: Io_uring Whilst testing opentrace against some of the tools I use nearly daily, I noticed some events were being missed. I tried tracing forester, and only the initial read of forest.toml was logged. It dawned on me that the reason for this was that forester eio io_uring to perform most of the IO. Most attempts to open files were bypassing the open system calls, and instead they were being performed by the kernel after reading a submission request for an openat2-style call! This is not news to seasoned, Linux systems programmers. Io_uring bypasses SECCOMP filters for exactly the same reasons. It is interesting to note two things here: We can tell that forester reads the configuration file probably using something like In_channel it does It appears that Uring is performing IO in both worker threads and directly. The file paths are empty at the moment as I cannot find a clean way to trace openat submission requests into Uring without it sometimes going very wrong. do_filp_open io_openat2, but this seems quite brittle, and often the filename is completely garbled, so I do not set it. If anyone has any ideas to trace Io_uring more reliably, I am all ears! Patrick Ferris 2025 5 26 Supervisions The first year students I teach will be doing their first year exams soon. I have been helping them revise for foundations of computer science and discrete mathematics. Whilst doing so, I have been thinking a good deal about Jon Sterling's post about assessment bureaucracy. In particular: Patrick Ferris 2025 5 19 https://patrick.sirref.org/weekly-2025-05-12/ weekly-2025-05-12 /weekly-2025-05-12/ Metashells This week, I feel I have been stuck fighting the OCaml ecosystem trying to keep my OxCaml work afloat. Aside from that, Ryan and I made some really nice progress with Shelter, culminating in Ryan describing it as a metashell. Patrick Ferris 2025 5 19 Shelter the metashell The main progress this week with Shelter was composing runc's terminal mode with entering raw terminal input mode on the Shelter side. This is inspired by Ryan's own work on capability interfaces. Shelter remains mostly intact, acting as an interactive shell. However, just before executing a command we switch to receiving and sending raw terminal inputs and outputs. This means tools like vim now work in Shelter! Not only that, but users can now activate zsh snapshot the session. This lets you alter the granularity of snapshots from the command-line. Patrick Ferris 2025 5 19 Louis Pouzin's "Shell" I spent some time reading part of the multics design documentation this week. Louis Pouzin coined the term "Shell" in this document and I was reminded yet again just how important it is to be a good writer even as a "computer science researcher". For example, this excerpt from the requirements section of the document The previous definitions imply that a command MUST be designed while keeping in mind the user, sitting at his console, wondering about what might be going on, mistyping or forgetting arguments, even if fully aware of the conventions, and possibly interfering with the command by hasty quits, carriage returns, and other temperamental reactions. And then later, when defining the "SHELL". We may envision a common procedure called automatically by the supervisor whenever a user types in some message at his console, at a time when he has and subroutine. The purpose of such a procedure is to create a medium of exchange into which one could activate any procedure, inside of another program if it were called. Hereafter, for simplification, we shall refer to that procedure as the "SHELL". It still surprises how little the undergraduate degree in computer science at Cambridge focuses on writing skills. Patrick Ferris 2025 5 19 OxCaml Last week, I got a toplevel with OxCaml working. This required a serious amount of work to understand the changes Janestreet have made to obscure parts of the OCaml compiler and then working those into tools like js_of_ocaml. This week, Janestreet pushed their latest rounds of changes and of course everything broke! I spent some more time fixing it all back up. I'm not entirely sure how maintainable this is. The problem is that, whilst things compile, the programs do not work together! Only when someone uses the program do the bugs surface. Patrick Ferris 2025 5 19 Other OCaml Work I worked on some other parts of the ecosystem this week. Patrick Ferris 2025 5 19 Ppxlib ppxlib as we try to keep the API predictable for users: Extension points are rewritten: this means the part that is rewritten is nicely delimited by the extension points start and end. Attributes extend: attributes do not rewrite the code they are attached to but rather extend the code with new AST nodes. . Patrick Ferris 2025 5 19 Tiff See Michael's notes. The two big changes to help with this are pretty common when speeding up these parts OCaml programs: Allocate less Does less work In terms of allocating less, the original implementation was using a char list to represent LZW strings. Manipulating these becomes quite costly, particularly since the most common operation is appending a single character to the end of a list. Converting this to use OCaml's immutable string saved a ton of allocations. In terms of doing less work, I opted to bypass Cstruct the code. In particular, LZW ends up reading potentially huge arrays full of bytes one-by-one. So reading each byte needs to be quite snappy. This is a bit of a trade-off in terms of "safety" but we are in control of this code so I'm not too worried about that. Here are some results decompressing a fairly large array of some elevation data. Patrick Ferris 2025 5 10 https://patrick.sirref.org/weekly-2025-05-04/ weekly-2025-05-04 /weekly-2025-05-04/ Hazel and OxCaml I missed a week of posting last week, mainly because I spent more time writing posts. Patrick Ferris 2025 5 10 Hazel of OCaml I mentioned previously that I was building a tool to transpile OCaml code to Hazel. This work is now in a good enough state that, along with one of my students, we have transpiled a good number of OCaml programs to help them write their evaluation for their third-year project. I wrote up a little summary of that work, which I've transcluded below. Patrick Ferris 2025 5 2 https://patrick.sirref.org/hazel-of-ocaml/ hazel-of-ocaml /hazel-of-ocaml/ A Transpiler from OCaml to Hazel Over the past few months, I have been piecing together a transpiler from Hazel to OCaml. This is, in part, to help one of my third-year undergraduate students who is working on type error debugging in Hazel. Patrick Ferris 2025 5 2 Typed Holes Hazel is a functional programming language with typed holes. Holes are pieces of your program that have not yet been filled in. Holes can appear anywhere in your program both as expression or types. Hazel can still evaluate your program in the presence of holes. To get a flavour of Hazel, take a regular map function for lists. fun xs -> case xs | [] => [] | x :: xs => f (x) :: map(f)(xs) end in map(fun x -> ?)([1, 2, 3])]]> ? familiar with OCaml types ? list Hazel supports local type inference but nothing involving unification variables. For example, a simple add_one function in Hazel fun x -> x + 1 ? -> Int. Patrick Ferris 2025 5 2 From OCaml to Hazel The ability to transpile OCaml programs to Hazel programs is motivated by one simple thought: there are more OCaml programs than there are Hazel programs. This could help bootstrap projects by alleviating the need to rewrite Patrick Ferris 2025 5 2 A Transformation of Syntax Hazel markets itself as an "Elm/ML-like functional programming language". From the previous example of map, it should be apparent just how close to OCaml the language is. It turns out that a majority of the transpiler is a transformation of syntax. Take a simple ADT for an arithmetic programming language. type expr = | Float of float | Add of expr * expr | Sub of expr * expr | Mul of expr * expr | Div of expr * expr And when we run hazel_of_ocaml over this OCaml type declaration. Not much has changed expect some syntax. Hazel does not have a notion of top-level expression so hazel_of_ocaml wraps the program into one set of value bindings. For the most part, Hazel acts as a subset of the pure, functional part of OCaml. At the time of writing, this subset is fairly limited If we try out the same map function but written in OCaml and transpiled to Hazel we get. let rec map f = function | -> | x :: xs -> f x :: map f xs Which becomes the following hazel program. fun x1 -> case x1 | [] => [] | x :: xs => f(x) :: map(f)(xs) end in ?]]> We could have a field day discussing the syntax of OCaml and Hazel taking a look at how to handle polymorphism in Hazel. Patrick Ferris 2025 5 2 Explicit Polymorphism Hazel has explicit polymorphism. So far, we have not seen it as we have let the types have holes in them. The map function in OCaml has the following type. val map : 'a -> 'b -> 'a list -> 'b list Craig's excellent blogpost on the matter ... type variables in signatures are implicitly universally-quantified So in reality, we have that map has the following type. b) -> a list -> b list]]> In Hazel, we have to explicitly type our map function to be polymorphic. Not only does this mean the type annotation requires universally quantified type variables, but we must also perform type application wherever we choose to apply the map forall b -> (a -> b) -> [a] -> [b] = typfun a -> typfun b -> fun f -> fun xs -> case xs | [] => [] | x :: xs => f (x) :: map@@(f)(xs) end in map@@(fun x -> ?)([1, 2, 3])]]> forall introduces a universally quantified type variable into our type annotation, and typfun requires @<T> where T is some type. This allows hazel to quite easily support higher rank polymorphism, but we will not worry too much about that. Patrick Ferris 2025 5 2 Propagating OCaml Types into Hazel Most often, OCaml users interact with prenex at the front of the type. OCaml does support quantifiers inside certain types like records. What this means for the transpiler is that we can reuse OCaml's type inference to safely instantiate the correct type annotations and type applications in Hazel! To do this, hazel_of_ocaml uses Merlin to inspect the type of the function in either a value binding or at the point of a function application. Take a simple, polymorphic length function. let rec length = function | -> 0 | _ :: xs -> 1 + length xs let int_len = length 1; 2; 3 let str_len = length "only"; "two" When we run this through hazel_of_ocaml with the -type flag we get. [a] -> Int = typfun a -> fun x1 -> case x1 | [] => 0 | _ :: xs => 1 + length@(xs) end in let int_len : Int = length@(1 :: 2 :: [3]) in let str_len : Int = length@("only" :: ["two"]) in ?]]> hazel_of_ocaml has correctly instantiated the type for length inside the recursive function and then in each case with the integer list and the string list. Patrick Ferris 2025 5 2 A Corpus of Hazel Programs The impetus for this work was to derive a corpus of ill-typed Hazel programs. Luckily, such a corpus exists for OCaml! Seidel et al. created a corpus of OCaml programs from their undergraduate students at UC San Diego. Some of these programs have been transpiled to Hazel. Patrick Ferris 2025 5 2 Future Work Hazel is a fun, research programming language. Potential third-year students may find it interesting to take this work further. For example, how would this look in terms of a module system? From a purely engineering perspective, plenty of work would be needed to convert a multi-library OCaml project to Hazel cmi Another line of research would be to have Hazel target one of the intermediate representations in OCaml which would give Hazel a fully functioning compiler to "native" code? Patrick Ferris 2025 5 10 OxCaml I spent some time this week getting more familiar with Oxidized OCaml. I have a habit of wrapping new OCaml tools and libraries into toplevel, browser applications. For example, try-irmin and try-eio. Naturally, I tried to wrap OxCaml into a toplevel so people could play around with the new modes that did not align so the raw Obj.repr I would say that I do question the time-spent/value trade-off, but a mostly working toplevel with OxCaml is available at: https://patrick.sirref.org/oxcaml. Patrick Ferris 2025 4 21 https://patrick.sirref.org/weekly-2025-04-21/ weekly-2025-04-21 /weekly-2025-04-21/ Shelter in the Forest I spent much of this week working on Shelter and things related to it. Some of that time was also spent on Forester. Patrick Ferris 2025 4 21 Forester I added two important quality-of-life features to my frontend to Forester this week. Patrick Ferris 2025 4 21 Bibtex Support I had previously mentioned adding support to Forester for Markdown. This week I added support for Bibtex too. From any *.bib file in your forest, Forester will now dutifully recognise it as a Bibtex file and convert, as best it can, all the entries into Reference trees. I'm becoming quite convinced of this model at the moment. I'm using Forester's Code.t as a target representation. In fact, to ease the process, I really shouldn't spend all my time on my website, I have reused my Yaml.t -> Code.t and Markdown.t -> Code.t functions in the Bibtex parser. To see it in action, you could have a look at the Build Systems à la Carte paper which is generated completely from Bibtex. Patrick Ferris 2025 4 21 Full Heading Support The eagled-eyed viewer may have noticed that the table of contents for this page has more than one level. I finally caved and spent an evening rejigging my Cmarkit.Doc.t -> Tree code which was hacky and broken and is now less hacky and less broken. In addition, headings support links and emphasis etc. Patrick Ferris 2025 4 21 Lunch with Jon Sterling I had a delightful lunch with Jon Sterling discussing the future of Forester, the nature of Patrick Ferris 2025 4 21 Shelter Fixes I spent a good chunk of my week fixing bugs in Shelter with the aim to perhaps setup a VM somewhere and let people kick the tyres of what we've got so far. The first bug is pretty annoying. At the moment, our filsystem backend is ZFS and we make heavy use of snapshots and cloning in order to provide time-travelling capabilities. Unfortunately, ZFS will take a snapshot before lots of disk activity would be snapshotted in a half finished state and this would cause all sorts of problems. Thanks to Mark Elvers for the pointer to how OBuilder uses ZFS for the OCaml macOS builders which unmount datasets immediately therefore inducing a flush of sorts. Shelter now follows a similar model with all of the slow downs that create. Anil and I discussed some amalgamation of overlayfs, tmpfs and ZFS to alleviate some of this but for now that's a premature optimisation. Patrick Ferris 2025 4 21 A small eDSL for Shelter Whilst testing Shelter, I ended up wanting a way to programmatically invoke the different run commands. This is similar to say a Dockerfile, but maybe with a little more expressivity. This lead me to revisit the Build systems à la Carte paper and rediscover selective applicative functors. module D = Shl Identity let shelterfile = let open D in let base_image = from "alpine" in let is_node_lst img = String.equal "v22.15.0" stdout img in let cmds base = let node_version = run "node --version" base in Select.if' Select.map ~f:is_node_lst node_version run " " run " " base in with_session "node" cmds base_image From this, we get a slightly more expressive way to describe images. Patrick Ferris 2025 4 21 Geocaml TIFF Library I was pleasantly surprised to receive a pull request from Michael adding support to ocaml-tiff for reading TIFF files compressed using LZW. I was also surprised to hear the TIFF LZW is a little different to others. In trying to get this PR merged, I moved the initialisation of the Eio eventloop to outside each individual test case. This one change then completely broke the entire test suite. After a period of debugging and help from Thomas Leonard the root cause was OCaml's OUnit2 Unix.fork lead to the issues. Read more about that issue on the Eio issue tracker. Patrick Ferris 2025 4 21 Part II Students As the new term begins, it signals that there are only just over two weeks for the final year undegrads at Cambridge to submit their dissertations. read about their projects and I'm very impressed. I'm sure the next two weeks will be stressful, but I'm proud of what they have accomplished over the past academic year. Patrick Ferris 2025 4 21 https://patrick.sirref.org/weekly-2025-04-14/ weekly-2025-04-14 /weekly-2025-04-14/ Catching Up! This is a bit of a catch-up post. A week or so in Belfast has thrown me a little off course with the weeklies. Patrick Ferris 2025 4 21 Outreachy December 2024 and Beyond In the last week or so, we have come to the end of the December 2024 round of Outreachy. For those who do not know about Outreachy, it facilitates and organises internships in open-source. Internships for those who are historically underrepresented and impacted by systematic bias in tech. We, the OCaml community, held our biannual Demo Day where our current cohort of interns can show off what they have been building for three months. This round, we only had one intern, Abdulaziz, who worked on the OCaml API diffing tool. The next round has just completed the contribution phase and our next cohort of interns is being selected. Patrick Ferris 2025 4 21 This week, I added support for OCaml 5.4 to ppxlib. A rather non-trivial change to the codebase due to changes in the representation in Longident Longident labelled tuples, a light-weight record-like syntax similar to labelled arguments in functions. let x = ~x:1, 10, ~y:1 val x : x:int * int * y:int let add ~x, ~y = x + y val add : x:int * y:int -> int As of writing this post, we are waiting for the full OCaml 5.4 feature freeze and magic number bump. Note this is support for OCaml 5.4 5.4) not a bump to use the 5.4 parsetree in ppxlib.]]> (ppxlib)[Ppxlib]The Future of I have now been helping maintain Ppxlib for a while and I am beginning to wonder about the long-term vision for the project. Ppxlib is a central component to any modern OCaml library or tool, Sherlocode reckons there's 18.3k instances of The biggest issue is still the tension between abstract types and pattern-matching. By exposing the Parsetree directly to user's, they can pattern-match against the AST and build their ppxes in a rather straight forward way. However, this comes with the risk that the type may change thus breaking their code. I have mentioned before about plans to use "views" to help with this, but I do not see that happening any time soon. It feels to me, we should be able to allow a user to select their AST to work with. How this interacts with modules like Ast_builder is unclear to me, but it would mean user's can remain on older ASTs even when ppxlib bumps the internal, main AST. Patrick Ferris 2025 4 21 Sherlorocq Rocq coq I have been interested in theorem provers for a while, more from an engineering perspective and less from a theoretical one. My good friend, Derek Sorenson, taught me about Rocq when we encoded mergeable replicated datatypes. Later I had a go at encoding decentralised information flow control Fstar During this process, I found it quite challenging to find Rocq code that I could learn from. Compared to using say OCaml's Stdlib This lead me to port Arthur Wendling's excellent Sherlocode to do the same thing only for Rocq code. Luckily, it uses regular expression matching with a trigram index so it is not tied to any particular programming language. Anyway, I've had some interest from the people at Inria to bring the server back to life. Their wish is my command! Sherlorocq is back online! Patrick Ferris 2025 3 31 https://patrick.sirref.org/weekly-2025-03-31/ weekly-2025-03-31 /weekly-2025-03-31/ Shelter, Hazel and More! Last week I focused on Shelter -- our idea that shells should have the same ability as reproducible build tools like Nix or Docker. To this end I now have a fairly fleshed out prototype. Patrick Ferris 2025 3 31 Shelter Prototype Shelter is a spin-off from the work Michael and I started with Shark. It takes the same ideas but applies them directly to a shell-like interface. We're still in the middle of working all of this, but you can read more about it at Shelter. Patrick Ferris 2025 3 31 Forester Hacking As you can probably tell, my website is still using Forester. I rebased my Markdown branch to include the new Atom syndication feature. Alongside that I added support for arbitrary HTML injection into Forester via codeblocks in Markdown. This was actually very straightforward thanks to Markup and being able to re-parse Forester syntax in the middle of converting a Markdown document. The HTML for the shell in Shelter uses this feature. If you are interested in taking this custom Forester for a spin, there's a branch on Github. In fact, nearly the only change beyond letting the core engine know about markdown files, is adding a new parser frontend. Patrick Ferris 2025 3 31 Hazel For one of my Part II students, I've been prototyping a transpiler from OCaml to Hazel. This has gone pretty well and now supports type annotations as well as straight-forward implementation translation. Consider the following OCaml map function. let rec map f = function | -> | x :: xs -> f x :: map f xs The tool, hazel_of_ocaml can translate this to Hazel code, including making the polymorphism explicit. forall b -> (a -> b) -> [a] -> [b] = typfun a -> typfun b -> fun f -> fun x1 -> case x1 | [] => [] | x :: xs => f(x) :: map(f)(xs) end in ?]]> You can copy and paste that codeblock into the hazel playground map@<a>@<b> Patrick Ferris 2025 3 31 Ppxlib I recently wrote about the painful experience of miragrating ppxlib to the 5.2 OCaml AST. This week, Nathan Rebours and I merged a PR to add the 5.3 AST! Ppxlib has been playing catch-up with the compiler and we decided it was best to try to catch up quickly and deal with the ecosystem fallout all at once rather than incrementally. With this new AST merged, ppx authors can now use the new Pexp_effect parsetree node. I'll write a little more about this in a separate post soon. 2025 2 17 https://patrick.sirref.org/weekly-2025-02-17/ weekly-2025-02-17 /weekly-2025-02-17/ Vpnkit, Void Processes, LSP Servers Previous weeklies used strong emphasis to distinguish sections. This comes from Forester's philosophy about atomicity of the content in your forest. However, subtrees are supported! I quickly hacked together the ability to use subheadings to indicate subtrees. This is strictly less expressive than the of Forester's default syntax as we cannot close heading sections in Markdown. This weekly uses subtrees. 2025 2 17 Vpnkit I spent some time this week trying to upgrade vpnkit to OCaml 5. I was originally working on a paper idea which might need benchmarks, but Anil and I decided we could simply point to the port I did and show how it has simplified much of the code. 2025 2 17 Void Processes void processes. A lot of the groundwork already exists in Jake Hillion's master's thesis. This week I added a feature that I needed to help build the processes we need for the shell I'm building: mount points with modes! pivot_root readonly or readwrite. /say/hey program has been statically compiled using musl-gcc let status = let void = empty |> mount ~mode:R ~src:hey_dir ~tgt:"say" |> exec "/say/hey" in let t = Void.spawn ~sw void in Promise.await Void.exit_status t There really is nothing else in there. Without specifying a root mount, the void process is started with an empty tmpfs root. Next on the list is networking! 2025 2 17 LSP Servers I got a little side-tracked building a library for writing LSP servers in OCaml: mlsp. This may seem a little unrelated, but it isn't. The LSP has become the de facto standard for communicating between an editor and a programming language environment. If you have used VSCode to write a program in Python, chances are you are using the official extension which gives you linting, formatting, code navigation etc. All of these features are communicating using the LSP. It seems Code Mirror 2025 2 17 Open-Source & Community Anil and I had a great conversation this week about building community especially as it pertains to open-source and OCaml. I've been going back over Governing the Commons, but have already discovered The Real World of Technology! More thoughts on all of this soon... maybe 2025 2 10 https://patrick.sirref.org/weekly-2025-02-10/ weekly-2025-02-10 /weekly-2025-02-10/ Teaching, teaching, teaching... Discrete Maths. I supervise three third year students for their final year project and another one I co-supervise with Michael Dales. However, I do end up spending at least two full days a week on teaching. Something I really enjoy and take seriously. The time it takes is also quite unpredictable; last week for instance all the third year students slides particularly challenging to understand from their lectures and asked if I could help explain what was going on, so I produced some materials for that. OCaml Within the OCaml universe I spent a good bit of time trying to maintain a couple of different packages: Mirage Crypto: this library provides cryptographic primitives for OCaml programs. Unfortunately, the maintainers removed direct support for Eio replacing it with a "Unix" alternative. This is not a fair swap as now Eio programs must make a dependency to Unix! Speaking to Anil, the best approach might be to make Eio programs handle this directly. I think this highlighted how fragmented open-source maintenance can be as no user of Eio seems to have bumped into this yet and the upstream maintainers did not communicate a rather large breaking change. Ppxlib: In addition to the 5.2 AST bump 5.3 AST bump right behind it. I plan to write up a more detailed post about the challenges of maintaining this part of ppxlib. Paris I spent some time in Paris at the end of the week. I enjoyed visiting the Musée d'Orsay and in particular their collection of impressionist paintings. Here's my favourite from that visit by Camille Pissarro. Woman in an Orchard (Spring Sunshine in the Meadow at Eragny) 2025 1 27 https://patrick.sirref.org/weekly-2025-01-27/ weekly-2025-01-27 /weekly-2025-01-27/ AT Protocol and an IR for Wikis AT Protocol This week I've been diving into the AT Protocol. The Authenticated Transfer Protocol, aka atproto, is a decentralized protocol for large-scale social web applications. The protocol could be a candidate for the glue that holds together a distributed, computational wiki network. The protocol, it seems, is very similar to IPFS. Thankfully, a few years ago, I was working on building out a suite of OCaml libraries for working with IPFS. For example, ocaml-cid, self-describing content-addressed identifiers. # let s = "zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA";; val s : string = "zb2rhe5P4gXftAwvA4eXQ5HJwsER2owDyS9sKaQRRVQPn93bA" # let cid = Cid.of_string s |> Result.get_ok;; val cid : Cid.t = <abstr> # Cid.pp_human Format.std_formatter cid;; cidv1 - base58btc - raw - ident sha2-256 length 32 digest 6e 6f f7 95 0a 36 18 7a 80 16 13 42 6e 85 8d ce 68 6c d7 d7 e3 c0 fc 42 ee 03 30 07 2d 24 5c 95 To this end I have built out some more OCaml libraries for working with atproto, including: atproto-data: the atproto data model, similar to JSON-LD. ocaml-did: an OCaml library for working with decentralized identifiers. atproto-lexicon: atproto's schema format, I've been building a quick tool for doing an OCaml translation from these schemas. I managed to get a post published from the OCaml library after fixing it up and porting it to Eio. An IR for Wikis I started working on a proof-of-concept intermediate representation for Wikis -- I imagine it a bit like malfunction but for computational wikis i.e. a target for Wiki building tools that allows different front-ends and servers to communicate in a common IR for exposing key functionalities of a wiki: Links: External links, cross-wiki backlinks Versioned, temporal feeds Etc. Other PhD Work I met with most of my Part II students this week, and I'm excited about their work. Progress reports are due this week and next they have a presentation to give. In discrete maths this week we did induction. Next up is a big section on sets, functions, bijections etc. Misc. I was happy to find the Information Controls Fellowship Program. 2025 1 20 https://patrick.sirref.org/weekly-2025-01-20/ weekly-2025-01-20 /weekly-2025-01-20/ Forester, ICFP, Wikis This week was my first full week back from the break and I found it challenging trying to get back into what I had been working on previously. ICFP Papers In conversation with Anil, we looked at options for submitting a paper to ICFP. I wrote up some notes on some options we discussed. Forester I spent some time this week converting this website to using Forester. I'm not a huge fan of the syntax, especially as a lot of my site's content was already in markdown. So I wrote a markdown frontend to forester which is available on Github. The markdown frontend integrates very nicely and only a few changes were needed in the core logic of Forester itself. Additionally, for any features not directly supported in markdown there is an escape hatch using code blocks such as: Personally, I'm still getting to grips with the bottom-up approach to building this site, atomically creating notes and reference cards that then are linked in many places. I'm excited to see how I can integrate some of the Forester concepts into "Shark". OCaml In the OCaml world, I spent time on ppxlib, reviewing PRs to update the lower bounds of the library, fixing effect syntax problems and bumping internal AST to 5.2. I've also spent some time looking into de-objecting Eio, making the OCaml types more friendly to new users. I need to revive my port of Vpnkit to Eio for the thoughts on ICFP 2025 too. I also want to modernise and make more public my OCaml code for creating little shells in OCaml too -- I think the ideas here really have legs and would like to find a conference to submit them too. I also met with the single, OCaml Outreachy intern working on ocaml-api-watch. Misc. I met with most of my Part II students this week which was nice to catch up and see how their projects are going. I also started marking work for my first year students who are at the induction part of their Discrete Maths course. References Zhengpeng Feng Clement Atzberger Sadiq Jaffer Jovana Knezevic Silja Sormunen Robin Young Madeline C. Lisaius Markus Immitzer David A. Coomes Anil Madhavapeddy Andrew Blake Srinivasan Keshav 2025 https://patrick.sirref.org/tessera2025/ tessera2025 /tessera2025/ TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis Reference http://arxiv.org/abs/2506.20380 10.48550/arXiv.2506.20380 Zhengpeng Feng Clement Atzberger Sadiq Jaffer Jovana Knezevic Silja Sormunen Robin Young Madeline C. Lisaius Markus Immitzer David A. Coomes Anil Madhavapeddy Andrew Blake Srinivasan Keshav 2025 Abstract Anton Lorenzen Leo White Stephen Dolan Richard Eisenberg Sam Lindley 2024 https://patrick.sirref.org/oxcaml-2024/ oxcaml-2024 /oxcaml-2024/ Oxidizing OCaml with Modal Memory Management Reference https://doi.org/10.1145/3674642 Proceedings of the ACM on Programming Languages 10.1145/3674642 Anton Lorenzen Leo White Stephen Dolan Richard Eisenberg Sam Lindley 2024 Abstract Programmers can often improve the performance of their programs by reducing heap allocations: either by allocating on the stack or reusing existing memory in-place. However, without safety guarantees, these optimizations can easily lead to use-after-free errors and even type unsoundness. In this paper, we present a design based on modes which allows programmers to safely reduce allocations by using stack allocation and in-place updates of immutable structures. We focus on three mode axes: affinity, uniqueness and locality. Modes are fully backwards compatible with existing OCaml code and can be completely inferred. Our work makes manual memory management in OCaml safe and convenient and charts a path towards bringing the benefits of Rust to OCaml. Eric Zhao Raef Maroof Anand Dukkipati Andrew Blinn Zhiyi Pan Cyrus Omar 2024 https://patrick.sirref.org/zhao-typeerror-2024/ zhao-typeerror-2024 /zhao-typeerror-2024/ Total Type Error Localization and Recovery with Holes Reference https://doi.org/10.1145/3632910 Proceedings of the ACM on Programming Languages 10.1145/3632910 Eric Zhao Raef Maroof Anand Dukkipati Andrew Blinn Zhiyi Pan Cyrus Omar 2024 Abstract Type systems typically only define the conditions under which an expression is well-typed, leaving ill-typed expressions formally meaningless. This approach is insufficient as the basis for language servers driving modern programming environments, which are expected to recover from simultaneously localized errors and continue to provide a variety of downstream semantic services. This paper addresses this problem, contributing the first comprehensive formal account of total type error localization and recovery: the marked lambda calculus. In particular, we define a gradual type system for expressions with marked errors, which operate as non-empty holes, together with a total procedure for marking arbitrary unmarked expressions. We mechanize the metatheory of the marked lambda calculus in Agda and implement it, scaled up, as the new basis for Hazel, a full-scale live functional programming environment with, uniquely, no meaningless editor states. The marked lambda calculus is bidirectionally typed, so localization decisions are systematically predictable based on a local flow of typing information. Constraint-based type inference can bring more distant information to bear in discovering inconsistencies but this notoriously complicates error localization. We approach this problem by deploying constraint solving as a type-hole-filling layer atop this gradual bidirectionally typed core. Errors arising from inconsistent unification constraints are localized exclusively to type and expression holes, i.e. the system identifies unfillable holes using a system of traced provenances, rather than localized in an ad hoc manner to particular expressions. The user can then interactively shift these errors to particular downstream expressions by selecting from suggested partially consistent type hole fillings, which returns control back to the bidirectional system. We implement this type hole inference system in Hazel. Vimala Soundarapandian Adharsh Kamath Kartik Nagar KC Sivaramakrishnan 2022 https://patrick.sirref.org/kcrsk-mrdts-2022/ kcrsk-mrdts-2022 /kcrsk-mrdts-2022/ Certified mergeable replicated data types Reference https://doi.org/10.1145/3519939.3523735 Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation 10.1145/3519939.3523735 Vimala Soundarapandian Adharsh Kamath Kartik Nagar KC Sivaramakrishnan 2022 Abstract Andrey Mokhov Georgy Lukyanov Simon Marlow Jeremie Dimino 2019 https://patrick.sirref.org/mokhov-selective-2019/ mokhov-selective-2019 /mokhov-selective-2019/ Selective applicative functors Reference https://doi.org/10.1145/3341694 Proceedings of the ACM on Programming Languages 10.1145/3341694 Andrey Mokhov Georgy Lukyanov Simon Marlow Jeremie Dimino 2019 Abstract Applicative functors and monads have conquered the world of functional programming by providing general and powerful ways of describing effectful computations using pure functions. Applicative functors provide a way to compose independent effects that cannot depend on values produced by earlier computations, and all of which are declared statically. Monads extend the applicative interface by making it possible to compose dependent effects, where the value computed by one effect determines all subsequent effects, dynamically. This paper introduces an intermediate abstraction called selective applicative functors that requires all effects to be declared statically, but provides a way to select which of the effects to execute dynamically. We demonstrate applications of the new abstraction on several examples, including two industrial case studies. Andrey Mokhov Neil Mitchell Simon Jones 2018 https://patrick.sirref.org/mokhov-build-systems/ mokhov-build-systems /mokhov-build-systems/ Build systems a la carte Reference https://doi.org/10.1145/3236774 Proceedings of the ACM on Programming Languages 10.1145/3236774 Andrey Mokhov Neil Mitchell Simon Jones 2018 Abstract Build systems are awesome, terrifying -- and unloved. They are used by every developer around the world, but are rarely the object of study. In this paper we offer a systematic, and executable, framework for developing and comparing build systems, viewing them as related points in landscape rather than as isolated phenomena. By teasing apart existing build systems, we can recombine their components, allowing us to prototype new build systems with desired properties. Eric Seidel Ranjit Jhala 2017 https://patrick.sirref.org/ocaml-corpus/ ocaml-corpus /ocaml-corpus/ A Collection of Novice Interactions with the OCaml Top-Level System Reference https://doi.org/10.5281/zenodo.806814 Zenodo 10.5281/zenodo.806814 Eric Seidel Ranjit Jhala 2017 Abstract Tim Storer 2017 https://patrick.sirref.org/storer2017bridging/ storer2017bridging /storer2017bridging/ Bridging the chasm: A survey of software engineering practice in scientific programming Reference ACM Computing Surveys (CSUR) 10.1145/3084225 Tim Storer 2017 Abstract The use of software is pervasive in all fields of science. Associated software development efforts may be very large, long-lived, and complex, requiring the commitment of significant resources. However, several authors have argued that the "gap" or "chasm" between software engineering and scientific programming is a serious risk to the production of reliable scientific results, as demonstrated in a number of case studies. This article reviews the research that addresses the gap, exploring how both software engineering and research practice may need to evolve to accommodate the use of software in science. Cyrus Omar Ian Voysey Michael Hilton Jonathan Aldrich Matthew Hammer 2017 https://patrick.sirref.org/omar-hazel-2017/ omar-hazel-2017 /omar-hazel-2017/ Hazelnut: a bidirectionally typed structure editor calculus Reference https://doi.org/10.1145/3093333.3009900 SIGPLAN Not. 10.1145/3093333.3009900 Cyrus Omar Ian Voysey Michael Hilton Jonathan Aldrich Matthew Hammer 2017 Abstract Structure editors allow programmers to edit the tree structure of a program directly. This can have cognitive benefits, particularly for novice and end-user programmers. It also simplifies matters for tool designers, because they do not need to contend with malformed program text. This paper introduces Hazelnut, a structure editor based on a small bidirectionally typed lambda calculus extended with holes and a cursor. Hazelnut goes one step beyond syntactic well-formedness: its edit actions operate over statically meaningful incomplete terms. Naively, this would force the programmer to construct terms in a rigid "outside-in" manner. To avoid this problem, the action semantics automatically places terms assigned a type that is inconsistent with the expected type inside a hole. This meaningfully defers the type consistency check until the term inside the hole is finished. Hazelnut is not intended as an end-user tool itself. Instead, it serves as a foundational account of typed structure editing. To that end, we describe how Hazelnut's rich metatheory, which we have mechanized using the Agda proof assistant, serves as a guide when we extend the calculus to include binary sum types. We also discuss various interpretations of holes, and in so doing reveal connections with gradual typing and contextual modal type theory, the Curry-Howard interpretation of contextual modal logic. Finally, we discuss how Hazelnut's semantics lends itself to implementation as an event-based functional reactive program. Our simple reference implementation is written using js_of_ocaml. Louis Pouzin 2013 https://patrick.sirref.org/pouzin-shell-2013/ pouzin-shell-2013 /pouzin-shell-2013/ The Origin of the Shell Reference https://people.csail.mit.edu/saltzer/Multics/Multics-Documents/MDN/MDN-4.pdf Multics Documents Louis Pouzin 2013 Abstract How RUNCOM was created for CTSS and the shell was designed for Multics by Louis Pouzin. Context Backlinks Related https://patrick.sirref.org/camcst/ camcst /camcst/ Department of Computer Science and Technology, University of Cambridge department https://www.cst.cam.ac.uk/ https://patrick.sirref.org/geocaml/ geocaml /geocaml/ Geocaml Github Organisation https://github.com/geocaml Geocaml is an attempt to provide the OCaml community with a suite of geospatial libraries, for example: A Geojson library in pure OCaml. A tool for querying carbon intensity APIs. ISO3166 country codes. https://patrick.sirref.org/ni-forests/ ni-forests /ni-forests/ Northern Irish Forests Project https://patricoferris.github.io/ni-forests/ A small and simple application written during COP26 to learn more about forests/habitats in Northern Ireland. It is available online and there's a more accessible, low-bandwidth text version too. https://patrick.sirref.org/outreachy/ outreachy /outreachy/ Outreachy Project https://www.outreachy.org/ Outreachy provides internships in open source and open science. Outreachy provides internships to people subject to systemic bias and impacted by underrepresentation in the technical industry where they are living. I am one of the coordinators for the OCaml community for Outreachy. I have also mentored multiple three-month internships for the community. Patrick Ferris 2026 2 9 https://patrick.sirref.org/outreachy-ocaml-tiff/ outreachy-ocaml-tiff /outreachy-ocaml-tiff/ Write support in OCaml TIFF library I am mentoring Tambe Salome during the December 2025 Outreachy round to add support for writing TIFF files in the ocaml-tiff library. 2022 12 1 https://patrick.sirref.org/topojson-0001/ topojson-0001 /topojson-0001/ Implement a non-blocking, streaming codec for TopoJSON https://github.com/geocaml/ocaml-geojson GeoJSON is a JSON standardised format for handling simple geographical data. One common use of GeoJSON is to add map overlays such as with the popular JavaScript library leaflet. One issue with GeoJSON is redundant data, for example when two polygons share boundaries ideally only one would contain the full data about the boundary, however, with GeoJSON the data is duplicated. TopoJSON is an extension to GeoJSON to encode topology. This allows for redundant data to be removed and file sizes to be greatly reduced. This is often very desirable especially when working with data in the browser. In a previous Outreachy internship a new OCaml library was implemented to provide an OCaml library for TopoJSON, this project will build on this adding more functionality to the library and providing a non-blocking, streaming codec version similar to the geojsone library. 2022 6 1 https://patrick.sirref.org/topojson-0002/ topojson-0002 /topojson-0002/ Extend OCaml's GeoJSON library to support TopoJSON https://github.com/geocaml/ocaml-geojson GeoJSON is a JSON standardised format for handling simple geographical data. One common use of GeoJSON is to add map overlays such as with the popular JavaScript library leaflet. One issue with GeoJSON is redundant data, for example when two polygons share boundaries ideally only one would contain the full data about the boundary, however, with GeoJSON the data is duplicated. TopoJSON is an extension to GeoJSON to encode topology. This allows for redundant data to be removed and file sizes to be greatly reduced. This is often very desirable especially when working with data in the browser. In a previous Outreachy internship a new OCaml library was implemented to provide an OCaml library for TopoJSON, this project will build on this adding more functionality to the library and providing a non-blocking, streaming codec version similar to the geojsone library. 2021 6 1 https://patrick.sirref.org/ocamlorg-0001/ ocamlorg-0001 /ocamlorg-0001/ Improve the ocaml.org Website https://ocaml.org OCaml.org is the main website for OCaml, a functional, typed, high-level programming language. This project revolves around improving the website on multiple different fronts including: layout, accessibility and content. Because a lot of the content of the website is about OCaml and the website itself is written in OCaml, this would be a great opportunity for interns to learn the language and functional programming concepts. https://patrick.sirref.org/ucam/ ucam /ucam/ University of Cambridge institution https://www.cam.ac.uk/ Contributions