Vpnkit (Experience Report) [vpnkit-er]

Patrick Ferris

The experience reports for ICFP are defined as:

The purpose of an Experience Report is to describe the experience of using functional programming in practice, whether in industrial application, tool development, programming education, or any other area.

Vpnkit is a tool used in the Docker 4 Mac stack to translate networking operations between the host (macOS) and the virtual machine (Linux). It is almost entirely written in OCaml making heavy use of the various Mirage libraries for building portable networking applications.

An interesting direction for this paper may be instead to focus on moving the OCaml ecosystem from one without algebraic effects, to a community with algebraic effects. Where this is most acutely felt is how libraries manage asynchronous IO. This almost feels like an ICFP paper in and of itself. A tentative structure could be:

Introduction to OCaml 5 work and effects.
Experiments pre OCaml 5, Eio.
Porting large applications (Irmin and Vpnkit) to Eio.
Current situation: algebraic effect-based zoo.
Future: Picos, Eio, etc.

This feels like a strong and interesting paper. Two key strengths are: a very measurable section at (3), very practical throughout etc.

It feels reminiscent of the Scala capability keynote from last year, but would be less technical?

Context

ICFP Paper Ideas [icfp25-ideas]

January 22, 2025

Ideas and abstracts for ICFP 2025.

Vpnkit (Experience Report) [vpnkit-er]

Patrick Ferris

The experience reports for ICFP are defined as:

The purpose of an Experience Report is to describe the experience of using functional programming in practice, whether in industrial application, tool development, programming education, or any other area.

Introduction to OCaml 5 work and effects.
Experiments pre OCaml 5, Eio.
Porting large applications (Irmin and Vpnkit) to Eio.
Current situation: algebraic effect-based zoo.
Future: Picos, Eio, etc.

This feels like a strong and interesting paper. Two key strengths are: a very measurable section at (3), very practical throughout etc.

It feels reminiscent of the Scala capability keynote from last year, but would be less technical?

Functional IO [functional-io]

Patrick Ferris

The exact nature of this paper is a little unclear to me. But here goes.

Programming languages that do something must do IO. Operating systems present different mechanisms for doing IO. This paper would be looking at explaining these mechanisms and how they fit into a programming language (either poorly or well) and perhaps calling for better designs for languages as it relates to modern, post-POSIX IO.

On linux there are maybe two abstractions over system calls we may wish to discuss: AIO and Uring.

On macOS we have the Grand Central Dispatch.

What does this all boil down too? It might be interesting to describe these various models for achieving asynchronous IO:

Parallelism: one model is purely thanks to having multiple OS threads. In this model, you can block on an IO operation on a separate running thread.
Polling: this is your epoll, select() etc. Here, you make what would normally be considered blocking calls but only when you're pretty sure you know the call won't block.
Submission and Completion Queues: IOCP and Uring, this is a OS-provided mechanisms for submitting an IO operation and receiving completions when the kernel has performed the task. Conceptually, this is a lot simpler and shifts the burden of multiplexing IO on the kernel (you may have some parameter configuration to help you).

Backlinks

Vpnkit, Void Processes, LSP Servers [weekly-2025-02-17]

February 17, 2025

Previous weeklies used strong emphasis to distinguish sections. This comes from Forester's philosophy about atomicity of the content in your forest.

However, subtrees are supported! I quickly hacked together the ability to use subheadings to indicate subtrees. This is strictly less expressive than the \subtree{} of Forester's default syntax as we cannot close heading sections in Markdown.

This weekly uses subtrees.

Vpnkit

February 17, 2025

I spent some time this week trying to upgrade vpnkit to OCaml 5. I was originally working on a paper idea which might need benchmarks, but Anil and I decided we could simply point to the port I did and show how it has simplified much of the code.

Void Processes [void-processes]

February 17, 2025

Work continued on implementing (and fully exploring) void processes. A lot of the groundwork already exists in Jake Hillion's master's thesis.

This week I added a feature that I needed to help build the processes we need for the shell I'm building: mount points with modes!

In addition to the root mount (taken care of with pivot_root), we need to be able to add additional mounts into the process' environment. These can now be added. All mount points can be mounted readonly or readwrite.

Here is the "Hello, World!" example (the /say/hey program has been statically compiled using musl-gcc).

let status =
  let void =
    empty 
    |> mount ~mode:R ~src:hey_dir ~tgt:"say"
    |> exec [ "/say/hey" ]
  in
  let t = Void.spawn ~sw void in
  Promise.await (Void.exit_status t)

There really is nothing else in there. Without specifying a root mount, the void process is started with an empty tmpfs root. Next on the list is networking!

LSP Servers

February 17, 2025

I got a little side-tracked building a library for writing LSP servers in OCaml: mlsp. This may seem a little unrelated, but it isn't. The LSP has become the de facto standard for communicating between an editor and a programming language environment. If you have used VSCode to write a program in Python, chances are you are using the official extension which gives you linting, formatting, code navigation etc. All of these features are communicating using the LSP.

It seems Code Mirror can already proxy over a websocket for LSP support too (we might not even need that as we can compile OCaml directly to JavaScript/Webassembly and have the whole thing running locally!).

Open-Source & Community

February 17, 2025

Anil and I had a great conversation this week about building community especially as it pertains to open-source and OCaml.

I've been going back over Governing the Commons, but have already discovered The Real World of Technology!

More thoughts on all of this soon... maybe