I am yet to jump head-first into using LLM-based tools like Claude Code or even ChatGPT to help with my programming in any serious way. After using some "free"-tier tools to try to better understand some tricky eBPF problems and to make sense of the semantics of POSIX shells, I was left unimpressed and ultimately felt my time was wasted.

I feel unwilling to send my money to the corporations behind these tools. This is for a variety of reasons including the deeply worrying relationship with the US Department of War (I struggle to find the meaningful difference between "opertational planning" vs. "operational decision-making" given the leaders of the US DoW), environmental concerns, exploitation of human labour and a fast-and-loose approach to copyright laws. All of which leaves the entire industry very unattractive to me.

Everybody can draw their line where they want, and I am, in good faith, trying to find where mine should be.

Unfortunately, my own decision to not use these tools is not completely in my control. What do I do about pull requests to open-source libraries I maintain that include agentic code? I am, after all, participating in the idea of open-source and inclusive collaboration. Agentic code also comes in many flavours, from blatant LLM-generated pull requests to heavily edited, human-in-the-loop contributions.

An immediate first step for me is to develop some policy for contributions to projects I maintain regarding code where LLMs have had a hand. This can help set expectations for contributors. Eio's "AI-generated Code" subsection is succinct and prohibits contributions that solely use AI.

It obfuscates how you think. Purely AI-generated code tells us little about how you think and the problems you might be having. This makes it harder to provide good feedback on PRs and issues.

It is often more work to review. Particularly for the OCaml ecosystem and libraries like Eio, it seems that these tools are not very good and generate a lot of believable code that is in actual fact completely wrong. PR comments and the code submitted with them can say completely different things.

It is a grey area for licensing. Models like ChatGPT have been trained on lots of code with different licenses and has been known to simply copy code as an answer to a prompt. We would like to avoid this headache as best we can.

Additionally, I plead to other developers to own their use of these tools. Please, make Claude a co-author of the commits where Claude has generated any code at the very least. This is the bare minimum. I have strong feelings about the etiquette of sending AI summarised information or LLM-generated code to other people, but before any of that, the decent thing is to remove any uncertainty in the receiving party's mind of how the information they are receiving came to be. In this way, I appreciate the OCaml compiler's AI notice which includes:

If a significant portion of your code, PR description, review comments or messages has been AI-generated, this must be disclosed, stating which tool was used and for what. Reviewing AI-produced code can require a different approach from human-written code. You should have an acknowledgement anyway even if another human contributor did some of the work.

Unfortunately, Jon Sterling was bitten by this recently. The feature in question was locally opening modules in types (added in OCaml 5.2). For example:

module M = struct
  type t = int
end

type t = M.(t list)

Here, the M.(t list) is an example of a new AST node called Ptyp_open. I had hastily not migrated Ptyp_open to older compilers. So what happened? Jon had used this new feature in his project with a 5.3 compiler with ppxlib 0.35. This is before our internal bump to the 5.2 AST so the code migrated all the way down to OCaml 4.14! Whilst passing from 5.2 to 5.1 we raised an error saying:

Error: migration error: module open in types is not supported before OCaml 5.02

The error is confusing (least of all because there is a typo on the OCaml version number). Since then, we have patched this issue and have planned to release a patched 0.35 in the not too distant future.

If you can bump to ppxlib>=0.36 then you will not face this issue.

We ran into a fairly interesting corner of the OCaml language whilst trying to migrate modular explicits in ppxlib. As a recap, in OCaml 5.5 you can now right function types that depend on modules. For example:

module type Add = sig
  type t
  val add : t -> t -> t
end
let add : (module A : Add) -> A.t -> A.t -> A.t =
  fun (module A : Add) a b -> A.add a b

We have to tell the compiler the module type of A by adding : Add. However, there is another very similar syntax for these kinds of constraints, (module A : module Add). And in fact, the former syntax was syntactic sugar for the latter. In OCaml 5.5, these two were distinguished from one another and (module A : Add) is the required syntax for modular explicits. As you might imagine, this is a bit of a headache in terms of ppxlib migrations. In the end, we try to preserve the original intent of the author where possible, and you can read more about it on the PR for distinguishing the two.