Part of my research at the moment is thinking about how shells are used. What commands are run? What constructs are used (e.g. redirections, compound commands...)? Dockerfiles are a defacto standard means for building container images and for the most part they are a shell with weird semantics and some meta-commands (sounds familiar!).
I had some assumptions about the kinds of shell commands people use (myself included) when writing Dockerfiles. For example, making use of AND-lists for grouping commands in RUN instructions. But I had no evidence to back this up. So I ran some analysis over the dataset from this paper from Henkel et al..
Total Dockerfiles: 218_253 (errors: 808)
Total Docker Instructions: 2_729_118
Total RUN instructions: 657_909 (errors: 4_847)
========
Count (Windows Dockerfiles): 841 / 218_253
Count (ENV commands): 268_898 / 2_729_118
Count (COPY commands): 165_968 / 2_729_118
Count (ADD commands): 86_552 / 2_729_118
Count (WORKDIR commands): 104_112 / 2_729_118
Count (using at least one &&): 267_113 / 657_909
Count (uses of exec): 25 / 657_909
Count (system pkg manager: [apt, apt-get, dnf, apk, pacman, nix, brew, yum]): 192_092 / 657_909
Count (internet retrievers: [curl, wget]): 95_433 / 657_909
Count (might use python): 9_082 / 657_909
Counts:
[(rm, 169_386), (echo, 113_768), (mkdir, 100_924), (cd, 74_904),
(chmod, 55_998), (set, 25_738), (mv, 24_034), (sed, 21_232),
(chown, 18_610), (grep, 16_604), (cp, 15_637), (export, 13_154),
([, 10_411), (touch, 3_152), (exit, 2_288), (printf, 2_275), (awk, 2_010),
(., 1_077), (test, 1_004), (source, 904), (pwd, 199), (unset, 105),
(diff, 40), (trap, 2)]
I think there's some evidence here to support some assumptions I already had:
A significant proportion of RUN instructions in Dockerfiles invoke package managers.
A significant proportion of RUN instructions invoke curl or wget.
There is something rather Zipfian about how shell built-in utilities are used.
There is limited used of exec (and therefore limited use of opening file descriptors directly via the shell, as opposed to a command running in a subprocess).
A lot of time is spent changing directories, either temporally (via cd in a RUN instruction, or by using WORKDIR).
Lots of ENV setting.