I came across the challenge of creating the smallest possible “Hello world” docker image from this engineering blog.
(Almost did a placement year there, thanks Covid)
An internal competition they had, where the challenge was:
To build and publish a Docker image that prints the phrase “hello IDBS engineering” (or similar) when run. At a minimum, you will need to build a Docker image, run that image locally to test and publish to a Docker registry.
We’ll keep it simple and the following dimensions will be taken into account: The image runs and prints to console. Image size - the smaller, the better.
I got about half way through this article before I decided that it sounded like a fun challenge to spend an afternoon on, so cracked on with trying it myself before I spoiled too much of their solution.
So here’s some of my thought process and journey towards creating the smallest possible “Hello World” docker image.
I figured the best way to start would be to find a baseline image to try and beat or improve from. Luckily Docker themselves have a ‘hello-world’ image.
❯ docker run hello-world
...
Hello from Docker!
...
❯ docker images
REPOSITORY SIZE
hello-world 13.3kB
So our score to beat is 13.3kB, smaller than I was expecting (or hoping), this may take some work to beat. The alternative baseline is an ubuntu base image:
Use the ubuntu base image, and simply echo “Hello world!”:
FROM ubuntu
CMD ["echo", "Hello World!"]
❯ docker build . -t ubuntu-hello
❯ docker run ubuntu-hello:latest
Hello There!
❯ docker images
REPOSITORY SIZE
ubuntu-hello 72.8MB
hello-world 13.3kB
The base ubuntu image is pretty gigantic, the actual Hello World is negligible in size compared to the overhead of the ubuntu base. So what other options are there for base images to use?
Alpine is a “great image base for utilities and even production applications” and based around musl-libc and BusyBox.
FROM alpine
CMD ["echo","Hello There!"]
❯ docker build . -t alpine-hello
❯ docker run alpine-hello:latest
Hello There!
❯ docker images
REPOSITORY SIZE
alpine-hello 5.61MB
ubuntu-hello 72.8MB
hello-world 13.3kB
Down to 5.61MB is definitely an improvement on the ubuntu base, but alpine is based on an image of busybox, and contains some ‘optional’ extras we don’t need. So how small is a plain busybox version?
BusyBox “provides a fairly complete environment for any small or embedded system”. It only comes with the very bare necessities, this is fine however as in our case we only need one of them: “echo”
FROM busybox
CMD ["echo","Hello There!"]
❯ docker build . -t busybox-hello
❯ docker run busybox-hello:latest
Hello There!
❯ docker images
REPOSITORY SIZE
busybox-hello 1.24MB
alpine-hello 5.61MB
ubuntu-hello 72.8MB
hello-world 13.3kB
An improvement! But still an order of magnitude larger than the hello-world example image. Just over 1MB is not bad, although the image still contains a bunch of utilities that we don’t need..
0 BYTES
The ‘SCRATCH’ docker base image contains nothing. It contains 0 dependencies, 0 utilities, and is exactly what we’re looking for. By removing all of the overhead, the minimum size is now just the smallest self contained executable we can build.
My first thought here was to start with a simple c program.
As a baseline, I started with the classic printf hello world.
// hellogcc.c
#include <stdio.h>
int main()
{
printf("Hello World\n");
return 0;
}
And then compile with gcc:
gcc hellogcc.c -static -o hello1
Compiling with gcc, (and the -static flag to include dependencies), results in a massive 892KB executable. Luckily we can quickly shrink this with some compiler flags.
gcc hellogcc.c -Os -s -static -o hello2
Telling the compiler to optimise for size using “-Os”, and “-s” to remove linker table info from the executable. This results in a 793KB executable, better - but still not good enough.
A bit more searching around brought me to this stackoverflow answer Which outlines how even though nothing from the c libraries is used in the actual program, the C runtime (CRT) startfiles that call the main function do call on some libraries.
The solution to this is to specify startfiles, then choose to not include them in compilation.
The new code:
#include <unistd.h>
void _start(void) {
write(1, "Hello world!", 12);
_exit(0);
}
gcc hello_write.c -s -static --nostartfiles -o hello3
File sizes so far:
❯ ls -lh | awk '{print $9 " : " $5}'
:
hello1 : 869K
hello2 : 793K
hello3 : 8.9K
This reduces the executable size down to just 8.9KB, which is finally smaller than the example hello-world docker image. But I think we can still go smaller..
Tiny C Compiler (TCC) is a tiny but hyper fast C compiler. Luckily for us, it also generally creates much smaller executables.
int main() {
write(1, "Hello World\n", 12);
return 0;
}
Compile with tcc:
tcc hello.c -o hellotcc
ls -lh | awk '{print $9 " : " $5}'
hello.c : 136
hellotcc : 3.0K
Down to just 3KB just by swapping compiler, not bad!
Results so far:
❯ docker images --format "table \t"
REPOSITORY SIZE
tcc-hello 2.99kB
gcc-hello 9.06kB
busybox-hello 1.24MB
alpine-hello 5.61MB
ubuntu-hello 72.8MB
hello-world 13.3kB
The next logical step to try and go smaller than compiled c is to just write straight assembly. Now, whilst I have some experience with ASM - the majority of that is for GameBoy or N64. Whilst I could sit down and learn my way around, this challenge was meant to be about Docker - so why reinvent the wheel when I can just Google it..
My first thought was to check the code golf stackexchange for a solution. I found one here, however right after that I found what is arguably the holy grail:
In an example of pure wizardry by Brian Raiter, an ELF executable “Hello, World” program in just 62 Bytes.
Every ELF executable has a 52 Byte header, containing data on: if it’s 32/64 bit, little or big-endian, the target architecture etc. By placing the instructions to write “hello, world” in the header, (and some other tricks absolutely violating every inch of the ELF file standard), he produces a 62 Byte executable.
So a simple Dockerfile that just copies this executable to the image:
FROM scratch
COPY hello hello
CMD ["/hello"]
REPOSITORY SIZE
asm-hello 62B
tcc-hello 2.99kB
gcc-hello 9.06kB
busybox-hello 1.24MB
alpine-hello 5.61MB
ubuntu-hello 72.8MB
hello-world 13.3kB
Unless I can break some laws of physics, it seems like 62 Bytes is the smallest possible size for a Docker image with an executable in it.
But can we cheat by not having to include an executable at all?
Yes.. kind of
FROM scratch
CMD ["dir/hello"]
❯ docker build . -t cheat-hello
This docker image simply runs an executable called ‘hello’ in the folder ‘dir’. This folder and executable don’t exist when the image is created, so it gives an error if ran normally, but this does mean that the actual Docker image is exactly 0 Bytes.
REPOSITORY SIZE
cheat-hello 0B
asm-hello 62B
tcc-hello 2.99kB
gcc-hello 9.06kB
busybox-hello 1.24MB
alpine-hello 5.61MB
ubuntu-hello 72.8MB
hello-world 13.3kB
The image is ran using a directory on the host machine linked as a volume in the container. If the directory linked contains an executable called ‘hello’, then the container will run the executable and output “hello world”.
❯ docker run -v ~/path/to/folder/containg/executable/:/dir cheat-hello
hello, world
Is this cheating? Yes. The original competition did specify that it should be
able to be published to a Docker registry. I did try for a while to link to the
host machine’s /bin/ to grab echo from there, which would make it somewhat
portable. However grabbing dependencies from the host machine is quite literally
the antithesis of the role of Docker, so even if figured it out, writing it down
seemed a little blasphemous.
Whilst the challenge quite quickly strayed away from Docker, learning to dismantle and violate it’s principles has been a great learning experience. As has diving into the ELF file format and some c compiler options.
Having now read the rest of the original challenge post - we both ended up at the same final answer. (Theirs is slightly longer only due to a longer message). Once you get to zero Docker overhead, there’s only so far you can go.
It was an absolute blast for an afternoon - and I’d love to come back and try a slightly modified challenge: perhaps a Docker quine (an image that recreates itself) or the smallest possible Docker web server etc.
https://idbs-engineering.com/docker/containers/2021/01/28/container-competition.html
http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html
http://timelessname.com/elfbin/
https://registry.hub.docker.com/_/busybox/
https://devopsdirective.com/posts/2021/04/tiny-container-image/
]]>This list is a culmination of my absolute favourite resources and inspiring posts and stories across the internet. These are all things that I find myself coming back to every few months, some are inspiring, some motivating, some frankly have just taken three or four tries to actually understand what’s happening..
All the links here are to resources that remind me of why I love programming and all the quirks and challenges that come along with it. Hopefully some of them can do that for you too.
From the source of ‘Quake 3’, a black magic approximation of an inverse square root function.
float Q_rsqrt( float number )
{
long i;
float x2, y;
const float threehalfs = 1.5F;
x2 = number * 0.5F;
y = number;
i = * ( long * ) &y; // evil floating point bit level hacking
i = 0x5f3759df - ( i >> 1 ); // what the fuck?
y = * ( float * ) &i;
y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration
// y = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration,
// this can be removed
return y;
}
A mesmerizing piece of code so inspiring I can’t help but return to bask in it’s glow every few months. No other lines of code have ever made me want to sit down and write code more than this has.
IOCCC, or the International Obfuscated C Code Contest is an annual “celebration of C’s syntactical opaqueness” that first started in 1984.
This particular 2012 submission by Yusuke Endoh (Also a developer of Ruby) is an ASCII fluid simulation, that can use it’s own source code as a starting configuration:
endoh1.c
# include<stdio.h>// .IOCCC Fluid- #
# include <unistd.h> //2012 _Sim!_ #
# include<complex.h> //|||| ,____. IOCCC- #
# define h for( x=011; 2012/* #
# */-1>x ++;)b[ x]//-' winner #
# define f(p,e) for(/* #
# */p=a; e,p<r; p+=5)// #
# define z(e,i) f(p,p/* #
## */[i]=e)f(q,w=cabs (d=*p- *q)/2- 1)if(0 <(x=1- w))p[i]+=w*/// ##
double complex a [ 97687] ,*p,*q ,*r=a, w=0,d; int x,y;char b/* ##
## */[6856]="\x1b[2J" "\x1b" "[1;1H ", *o= b, *t; int main (){/** ##
## */for( ;0<(x= getc ( stdin) );)w=x >10?32< x?4[/* ##
## */*r++ =w,r]= w+1,*r =r[5]= x==35, r+=9:0 ,w-I/* ##
## */:(x= w+2);; for(;; puts(o ),o=b+ 4){z(p [1]*/* ##
## */9,2) w;z(G, 3)(d*( 3-p[2] -q[2]) *P+p[4 ]*V-/* ##
## */q[4] *V)/p[ 2];h=0 ;f(p,( t=b+10 +(x=*p *I)+/* ##
## */80*( y=*p/2 ),*p+=p [4]+=p [3]/10 *!p[1]) )x=0/* ##
## */ <=x &&x<79 &&0<=y&&y<23?1[1 [*t|=8 ,t]|=4,t+=80]=1/* ##
## */, *t |=2:0; h=" '`-.|//,\\" "|\\_" "\\/\x23\n"[x/** ##
## */%80- 9?x[b] :16];;usleep( 12321) ;}return 0;}/* ##
#### ####
###############################################################################
**###########################################################################*/
Self-documenting code in the worst way..
Fixing an JSON parsing bug in an 8 year old game that’s made over $6.4 billion since launch. An incredible story of profiling, decompilation and reverse-engineering to fix a 6 minute load time that dropped down to under 2 minutes.
Not exactly an internet gem, but a book that I really enjoyed reading, with some serious great advice and guidelines on writing better Java code. A must read in my opinion for someone at an intermediate or so level of Java.
Personally it really helped me overcome some imposter syndrome type feelings, and something I like to come back to every now and then to refresh my memory.
A classic funny tech support story from the mid 90s, a university professor that “can’t send an email more than 500 miles”. A great story if you’ve never read it before, and certainly one of my favourites out there.
A compression challenge to compress 1GB of text from Wikipedia. Marcus Hutter launched the challenge in 2006 with a €50,000 prize, and has recently (2020) been upped to a €500,000 prize.
The motivation behind the contest is the idea that general compression is closely related to intelligence, and whilst intelligence is a difficult concept to quantify, file sizes give a hard quantitative score. The contest has been largely dominated by Alexander Rhatushnyak, although the most recent winner is Artemiy Margaritov in May 2021 who compressed the 1GB file down to just 115MB.
The compression techniques used are pretty magical, and the contest as a whole is a great rabbit hole to get lost in for an afternoon.
Hours and hours of detailed video content building breadboard computers. Fantastic for learning how computers work, both the 8-bit and 6502 computers from scratch series are a great intro into that space. Covering from the very bottom with logic gates, all the way to functional breadboard computers and ASM.
Programming challenges with the winner decided by the fewest number of bytes in the source code. A celebration of language quirks, specialised languages, arguably useless optimisation.
Code Golf thread for the classic Fizz Buzz programming exercise, great examples of some of the fun and crazy solutions across various languages.
It’s full of impressive ingenuity, solving useless problems. Here are some of my favourite answers across the site:
3 } 1 " $ .
! $ > ) } g 4
_ . { $ ' ) ) \
< $ \ . \ . @ \ }
F \ $ / ; z ; u ; <
% < _ > _ . . $ > B /
< > } ) ) ' % < > {
> ; e " - < / _ %
; \ / { } / > .
\ ; . z ; i ;
. . > ( ( '
# Python 2, 56 bytes
i=0;exec"print i%3/2*'Fizz'+i%5/4*'Buzz'or-~i;i+=1;"*100
A great read on pushing Python-based object detection pipeline to the limit. Even more impressive is the two follow on articles, pushing all the way to 2530 FPS.
]]>First ever real programming project with a purpose was a chrome browser extension. My school at the time used an online Virtual Learning Environment (VLE) to set homework tasks and share files/etc. for classes.
The homework/task list was extremely inefficient - just a list of homework titles and when it was due:
[ Assignment ] | [Due Date]
Maths - Complete the ... | [Tomorrow]
English - Read from 3... | [Tomorrow]
Physics - Past paper ... | [ Friday ]
This resulted in having to open a bunch of pages to see the full homework tasks and to be able to plan what work to do and when.
Even marking off tasks that you had already completed required opening another webpage!!
In order to fix this, I learnt the basics of JavaScript, JQuery and building chrome extensions. Using JQuery I called each of the links to tasks, and added extra columns to show the full description on the main page, as well as to add the mark as done button to each task.
This resulted in a page that looked more like this:
[ Assignment ] | [Due Date] | [Mark As Done]
Maths - Complete the handout | [Tomorrow] | [X]
English - Read from 314 - 512, make notes | [Tomorrow] | [ ]
Physics - Finish past paper questions from class | [ Friday ] | [X]
The implementation was messy, and since it loaded two pages for every task set - extremely slow. And whilst it wasn’t the biggest change, it streamlined the user experience, and so achieved its purpose.
]]>