Docker is weird

shykes

Hi, I'm the creator of Docker and came across this post. It's always fun to see other engineers think about how you designed a system. I haven't been involved in developing Docker in many years, but it's a fun trip down memory lane. I'm happy to answer any questions you may have.

suprjami

Always nice to see someone demystifying a complex thing by learning the building blocks it's made from. Knowing this, you might find rootless Podman's fork-exec method of starting containers to be more sensible. I certainly do. No root, no daemon, no socket. If you want to learn more about the Linux Kernel namespaces which work together to make a container, this LWN series is really good: https://lwn.net/Articles/531114/

QGQBGdeZREunxLe

A good talk on the tech behind containers. Containers From Scratch • Liz Rice • GOTO 2018 https://www.youtube.com/watch?v=8fi7uSYlOdc

tyingq

Poke around in bocker[1]. For technical people, it makes the same points the article makes in a way that's very clear. [1] "Docker implemented in around 100 lines of bash" https://github.com/p8952/bocker

gzapp

That part about the threading was a huge source of issues back in the early days. Goroutines could hop threads and it's the threads namespace/cgroup configuration that gets inherited when forking.. One of the gnarliest issues I recall was where network namespaces could occasionally not be cleaned up correctly. So "randomly" (but more likely under certain loads) the networking of a reused thread would get jacked and Docker could no longer pull images.

j16sdiz

Every statement in the article make me ask: "what's weird about it?" Everything in that list feel natural when you know how things evolves over many years. Maybe it is just me getting older and less curious.

assimpleaspossi

>>Images are just folders that Docker puts somewhere and then calls chroot (or pivot_root) to make that folder your container process’s new root directory. And folders are graphical representations of directories--sometimes, not always--and not directories themselves. When will the newbies get this?

slowin

I prefer Podman most of the time, but Docker is/was amazing and a great DX put on top of the so called "5 different linux kernel features in trenchcoat". That's the selling point, it's super lightweight and takes full advantage of the OS features that were already there (or being developed) but were awkward to use and required tons of specialized knowledge.

kazinator

> And if you know where to look, you can even find that process in /proc. This is very important for debugging! Not just that process, but the processes running in that Docker can be found in /proc. Why this is useful is that you can run gdb outside of that docker and point it to a process in the docker, including setting the file system root to the docker one, so that it gdb reads the correct objects from the correct paths, to get the right symbols and addresses.

miladyincontrol

For as "terrible of a language" Go is alleged to be for a container runtime, its not like crun or youki are exactly faster in many meaningful ways unless you count churning containers running true and exiting. And implying the system kernel still has to run the container kinda ignores kata containers and gvisor runtimes.

0xbadcafebee

Docker hasn't required root for 7 years, I wish people would stop repeating that. It uses a root daemon by default because it makes things easier. Also, Containers are only as fake as a "Desktop" in an OS, which isn't real either. It's a user, network & process namespace with a control bus and a bunch of other weird things. A Desktop is a Container with a GUI. Other weird facts about OSes: a filesystem is a database, the Linux kernel is a monolithic program, "hidden" dot-files were a Unix bug that got carried forward, deleting a file doesn't remove the file from your machine, virtual memory is a lie, your computer can't run multiple programs at once (it's just running one at a time real fast), in 12 years most systems will crash due to one number getting too big, you technically never need to reboot because you can launch a different version of Linux while already running Linux, you can freeze a program to a file and then later resume running that same program in the same state (but nobody uses it), files aren't actually stored in directories, there's a maximum number of files (and size of file) on most filesystems (which you can use to lock up a machine, despite file quotas existing and nobody using them), you can lock up most systems by forking too many times / using too much RAM / accessing disk too much, there's a default number of open files and/or TCP ports which is artificially low and you can use to again lock up a machine, pipes are just two file descriptors connected to one memory buffer, an IP address is just one long number split up into bytes, etc.

fulafel

> I decided to make my own container runtime for fun [...] Unlike VMs, containers are just processes Try to make your own VM runtime next, you might be surprised!

schmichael

> Go is kind of a terrible language for a container runtime I love Go and use it for everything I can but will absolutely concede this point. It's clearly possible to write the most successful container runtimes on earth in Go, but you absolutely have to fight the language/runtime sometimes. Since containers are based on processes, handing off a container spec to a minimal rust runtime seems like what we should all be doing by now. crun has just proven good enough that AFAIK no one has RIIR.

adrianlee_dev

This is a better way to understand it: we can think of a container as an isolated process. When deploying containers, having a clear understanding of the boundaries between containers and the host machine in terms of permissions, resources, and data is extremely helpful for troubleshooting issues.

thayne

Today, I would definitely choose rust over go. And I lament that so much container software uses go rather than rust. But back when docker started rust was a lot less mature and probably wouldn't have been a good choice.

sreekanth850

I started hating Docker after seeing all the cargo culting on Reddit. Whenever someone asks about deploying an app on Linux, there's always at least one comment saying, Just use Docker. We finally ended up building our own deployment automation tool for our entire stack that doesn't use Docker at all.

VCFundedGenYer

Docker does not need root. Side note almost nothing about Docker is weird. This article chooses rather boring design decisions and picks on them.

vbezhenar

What made Docker unique IMO is not the container technology itself. I used chroot 20 years ago and it worked just fine. Yes, no network namespaces; no process namespaces, but I don't even think these are that important. What made Docker unique were there things: 1. Dockerfile concept. Building images layer by layer. Caching layers. Everything in a single file. It made possible to iterate on image quickly. It made possible to implement CI builds efficiently. 2. Docker network registry concept. Putting universal images onto a server and share them with the world. 3. Momentum. A lot of software that was traditionally complicated to deploy, suddenly appeared in docker servers. I remember how fascinated I was, when I could deploy wordpress website with tiny docker-compose. And how contained it was. I could just delete every trace of it from my host. I always felt uneasy installing various things onto my host and docker just solved that uneasiness.

Pxtl

Honestly it took me so long to pick up containerisation because of how it's marketed. "Orchestrate a zillion fake computers, ultralight VMs, livestock not pets" big whoop. "It's an idempotent language for OS level dependencies so you can be sure exactly what is running on your server" Gimme gimme gimme!

razighter777

This only scratches the tip of the iceberg. People's minds are blown when they realize Linux has no concept of a container. It's not a vm: It's all namespaces, baby! Unshare one or more of your mnt, pid, net, ipc, uts, user, cgroup, or time namespaces and you have a "container".