Post-installation steps for Linux. Estimated reading time: 15 minutes. This section contains optional procedures for configuring Linux hosts to work better with Docker. Manage Docker as a non-root user. The Docker daemon binds to a Unix socket instead of a TCP port. OpenJDK on Alpine Linux. When running Docker containers, you want them to as small as possible to allow quick starting, stopping, downloading, scaling, etc. Alpine Linux is a suitable Linux distribution for small containers and is being used quite often. There can be some thread challenges with Alpine Linux though. See for example here and here. Example Dockerfile for your own Node.js project. If you're doing your npm install/npm ci or yarn install from your Dockerfile, then you'll probably want to add nodemodules to your.dockerignore file first, so that it doesn't get sent to the docker daemon. Node.js is a JavaScript-based platform for server-side and networking applications.

When you’re choosing a base image for your Docker image, Alpine Linux is often recommended.Using Alpine, you’re told, will make your images smaller and speed up your builds.And if you’re using Go that’s reasonable advice.

But if you’re using Python, Alpine Linux will quite often:

1. Make your builds much slower.
2. Make your images bigger.
3. Waste your time.
4. On occassion, introduce obscure runtime bugs.

Let’s see why Alpine is recommended, and why you probably shouldn’t use it for your Python application.

Why people recommend Alpine

Let’s say we need to install gcc as part of our image build, and we want to see how Alpine Linux compares to Ubuntu 18.04 in terms of build time and image size.

First, I’ll pull both images, and check their size:

As you can see, the base image for Alpine is much smaller.

Next, we’ll try installing gcc in both of them.First, with Ubuntu:

Note: Outside the very specific topic under discussion, the Dockerfiles in this article are not examples of best practices, since the added complexity would obscure the main point of the article.

To ensure you’re writing secure, correct, fast Dockerfiles, consider my Python on Docker Production Handbook, which includes a packaging process and >70 best practices.

We can then build and time that:

Now let’s make the equivalent Alpine Dockerfile:

And again, build the image and check its size:

As promised, Alpine images build faster and are smaller: 15 seconds instead of 30 seconds, and the image is 105MB instead of 150MB.That’s pretty good!

But when we switch to packaging a Python application, things start going wrong.

Let’s build a Python image

We want to package a Python application that uses pandas and matplotlib.So one option is to use the Debian-based official Python image (which I pulled in advance), with the following Dockerfile:

And when we build it:

The resulting image is 363MB.

Can we do better with Alpine? Let’s try:

And now we build it:

What’s going on?

Standard PyPI wheels don’t work on Alpine

If you look at the Debian-based build above, you’ll see it’s downloading matplotlib-3.1.2-cp38-cp38-manylinux1_x86_64.whl.This is a pre-compiled binary wheel.Alpine, in contrast, downloads the source code (matplotlib-3.1.2.tar.gz), because standard Linux wheels don’t work on Alpine Linux.

Why?Most Linux distributions use the GNU version (glibc) of the standard C library that is required by pretty much every C program, including Python.But Alpine Linux uses musl, those binary wheels are compiled against glibc, and therefore Alpine disabled Linux wheel support.

Most Python packages these days include binary wheels on PyPI, significantly speeding install time.But if you’re using Alpine Linux you need to compile all the C code in every Python package that you use.

Which also means you need to figure out every single system library dependency yourself.In this case, to figure out the dependencies I did some research, and ended up with the following updated Dockerfile:

And then we build it, and it takes…

… 25 minutes, 57 seconds! And the resulting image is 851MB.

Here’s a comparison between the two base images:

Base image	Time to build	Image size	Research required
python:3.8-slim	30 seconds	363MB	No
python:3.8-alpine	1557 seconds	851MB	Yes

Alpine builds are vastly slower, the image is bigger, and I had to do a bunch of research.

Can’t you work around these issues?

Build time

For faster build times, Alpine Edge, which will eventually become the next stable release, does have matplotlib and pandas.And installing system packages is quite fast.As of January 2020, however, the current stable release does not include these popular packages.

Even when they are available, however, system packages almost always lag what’s on PyPI, and it’s unlikely that Alpine will ever package everything that’s on PyPI.In practice most Python teams I know don’t use system packages for Python dependencies, they rely on PyPI or Conda Forge.

Image size

Some readers pointed out that you can remove the originally installed packages, or add an option not to cache package downloads, or use a multi-stage build.One reader attempt resulted in a 470MB image.

So yes, you can get an image that’s in the ballpark of the slim-based image, but the whole motivation for Alpine Linux is smaller images and faster builds.With enough work you may be able to get a smaller image, but you’re still suffering from a 1500-second build time when they you get a 30-second build time using the python:3.8-slim image.

But wait, there’s more!

Alpine Linux can cause unexpected runtime bugs

While in theory the musl C library used by Alpine is mostly compatible with the glibc used by other Linux distributions, in practice the differences can cause problems.And when problems do occur, they are going to be strange and unexpected.

Some examples:

1. Alpine has a smaller default stack size for threads, which can lead to Python crashes.
2. One Alpine user discovered that their Python application was much slower because of the way musl allocates memory vs. glibc.
3. I once couldn’t do DNS lookups in Alpine images running on minikube (Kubernetes in a VM) when using the WeWork coworking space’s WiFi.The cause was a combination of a bad DNS setup by WeWork, the way Kubernetes and minikube do DNS, and musl’s handling of this edge case vs. what glibc does.musl wasn’t wrong (it matched the RFC), but I had to waste time figuring out the problem and then switching to a glibc-based image.
4. Another user discovered issues with time formatting and parsing.

Most or perhaps all of these problems have already been fixed, but no doubt there are more problems to discover.Random breakage of this sort is just one more thing to worry about.

Don’t use Alpine Linux for Python images

Unless you want massively slower build times, larger images, more work, and the potential for obscure bugs, you’ll want to avoid Alpine Linux as a base image.For some recommendations on what you should use, see my article on choosing a good base image.

Estimated reading time: 15 minutes

This section contains optional procedures for configuring Linux hosts to workbetter with Docker.

Manage Docker as a non-root user

The Docker daemon binds to a Unix socket instead of a TCP port. By defaultthat Unix socket is owned by the user root and other users can only access itusing sudo. The Docker daemon always runs as the root user.

If you don’t want to preface the docker command with sudo, create a Unixgroup called docker and add users to it. When the Docker daemon starts, itcreates a Unix socket accessible by members of the docker group.

Warning

The docker group grants privileges equivalent to the rootuser. For details on how this impacts security in your system, seeDocker Daemon Attack Surface.

Note:

To run Docker without root privileges, seeRun the Docker daemon as a non-root user (Rootless mode).

To create the docker group and add your user:

1. Create the docker group.

2. Add your user to the docker group.

3. Log out and log back in so that your group membership is re-evaluated.

   If testing on a virtual machine, it may be necessary to restart the virtual machine for changes to take effect.

   On a desktop Linux environment such as X Windows, log out of your session completely and then log back in.

   On Linux, you can also run the following command to activate the changes to groups:

4. Verify that you can run docker commands without sudo.

   This command downloads a test image and runs it in a container. When thecontainer runs, it prints an informational message and exits.

   If you initially ran Docker CLI commands using sudo before addingyour user to the docker group, you may see the following error,which indicates that your ~/.docker/ directory was created withincorrect permissions due to the sudo commands.

   To fix this problem, either remove the ~/.docker/ directory(it is recreated automatically, but any custom settingsare lost), or change its ownership and permissions using thefollowing commands:

Configure Docker to start on boot

Install Docker On Alpine Linux

Most current Linux distributions (RHEL, CentOS, Fedora, Debian, Ubuntu 16.04 andhigher) use systemd to manage which services start when the systemboots. On Debian and Ubuntu, the Docker service is configured to start on bootby default. To automatically start Docker and Containerd on boot for otherdistros, use the commands below:

To disable this behavior, use disable instead.

If you need to add an HTTP Proxy, set a different directory or partition for theDocker runtime files, or make other customizations, seecustomize your systemd Docker daemon options.

Use a different storage engine

For information about the different storage engines, seeStorage drivers.The default storage engine and the list of supported storage engines depend onyour host’s Linux distribution and available kernel drivers.

Configure default logging driver

Docker provides the capability tocollect and view log data from all containers running on a host via a series oflogging drivers. The default logging driver, json-file, writes log data toJSON-formatted files on the host filesystem. Over time, these log files expandin size, leading to potential exhaustion of disk resources.

To alleviate such issues, either configure the json-file logging driver toenable log rotation, use analternative logging driversuch as the “local” logging driverthat performs log rotation by default, or use a logging driver that sendslogs to a remote logging aggregator.

Configure where the Docker daemon listens for connections

By default, the Docker daemon listens for connections on a UNIX socket to acceptrequests from local clients. It is possible to allow Docker to accept requestsfrom remote hosts by configuring it to listen on an IP address and port as wellas the UNIX socket. For more detailed information on this configuration optiontake a look at “Bind Docker to another host/port or a unix socket” section ofthe Docker CLI Reference article.

Install Docker On Alpine Linux Mint

Secure your connection

Before configuring Docker to accept connections from remote hosts it is critically important that youunderstand the security implications of opening docker to the network. If steps are not taken to secure the connection, it is possible for remote non-root users to gain root access on the host. For more information on how to use TLS certificates to secure this connection, check this article on how to protect the Docker daemon socket.

Configuring Docker to accept remote connections can be done with the docker.servicesystemd unit file for Linux distributions using systemd, such as recent versionsof RedHat, CentOS, Ubuntu and SLES, or with the daemon.json file which isrecommended for Linux distributions that do not use systemd.

systemd vs daemon.json

Configuring Docker to listen for connections using both the systemd unit file and the daemon.json file causes a conflict that prevents Docker from starting.

Configuring remote access with systemd unit file

1. Use the command sudo systemctl edit docker.service to open an override file for docker.service in a text editor.

2. Add or modify the following lines, substituting your own values.

3. Save the file.

4. Reload the systemctl configuration.

5. Restart Docker.

6. Check to see whether the change was honored by reviewing the output of netstat to confirm dockerd is listening on the configured port.

Configuring remote access with daemon.json

1. Set the hosts array in the /etc/docker/daemon.json to connect to the UNIX socket and an IP address, as follows:

2. Restart Docker.

3. Check to see whether the change was honored by reviewing the output of netstat to confirm dockerd is listening on the configured port.

Enable IPv6 on the Docker daemon

To enable IPv6 on the Docker daemon, seeEnable IPv6 support.

Troubleshooting

Kernel compatibility

Docker cannot run correctly if your kernel is older than version 3.10 or if itis missing some modules. To check kernel compatibility, you can download andrun the check-config.shscript.

The script only works on Linux, not macOS.

Cannot connect to the Docker daemon

If you see an error such as the following, your Docker client may be configuredto connect to a Docker daemon on a different host, and that host may not bereachable.

To see which host your client is configured to connect to, check the value ofthe DOCKER_HOST variable in your environment.

If this command returns a value, the Docker client is set to connect to aDocker daemon running on that host. If it is unset, the Docker client is set toconnect to the Docker daemon running on the local host. If it is set in error,use the following command to unset it:

You may need to edit your environment in files such as ~/.bashrc or~/.profile to prevent the DOCKER_HOST variable from being seterroneously.

If DOCKER_HOST is set as intended, verify that the Docker daemon is runningon the remote host and that a firewall or network outage is not preventing youfrom connecting.

IP forwarding problems

If you manually configure your network using systemd-network with systemdversion 219 or higher, Docker containers may not be able to access your network.Beginning with systemd version 220, the forwarding setting for a given network(net.ipv4.conf.<interface>.forwarding) defaults to off. This settingprevents IP forwarding. It also conflicts with Docker’s behavior of enablingthe net.ipv4.conf.all.forwarding setting within containers.

To work around this on RHEL, CentOS, or Fedora, edit the <interface>.networkfile in /usr/lib/systemd/network/ on your Docker host(ex: /usr/lib/systemd/network/80-container-host0.network) and add thefollowing block within the [Network] section.

This configuration allows IP forwarding from the container as expected.

DNS resolver found in resolv.conf and containers can't use it

Linux systems which use a GUI often have a network manager running, which uses adnsmasq instance running on a loopback address such as 127.0.0.1 or127.0.1.1 to cache DNS requests, and adds this entry to/etc/resolv.conf. The dnsmasq service speeds upDNS look-ups and also provides DHCP services. This configuration does not workwithin a Docker container which has its own network namespace, becausethe Docker container resolves loopback addresses such as 127.0.0.1 toitself, and it is very unlikely to be running a DNS server on its ownloopback address.

If Docker detects that no DNS server referenced in /etc/resolv.conf is a fullyfunctional DNS server, the following warning occurs and Docker uses the publicDNS servers provided by Google at 8.8.8.8 and 8.8.4.4 for DNS resolution.

If you see this warning, first check to see if you use dnsmasq:

If your container needs to resolve hosts which are internal to your network, thepublic nameservers are not adequate. You have two choices:

• You can specify a DNS server for Docker to use, or
• You can disable dnsmasq in NetworkManager. If you do this, NetworkManageradds your true DNS nameserver to /etc/resolv.conf, but you lose thepossible benefits of dnsmasq.

You only need to use one of these methods.

Specify DNS servers for Docker

The default location of the configuration file is /etc/docker/daemon.json. Youcan change the location of the configuration file using the --config-filedaemon flag. The documentation below assumes the configuration file is locatedat /etc/docker/daemon.json.

1. Create or edit the Docker daemon configuration file, which defaults to/etc/docker/daemon.json file, which controls the Docker daemonconfiguration.

2. Add a dns key with one or more IP addresses as values. If the file hasexisting contents, you only need to add or edit the dns line.

   If your internal DNS server cannot resolve public IP addresses, include atleast one DNS server which can, so that you can connect to Docker Hub and sothat your containers can resolve internet domain names.

   Save and close the file.

3. Restart the Docker daemon.

4. Verify that Docker can resolve external IP addresses by trying to pull animage:

5. If necessary, verify that Docker containers can resolve an internal hostnameby pinging it.

Disable dnsmasq

Ubuntu

If you prefer not to change the Docker daemon’s configuration to use a specificIP address, follow these instructions to disable dnsmasq in NetworkManager.

1. Edit the /etc/NetworkManager/NetworkManager.conf file.

2. Comment out the dns=dnsmasq line by adding a # character to the beginningof the line.

   Save and close the file.

3. Restart both NetworkManager and Docker. As an alternative, you can rebootyour system.

RHEL, CentOS, or Fedora

To disable dnsmasq on RHEL, CentOS, or Fedora:

1. Disable the dnsmasq service:

2. Configure the DNS servers manually using theRed Hat documentation.

Allow access to the remote API through a firewall

If you run a firewall on the same host as you run Docker and you want to accessthe Docker Remote API from another host and remote access is enabled, you needto configure your firewall to allow incoming connections on the Docker port,which defaults to 2376 if TLS encrypted transport is enabled or 2375otherwise.

Two common firewall daemons areUFW (Uncomplicated Firewall) (oftenused for Ubuntu systems) and firewalld (often usedfor RPM-based systems). Consult the documentation for your OS and firewall, butthe following information might help you get started. These options are fairlypermissive and you may want to use a different configuration that locks yoursystem down more.

• UFW: Set DEFAULT_FORWARD_POLICY='ACCEPT' in your configuration.

• firewalld: Add rules similar to the following to your policy (one forincoming requests and one for outgoing requests). Be sure the interface namesand chain names are correct.

Your kernel does not support cgroup swap limit capabilities

On Ubuntu or Debian hosts, You may see messages similar to the following whenworking with an image.

This warning does not occur on RPM-based systems, which enable thesecapabilities by default.

If you don’t need these capabilities, you can ignore the warning. You can enablethese capabilities on Ubuntu or Debian by following these instructions. Memoryand swap accounting incur an overhead of about 1% of the total available memoryand a 10% overall performance degradation, even if Docker is not running.

1. Log into the Ubuntu or Debian host as a user with sudo privileges.

2. Edit the /etc/default/grub file. Add or edit the GRUB_CMDLINE_LINUX lineto add the following two key-value pairs:

   Save and close the file.

3. Update GRUB.

   If your GRUB configuration file has incorrect syntax, an error occurs.In this case, repeat steps 2 and 3.

   The changes take effect when the system is rebooted.

Next steps

• Take a look at the Get started training modules to learn how to build an image and run it as a containerized application.
• Review the topics in Develop with Docker to learn how to build new applications using Docker.

Docker, Docker documentation, requirements, apt, installation, ubuntu, install, uninstall, upgrade, update