Container management with Docker

Copyright Notice

This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise.

Author

Kamil Baran - http://www.kamilbaran.pl

What is Docker?

World before Docker
- no virtualization
- hypervisor-based virtualization
- container-based virtualization

Architecture

Docker uses a client-server architecture

Main Docker elements

deamon: process that runs on a host machine (server)
client: primary Docker interface
image: read-only template (build component)
registry: public or private image repositories (distribution, ship component)
container: created form image, holds everything that is needed for an application to run (run component)

Benefits of Docker

separation of roles and concerns
- developers focuses on building applications
- system administrators focuses on deployment
portability: build in one environment, distributed and run on many others
faster development, testing, deployment
scalability: easy to spin up new containers or migrate to more powerful hosts
better resource utilization: more apps on one host

The underlying technology

namespaces
- pid namespace: used for process isolation (Process ID)
- net namespace: used for managing network interfaces
- mnt namespace: used for managing mount-points
- ipc namespace: used for managing access to IPC resources (InterProcess Communication)
- uts namespace: used for isolating kernel and version identifiers (Unix Timesharing System)
control groups (cgroups)
- used for sharing available hardware resources
- and setting up limits and constraints
union file system (UnionFS)
- file system that operate by creating layers
- many layers are merged and visible as one consistent file system
- many available file systems: AUFS, btrfs, vfs, DeviceMapper
container format
- two supported container formats: libcontainer, LXC

Getting started

Installation of Docker engine and client

# easiest way to install Docker
$ wget -qO- https://get.docker.com/ | sh
# to use docker as ubuntu user without using sudo (optional)
$ sudo usermod -aG docker student
# to check the installation
$ docker version

Manual installation steps, more details and installation instructions for other operating systems

Hello world example

$ docker run hello-world
$ docker images
$ docker ps -a

Dockerized bash terminal

$ docker run -it ubuntu
$ docker run -it ubuntu:latest
$ docker run -it ubuntu:14.04 bash
$ docker run -it ubuntu ps -aux

docker run -t: allocate a pseudo-tty
docker run -i (--interactive): keep STDIN open even if not attached
use CTRL + p + q to detach from running container
use attach command to reattach to a detached container

$ docker attach container_name

the importance of PID 1
PID in the container and Docker host:

$ ps -fe | grep $(pidof docker)

installing packages: mc, vim

Investigating containers and images

docker inspect displays low-level information on a container or image

$ docker inspect webapp
$ docker inspect --format='{{.NetworkSettings.IPAddress}}' webapp
$ docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' webapp

docker diff displays changes on a container's filesystem

$ docker diff webapp
$ docker diff webapp | grep ^A

docker history shows the history of an image (layers)

$ docker history ubuntu
$ docker history --no-trunc ubuntu

docker logs fetches the logs of a container

$ docker logs webapp
$ docker logs --tail 15 webapp
$ docker logs -f webapp
$ docker logs -t webapp
$ docker logs --since 1h webapp

docker top displays all running processes of a container

$ docker top webapp
$ docker exec webapp ps aux

Cleaning up and keeping clean

$ docker image ls
$ docker ps -a

docker run --rm automatically removes the container when it exits
docker run --name assigns your own, meaningful name of the container

$ docker run --rm -it --name=test ubuntu bash

container name must be unique and you can change it whenever you like

$ docker rename test meaningful_name

docker rm removes a container
docker image rm removes an image

$ docker rm meaningful_name

# removes all not running containers
$ docker rm $(docker ps -aqf "status=exited")
$ docker rm $(docker ps -a | grep 'Exited' | awk '{print $1}')

# removes all images without a tag (dangling)
$ docker image prune
$ docker image rm $(docker image ls -qf "dangling=true")
$ docker rmi $(docker images | grep '^<none>' | awk '{print $3}')

Exercises

Storage and data persistence

Within the container

data is only visible inside the container
data is not persisted outside of the container
data is lost if the container is removed

Directly on Docker host

$ docker run -v /host/dir:/container/dir:rw ...
$ docker run -v /home/ubuntu/docker/training_httpd1/html:/var/www/html/ -d --name www --net host training:httpd1
$ docker run -v $PWD/html:/var/www/html/ -d --name www --net host training:httpd1
$ docker run -v $PWD/html:/var/www/html/:ro -d --name www --net host training:httpd1

data is visible inside the container, Docker host and can be shared between containers
data is persisted outside of the container even if the container is removed
this provides near bare metal performance
host directory can be an existing NFS share, formatted block device or anything that can be mounted on Docker host

Docker Volumes

$ docker run -itd -v /data --name data1 ubuntu
$ docker inspect data1
$ docker exec data1 touch /data/file1
$ docker exec data1 ls -l /data/
$ docker run -itd --volumes-from data1 --name data2 ubuntu
$ docker inspect data2
$ docker rm -fv data1 data2

$ docker volume create --name kb_volume
$ docker run --rm -it -v kb_volume:/data ubuntu touch /data/kb
$ docker run --rm -it -v kb_volume:/data ubuntu ls -l /data
$ docker volume rm kb_volume
$ docker volume ls

data is visible inside the container and can be shared between containers
data is persisted outside of the container even if the container is removed
- use docker run rm -v to remove a container with its volumes (unless the other container uses them)
this provides near bare metal performance
it solves the problem with privileges (users and groups with different IDs on host and in the container)

Creating groups and users with custom ID

$ groupadd -r -g 27017  mongodb
$ useradd -r -u 27017 -g mongodb mongodb

Backup and restore data from volumes

# backup data from one container
$ docker run -itd -v /data --name data1 ubuntu
$ docker exec data1 touch /data/file1
$ docker exec data1 chown www-data:www-data /data/file1
$ docker run --rm --volumes-from data1 ubuntu ls -l /data
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu tar -cvpf /backup/backup.tar /data
$ docker rm -fv data1
# restore data into brand new container
$ docker run -itd -v /data --name data1 ubuntu
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu tar -xvpf /backup/backup.tar
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu bash -c "cd /data && tar -xvf /backup/backup.tar --strip 1"
$ docker run --rm --volumes-from data1 ubuntu ls -l /data

Outside Docker Host

Networking

$ docker network --help
$ docker network ls

Docker host network

the host network adds a container on the hosts network stack
the network configuration inside the container is identical to the host

$ docker run --name db1  -d --net host training:mongod
$ docker run --name www -d --net host training:httpd1
$ docker inspect www
$ docker network inspect host

Without network interface

the none network adds a container to a container-specific network stack
use docker exec command to connect to the container

$ docker run --name networkless --net none -it --rm ubuntu bash

Default network (bridge)

$ docker run --name db2 -d --volumes-from db1 training:mongod
$ docker run --name www -d training:httpd1
$ docker run --name www -d -P training:httpd1
$ docker run --name www -d -p 80:80 training:httpd1
$ docker run --name www -d -p 127.0.0.1:88:80 training:httpd1
$ docker run --name www -d -p 80:80 --link db2 training:httpd1
$ docker run --name www -d -p 80:80 --link db2:db training:httpd1
$ docker inspect www
$ docker network inspect host

this is a default network for all containers
containers are able to communicate with each other using IP addresses
Docker does not support automatic service discovery on the default bridge network
to communicate by using names in this network, you must connect the containers via the legacy docker run --link option

Custom user networks (bridge)

$ docker network create --driver bridge --subnet 10.1.2.0/24 net1
$ docker network create --subnet 10.1.2.0/24 --gateway=10.1.2.1 net1
$ docker run -d --network net1 --name mongo --network-alias db training:mongod
$ docker run -d --network net1 --name apache --network-alias www --hostname httpd -p 80:80 -e MONGO_CS="mongodb://mongo:27017" training:httpd
$ docker network ls
$ docker network inspect net1
$ docker network disconnect net1 www
$ docker network connect net1 www