Container management with Docker

From Training Material
Revision as of 13:27, 6 April 2020 by Kbaran (talk | contribs) (→‎Custom user networks (bridge))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search


Author

What is Docker?

  • World before Docker
    • no virtualization
    • hypervisor-based virtualization
    • container-based virtualization

Architecture

  • Docker uses a client-server architecture

Architecture.svg

Main Docker elements

  • deamon: process that runs on a host machine (server)
  • client: primary Docker interface
  • image: read-only template (build component)
  • registry: public or private image repositories (distribution, ship component)
  • container: created form image, holds everything that is needed for an application to run (run component)


Benefits of Docker

  • separation of roles and concerns
    • developers focuses on building applications
    • system administrators focuses on deployment
  • portability: build in one environment, distributed and run on many others
  • faster development, testing, deployment
  • scalability: easy to spin up new containers or migrate to more powerful hosts
  • better resource utilization: more apps on one host


The underlying technology

  • namespaces
    • pid namespace: used for process isolation (Process ID)
    • net namespace: used for managing network interfaces
    • mnt namespace: used for managing mount-points
    • ipc namespace: used for managing access to IPC resources (InterProcess Communication)
    • uts namespace: used for isolating kernel and version identifiers (Unix Timesharing System)
  • control groups (cgroups)
    • used for sharing available hardware resources
    • and setting up limits and constraints
  • union file system (UnionFS)
    • file system that operate by creating layers
    • many layers are merged and visible as one consistent file system
    • many available file systems: AUFS, btrfs, vfs, DeviceMapper
  • container format
    • two supported container formats: libcontainer, LXC


Getting started

Installation of Docker engine and client

# easiest way to install Docker
$ wget -qO- https://get.docker.com/ | sh
# to use docker as ubuntu user without using sudo (optional)
$ sudo usermod -aG docker student
# to check the installation
$ docker version


Hello world example

$ docker run hello-world
$ docker images
$ docker ps -a


Dockerized bash terminal

$ docker run -it ubuntu
$ docker run -it ubuntu:latest
$ docker run -it ubuntu:14.04 bash
$ docker run -it ubuntu ps -aux
  • docker run -t: allocate a pseudo-tty
  • docker run -i (--interactive): keep STDIN open even if not attached
  • use CTRL + p + q to detach from running container
  • use attach command to reattach to a detached container
$ docker attach container_name
  • the importance of PID 1
  • PID in the container and Docker host:
$ ps -fe | grep $(pidof docker)
  • installing packages: mc, vim


Investigating containers and images

  • docker inspect displays low-level information on a container or image
$ docker inspect webapp
$ docker inspect --format='{{.NetworkSettings.IPAddress}}' webapp
$ docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' webapp
  • docker diff displays changes on a container's filesystem
$ docker diff webapp
$ docker diff webapp | grep ^A
  • docker history shows the history of an image (layers)
$ docker history ubuntu
$ docker history --no-trunc ubuntu
  • docker logs fetches the logs of a container
$ docker logs webapp
$ docker logs --tail 15 webapp
$ docker logs -f webapp
$ docker logs -t webapp
$ docker logs --since 1h webapp
  • docker top displays all running processes of a container
$ docker top webapp
$ docker exec webapp ps aux


Cleaning up and keeping clean

$ docker image ls
$ docker ps -a
  • docker run --rm automatically removes the container when it exits
  • docker run --name assigns your own, meaningful name of the container
$ docker run --rm -it --name=test ubuntu bash
  • container name must be unique and you can change it whenever you like
$ docker rename test meaningful_name
  • docker rm removes a container
  • docker image rm removes an image
$ docker rm meaningful_name
# removes all not running containers
$ docker rm $(docker ps -aqf "status=exited")
$ docker rm $(docker ps -a | grep 'Exited' | awk '{print $1}')

# removes all images without a tag (dangling)
$ docker image prune
$ docker image rm $(docker image ls -qf "dangling=true")
$ docker rmi $(docker images | grep '^<none>' | awk '{print $3}')

Exercises

Storage and data persistence

Within the container

  • data is only visible inside the container
  • data is not persisted outside of the container
  • data is lost if the container is removed


Directly on Docker host

$ docker run -v /host/dir:/container/dir:rw ...
$ docker run -v /home/ubuntu/docker/training_httpd1/html:/var/www/html/ -d --name www --net host training:httpd1
$ docker run -v $PWD/html:/var/www/html/ -d --name www --net host training:httpd1
$ docker run -v $PWD/html:/var/www/html/:ro -d --name www --net host training:httpd1


  • data is visible inside the container, Docker host and can be shared between containers
  • data is persisted outside of the container even if the container is removed
  • this provides near bare metal performance
  • host directory can be an existing NFS share, formatted block device or anything that can be mounted on Docker host


Docker Volumes

$ docker run -itd -v /data --name data1 ubuntu
$ docker inspect data1
$ docker exec data1 touch /data/file1
$ docker exec data1 ls -l /data/
$ docker run -itd --volumes-from data1 --name data2 ubuntu
$ docker inspect data2
$ docker rm -fv data1 data2


$ docker volume create --name kb_volume
$ docker run --rm -it -v kb_volume:/data ubuntu touch /data/kb
$ docker run --rm -it -v kb_volume:/data ubuntu ls -l /data
$ docker volume rm kb_volume
$ docker volume ls


  • data is visible inside the container and can be shared between containers
  • data is persisted outside of the container even if the container is removed
    • use docker run rm -v to remove a container with its volumes (unless the other container uses them)
  • this provides near bare metal performance
  • it solves the problem with privileges (users and groups with different IDs on host and in the container)


Creating groups and users with custom ID

$ groupadd -r -g 27017  mongodb
$ useradd -r -u 27017 -g mongodb mongodb


Backup and restore data from volumes

# backup data from one container
$ docker run -itd -v /data --name data1 ubuntu
$ docker exec data1 touch /data/file1
$ docker exec data1 chown www-data:www-data /data/file1
$ docker run --rm --volumes-from data1 ubuntu ls -l /data
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu tar -cvpf /backup/backup.tar /data
$ docker rm -fv data1
# restore data into brand new container
$ docker run -itd -v /data --name data1 ubuntu
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu tar -xvpf /backup/backup.tar
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu bash -c "cd /data && tar -xvf /backup/backup.tar --strip 1"
$ docker run --rm --volumes-from data1 ubuntu ls -l /data


Outside Docker Host


Networking

$ docker network --help
$ docker network ls


Docker host network

  • the host network adds a container on the hosts network stack
  • the network configuration inside the container is identical to the host
$ docker run --name db1  -d --net host training:mongod
$ docker run --name www -d --net host training:httpd1
$ docker inspect www
$ docker network inspect host


Without network interface

  • the none network adds a container to a container-specific network stack
  • use docker exec command to connect to the container
$ docker run --name networkless --net none -it --rm ubuntu bash


Default network (bridge)

$ docker run --name db2 -d --volumes-from db1 training:mongod
$ docker run --name www -d training:httpd1
$ docker run --name www -d -P training:httpd1
$ docker run --name www -d -p 80:80 training:httpd1
$ docker run --name www -d -p 127.0.0.1:88:80 training:httpd1
$ docker run --name www -d -p 80:80 --link db2 training:httpd1
$ docker run --name www -d -p 80:80 --link db2:db training:httpd1
$ docker inspect www
$ docker network inspect host


  • this is a default network for all containers
  • containers are able to communicate with each other using IP addresses
  • Docker does not support automatic service discovery on the default bridge network
  • to communicate by using names in this network, you must connect the containers via the legacy docker run --link option


Custom user networks (bridge)

$ docker network create --driver bridge --subnet 10.1.2.0/24 net1
$ docker network create --subnet 10.1.2.0/24 --gateway=10.1.2.1 net1
$ docker run -d --network net1 --name mongo --network-alias db training:mongod
$ docker run -d --network net1 --name apache --network-alias www --hostname httpd -p 80:80 -e MONGO_CS="mongodb://mongo:27017" training:httpd
$ docker network ls
$ docker network inspect net1
$ docker network disconnect net1 www
$ docker network connect net1 www