Kbaran: /* Custom user networks (bridge) */

2020-04-06T13:27:31Z

Custom user networks (bridge)

New page

[[Category:Docker]]
[[Category:course_code_dockcm]]

{{Copyright Notice}}

== Author ==
* Kamil Baran - http://www.kamilbaran.pl

== What is Docker? ==
* World before Docker
** no virtualization
** hypervisor-based virtualization
** container-based virtualization

=== Architecture ===
* Docker uses a client-server architecture
[[File:architecture.svg|x400px]]
 

=== Main Docker elements ===
* '''deamon''': process that runs on a host machine (server)
* '''client''': primary Docker interface
* '''image''': read-only template (build component)
* '''registry''': public or private image repositories (distribution, ship component)
* '''container''': created form image, holds everything that is needed for an application to run (run component)
 

=== Benefits of Docker ===
* separation of roles and concerns
** developers focuses on building applications
** system administrators focuses on deployment
* portability: build in one environment, distributed and run on many others
* faster development, testing, deployment
* scalability: easy to spin up new containers or migrate to more powerful hosts
* better resource utilization: more apps on one host
 

=== The underlying technology ===
* '''namespaces'''
** '''pid''' namespace: used for process isolation (Process ID)
** '''net''' namespace: used for managing network interfaces
** '''mnt''' namespace: used for managing mount-points
** '''ipc''' namespace: used for managing access to IPC resources (InterProcess Communication)
** '''uts''' namespace: used for isolating kernel and version identifiers (Unix Timesharing System)
* '''control groups''' (cgroups)
** used for sharing available hardware resources
** and setting up limits and constraints
* '''union file system''' (UnionFS)
** file system that operate by creating layers
** many layers are merged and visible as one consistent file system
** many available file systems: '''AUFS''', btrfs, vfs, DeviceMapper
* '''container format'''
** two supported container formats: '''libcontainer''', LXC
 

== Getting started ==

=== Installation of Docker engine and client ===
<syntaxhighlight lang="Bash">
# easiest way to install Docker
$ wget -qO- https://get.docker.com/ | sh
# to use docker as ubuntu user without using sudo (optional)
$ sudo usermod -aG docker student
# to check the installation
$ docker version
</syntaxhighlight>
* [https://docs.docker.com/engine/installation/ubuntulinux/ Manual installation steps, more details and installation instructions for other operating systems]
 

=== Hello world example ===
<syntaxhighlight lang="Bash">
$ docker run hello-world
$ docker images
$ docker ps -a
</syntaxhighlight>
 

=== Dockerized bash terminal ===
<syntaxhighlight lang="Bash">
$ docker run -it ubuntu
$ docker run -it ubuntu:latest
$ docker run -it ubuntu:14.04 bash
$ docker run -it ubuntu ps -aux
</syntaxhighlight>

* docker run '''-t''': allocate a pseudo-tty
* docker run '''-i''' (--interactive): keep STDIN open even if not attached
* use '''CTRL + p + q''' to detach from running container
* use '''attach''' command to reattach to a detached container
<syntaxhighlight lang="Bash">$ docker attach container_name</syntaxhighlight>
* the importance of PID 1
* PID in the container and Docker host:
<syntaxhighlight lang="Bash">$ ps -fe | grep $(pidof docker)</syntaxhighlight>
* installing packages: mc, vim
 

=== Investigating containers and images ===
* docker '''inspect''' displays low-level information on a container or image
<syntaxhighlight lang="Bash">
$ docker inspect webapp
$ docker inspect --format='{{.NetworkSettings.IPAddress}}' webapp
$ docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' webapp
</syntaxhighlight>
* docker '''diff''' displays changes on a container's filesystem
<syntaxhighlight lang="Bash">
$ docker diff webapp
$ docker diff webapp | grep ^A
</syntaxhighlight>
* docker '''history''' shows the history of an image (layers)
<syntaxhighlight lang="Bash">
$ docker history ubuntu
$ docker history --no-trunc ubuntu
</syntaxhighlight>
* docker '''logs''' fetches the logs of a container
<syntaxhighlight lang="Bash">
$ docker logs webapp
$ docker logs --tail 15 webapp
$ docker logs -f webapp
$ docker logs -t webapp
$ docker logs --since 1h webapp
</syntaxhighlight>
* docker '''top''' displays all running processes of a container
<syntaxhighlight lang="Bash">
$ docker top webapp
$ docker exec webapp ps aux
</syntaxhighlight>
 

=== Cleaning up and keeping clean ===
<syntaxhighlight lang="Bash">
$ docker image ls
$ docker ps -a
</syntaxhighlight>
* docker run '''--rm''' automatically removes the container when it exits
* docker run '''--name''' assigns your own, meaningful name of the container
<syntaxhighlight lang="Bash">$ docker run --rm -it --name=test ubuntu bash</syntaxhighlight>
* container name must be unique and you can change it whenever you like
<syntaxhighlight lang="Bash">$ docker rename test meaningful_name</syntaxhighlight>
* docker '''rm''' removes a container
* docker '''image rm''' removes an image
<syntaxhighlight lang="bash">$ docker rm meaningful_name</syntaxhighlight>
<syntaxhighlight lang="bash">
# removes all not running containers
$ docker rm $(docker ps -aqf "status=exited")
$ docker rm $(docker ps -a | grep 'Exited' | awk '{print $1}')

# removes all images without a tag (dangling)
$ docker image prune
$ docker image rm $(docker image ls -qf "dangling=true")
$ docker rmi $(docker images | grep '^<none>' | awk '{print $3}')
</syntaxhighlight>
''Exercises''
 

== Storage and data persistence ==

=== Within the container ===
* data is only visible inside the container
* data is not persisted outside of the container
* data is lost if the container is removed
 

=== Directly on Docker host ===
<syntaxhighlight lang="Bash">
$ docker run -v /host/dir:/container/dir:rw ...
$ docker run -v /home/ubuntu/docker/training_httpd1/html:/var/www/html/ -d --name www --net host training:httpd1
$ docker run -v $PWD/html:/var/www/html/ -d --name www --net host training:httpd1
$ docker run -v $PWD/html:/var/www/html/:ro -d --name www --net host training:httpd1
</syntaxhighlight>
 

* data is visible inside the container, Docker host and can be shared between containers
* data is persisted outside of the container even if the container is removed
* this provides near bare metal performance
* host directory can be an existing NFS share, formatted block device or anything that can be mounted on Docker host
 

=== Docker Volumes ===
<syntaxhighlight lang="Bash">
$ docker run -itd -v /data --name data1 ubuntu
$ docker inspect data1
$ docker exec data1 touch /data/file1
$ docker exec data1 ls -l /data/
$ docker run -itd --volumes-from data1 --name data2 ubuntu
$ docker inspect data2
$ docker rm -fv data1 data2
</syntaxhighlight>
 

<syntaxhighlight lang="Bash">
$ docker volume create --name kb_volume
$ docker run --rm -it -v kb_volume:/data ubuntu touch /data/kb
$ docker run --rm -it -v kb_volume:/data ubuntu ls -l /data
$ docker volume rm kb_volume
$ docker volume ls
</syntaxhighlight>
 

* data is visible inside the container and can be shared between containers
* data is persisted outside of the container even if the container is removed
** use docker run '''rm -v''' to remove a container with its volumes (unless the other container uses them)
* this provides near bare metal performance
* it solves the problem with privileges (users and groups with different IDs on host and in the container)
 

''' Creating groups and users with custom ID '''
<syntaxhighlight lang="Bash">
$ groupadd -r -g 27017 mongodb
$ useradd -r -u 27017 -g mongodb mongodb
</syntaxhighlight>
 

==== Backup and restore data from volumes ====
<syntaxhighlight lang="Bash">
# backup data from one container
$ docker run -itd -v /data --name data1 ubuntu
$ docker exec data1 touch /data/file1
$ docker exec data1 chown www-data:www-data /data/file1
$ docker run --rm --volumes-from data1 ubuntu ls -l /data
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu tar -cvpf /backup/backup.tar /data
$ docker rm -fv data1
# restore data into brand new container
$ docker run -itd -v /data --name data1 ubuntu
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu tar -xvpf /backup/backup.tar
$ docker run --rm --volumes-from data1 -v $PWD:/backup ubuntu bash -c "cd /data && tar -xvf /backup/backup.tar --strip 1"
$ docker run --rm --volumes-from data1 ubuntu ls -l /data
</syntaxhighlight>
 

=== Outside Docker Host ===
 

== Networking ==
<syntaxhighlight lang="Bash">
$ docker network --help
$ docker network ls
</syntaxhighlight>
 

=== Docker host network ===
* the '''host network''' adds a container on the hosts network stack
* the network configuration inside the container is identical to the host
<syntaxhighlight lang="Bash">
$ docker run --name db1 -d --net host training:mongod
$ docker run --name www -d --net host training:httpd1
$ docker inspect www
$ docker network inspect host
</syntaxhighlight>
 

=== Without network interface ===
* the '''none network''' adds a container to a container-specific network stack
* use ''docker exec'' command to connect to the container
<syntaxhighlight lang="Bash">
$ docker run --name networkless --net none -it --rm ubuntu bash
</syntaxhighlight>
 

=== Default network (bridge) ===
<syntaxhighlight lang="Bash">
$ docker run --name db2 -d --volumes-from db1 training:mongod
$ docker run --name www -d training:httpd1
$ docker run --name www -d -P training:httpd1
$ docker run --name www -d -p 80:80 training:httpd1
$ docker run --name www -d -p 127.0.0.1:88:80 training:httpd1
$ docker run --name www -d -p 80:80 --link db2 training:httpd1
$ docker run --name www -d -p 80:80 --link db2:db training:httpd1
$ docker inspect www
$ docker network inspect host
</syntaxhighlight>
 
* this is a default network for all containers
* containers are able to communicate with each other using IP addresses
* Docker does not support automatic service discovery on the default bridge network
* to communicate by using names in this network, you must connect the containers via the legacy docker '''run --link''' option
 

=== Custom user networks (bridge) ===
<syntaxhighlight lang="Bash">
$ docker network create --driver bridge --subnet 10.1.2.0/24 net1
$ docker network create --subnet 10.1.2.0/24 --gateway=10.1.2.1 net1
$ docker run -d --network net1 --name mongo --network-alias db training:mongod
$ docker run -d --network net1 --name apache --network-alias www --hostname httpd -p 80:80 -e MONGO_CS="mongodb://mongo:27017" training:httpd
$ docker network ls
$ docker network inspect net1
$ docker network disconnect net1 www
$ docker network connect net1 www
</syntaxhighlight>

Container management with Docker - Revision history

Kbaran: /* Custom user networks (bridge) */