This Blog is to share our knowledge and expertise on Linux System Administration and VMware Administration

Showing posts with label Docker. Show all posts
Showing posts with label Docker. Show all posts

Thursday, February 15, 2018

Working with Docker Container - Labeling and filtering containers

Thursday, February 15, 2018 0
Labeling and filtering containers

With Docker 1.6, a feature has been added to label containers and images, through which we can attach arbitrary key-value metadata to them. You can think of them as environment variables, which are not available to running applications inside containers but they are available to programs (Docker CLI) that are managing images and containers. Labels attached to images also get applied to containers started via them. We can also attach labels to containers while starting them.

Docker also provides filters to containers, images, and events, which we can use in conjunction with labels to narrow down our searches.

For this document, let's assume that we have an image with the label, distro=centos.

root@Docker:~# docker images
REPOSITORY                 TAG                 IMAGE ID                CREATED            SIZE
nskselvan/nsk                   latest              b2f0c17eed23        2 months ago        197MB
centos-latest                     latest              2083898799b1       2 months ago        197MB
wordpress                         latest              224b7eef6944        3 months ago        408MB
mysql                                 5.7                 b4e78b89bcf3        3 months ago        412MB
registry                               2                    28525f9a6e46       3 months ago        33.2MB
localhost:5000/reg            latest               28525f9a6e46       3 months ago        33.2MB
centos                               latest              196e0ce0c9fb        3 months ago        197MB
localhost:5000/centos-ka  latest              196e0ce0c9fb        3 months ago        197MB

As you can see from the preceding screenshot, if we use filters with the docker images command, we only get an image where the corresponding label is found in the image's metadata.

To start the container with the --label/-l option, run the following command:
root@Docker:~# docker run --label environment=test centos date
Thu Feb 15 06:21:09 UTC 2018

Let's start a container without a label and start two others with the same label:
root@Docker:~# docker run --name container1 centos date
Thu Feb 15 06:22:13 UTC 2018
root@Docker:~# docker run --name tcontainer1 --label environment=test centos date
Thu Feb 15 06:23:01 UTC 2018
root@Docker:~# docker run --name tcontainer2 --label environment=test centos date
Thu Feb 15 06:23:20 UTC 2018

If we list all the containers without a label, we will see all the containers, but if we use label, then we get only containers, which matches the label.




Docker attaches label metadata to containers while starting them and matches the label while listing them or other related operations
We can list all the labels attached to a container through the inspect command

root@Docker:~# docker inspect -f '{{.Config.Labels}}' tcontainer1
map[build-date:20170911 environment:test license:GPLv2 name:CentOS Base Image vendor:CentOS]
root@Docker:~# docker inspect -f '{{.Config.Labels}}' tcontainer2
map[license:GPLv2 name:CentOS Base Image vendor:CentOS build-date:20170911 environment:test]

Wednesday, February 14, 2018

Working with Docker Container - Docker info Command

Wednesday, February 14, 2018 0
Docker info Command

As we know, images are required to run containers. You can either use existing images or create new custom images. You will need to create custom images to suit your development and deployment environment. Once you create an image, you can share it through the public or private registry. Before we explore more about Docker images, let's look at the output of the docker info command:

root@Docker:~# docker info
Containers: 26
 Running: 5
 Paused: 0
 Stopped: 21
Images: 6
Server Version: 17.09.0-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 87
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 06b9cb35161009dcb7123345749fef02f7cea8e0
runc version: 3f2f8b84a77f73d38244dd690525642a72156c64
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-96-generic
Operating System: Ubuntu 16.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 3.859GiB
Name: Docker
ID: S3LP:G73H:NJWO:SVEG:I7U5:UB5M:B3LC:WAQV:AINP:7YYE:I7EN:MRFF
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: nskselvan
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support
root@Docker:~#

The preceding command gives the current system-wide info as follows:

It has 26 containers and 6 images.


The current storage driver, devicemapper, and its related information, such as thin pool name, data, metadata file, and so on. Other types of storage drivers are aufs, btrfs, overlayfs, vfs, and so on. Devicemapper, btrfs, and overlayfs have native support in the Linux kernel. AUFS support needs a patched kernel. 


To leverage the kernel features that enable containerization, the Docker daemon has to talk to the Linux kernel. This is done through the execution driver. libconatiner or native is one of that type. The others are libvirt, lxc, and so on.


The kernel version on the host operating system.
The user account that is registered on the registry mentioned in the next section to pull/push images.

Sunday, February 4, 2018

Working with Docker Container - Creating an image from the container

Sunday, February 04, 2018 0

Creating an image from the container

There are a couple of ways to create images, one is by manually committing layers and the other way is through Dockerfiles.

As we start a new container, a read/write layer gets attached to it. This layer will get destroyed if we do not save it. In this recipe, we will see how to save that layer and make a new image from the running or stopped container using the docker commit command.

Syntax: docker commit -a|--author[=""] -m|--message[=""] CONTAINER [REPOSITORY[:TAG]]

root@Docker:~# docker run -i -t centos /bin/bash
[root@91fb5dbccd0b /]# yum install -y httpd
Loaded plugins: fastestmirror, ovl
base                                                                | 3.6 kB  00:00:00
extras                                                              | 3.4 kB  00:00:00
updates                                                           | 3.4 kB  00:00:00
(1/4): base/7/x86_64/group_gz                       | 156 kB  00:00:00
(2/4): extras/7/x86_64/primary_db                  | 166 kB  00:00:00
(3/4): base/7/x86_64/primary_db                    | 5.7 MB  00:00:11
(4/4): updates/7/x86_64/primary_db               | 6.0 MB  00:00:13
...........
...........



As you can see, the new image is now being committed to the local repository with  nskselvan/centos  as a name and httpd as a tag.

we saw that while starting a container, a read/write filesystem layer will be created on top of the existing image layers from which the container started, and with the installation of a package, some files would have been added/modified in that layer. All of those changes are currently in the ephemeral read/write filesystem layer, which is assigned to the container. If we stop and delete the container, then all of the earlier mentioned modifications would be lost.

Using commit, we create a new layer with the changes that have occurred since the container started, which get saved in the backend storage driver.


root@Docker:~# docker diff 91fb5dbccd0b
C /.wh..wh.plnk
A /.wh..wh.plnk/1010.142994
.....................
A /boot/grub/splash.xpm.gz
C /etc/group
...................
C /etc/gshadow-
A /etc/httpd
A /etc/httpd/conf
A /etc/httpd/conf/httpd.conf
A /etc/httpd/conf/magic
A /etc/httpd/conf.d
...................
A /etc/httpd/conf.modules.d

We can see a prefix before each entry of the output. The following is a list of those prefixes:

    A: This is for when a file/directory has been added
    C: This is for when a file/directory has been modified
    D: This is for when a file/directory has been deleted


For more details, please check docker commit --help 

Friday, February 2, 2018

Working with Docker Container - Injecting a new process to a running container

Friday, February 02, 2018 0

 Injecting a new process to a running container

While doing development and debugging, we might want to look inside the already running container. There are a few utilities, such as nsenter (https://github.com/jpetazzo/nsenter), that allow us to enter into the namespace of the container to inspect it. With the exec option, we can inject a new process inside a running container.

Syntax:
    docker exec [-d|--detach[=false]] [--help] [-i|--interactive[=false]] [-t|--tty[=false]] CONTAINER COMMAND [ARG...]

Let's start an centos container and then inject uptime into that


root@Docker:~# id='docker run -d mysql'
root@Docker:~# docker run -it $id bash
root@b4e78b89bcf3#

For help with the docker run use --help

Friday, January 26, 2018

Working with Docker Container - Creating an account with Docker Hub

Friday, January 26, 2018 0

Creating an account with Docker Hub

Docker Hub is like GitHub for images. It is a public registry on which you can host images both public and private, share them and collaborate with others. It has integration with GitHub, Bitbucket, and can trigger automated builds.

As of now, the creation of an account on Docker Hub is free. A repository can hold different versions of an image. You can create any number of public repositories for your images. By default, you will have one private repository, which will not be accessible to the public. You can buy more private repositories. You can create an account either through a web browser or from the command line.

To create an account through a web browser on Docker Hub, visit https://hub.docker.com/account/signup/ and create an account:
 

Give the required details & and get created docker login.

To create an account using the command line, run the following command and submit the required details:
root@Docker:~# docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username (nskselvan):
Password:

Sunday, January 21, 2018

Working with Docker Container - Setting the restart policy on a container

Sunday, January 21, 2018 0

Setting the restart policy on a container

Restart policy is added with the run command with flags to specify the restart policy. With this policy, we can configure containers to start at boot time. This option is also very useful when a container dies accidentally.

Syntax:docker run --restart=POLICY [ OPTIONS ]  IMAGE[:TAG]  [COMMAND]  [ARG...] 

root@Docker:~# docker run --restart=always -d -i -t centos /bin/bash
fb83a04222c73d0e80f84103d4fe58c3ccd95e4a44b39386a937427bda2ad465

Options: 
no            : This does not start the container if it dies
on-failure : This restarts the container if it fails with nonzero exit code
always     : This always restarts the container without worrying about the return code

You can also give an optional restart count with the on-failure policy as follows:

root@Docker:~# docker run --restart=on-failure:3 -d -i -t centos /bin/bash
96b07463e85f3452cbbbf0de1298d4311634efef20f60a1d164e18f83ea19e15

For help with the docker run use --help

Saturday, January 20, 2018

Working with Docker Container - Accessing the host device inside the container

Saturday, January 20, 2018 0

Accessing the host device inside the container

 we can give access of the host device to a container with the --device option to the run command. Earlier, one has bind mount it with the -v option and that had to be done with the --privileged option.

Syntax :
     docker run --device=<Host Device>:<Container Device Mapping>:<Permissions>   [ OPTIONS ]  IMAGE[:TAG]  [COMMAND]  [ARG...]

#docker run --device=/dev/sdc:/dev/xvdc -i -t centos /bin/bash


The preceding command will access /dev/sdc inside the container.

For help with the docker run use --help

Friday, January 19, 2018

Working with Docker Container - Exposing a port while starting a container

Friday, January 19, 2018 0

Exposing a port while starting a container

There are a number of ways by which ports on the container can be exposed. One of them is through the run command, which we will cover in this chapter. The other ways are through the Docker file and the --link command.

Syntax:  docker run --expose=PORT [ OPTIONS ]  IMAGE[:TAG]  [COMMAND]  [ARG...]

to expose port 22 while starting a container, run the following command:

root@Docker:~# docker run --expose=22 -i -t centos /bin/bash
[root@88a0e1ab48df /]#
root@Docker:~# docker ps
CONTAINER ID  IMAGE   COMMAND  CREATED          STATUS            PORTS       NAMES
88a0e1ab48df    centos     "/bin/bash"   27 seconds ago  Up 26 seconds   22/tcp  amazing_bohr

For help with the docker run use --help

Thursday, January 18, 2018

Working with Docker Container - Looking at the logs of containers

Thursday, January 18, 2018 0

Looking at the logs of containers

If the container emits logs or output on STDOUT/STDERR, then we can get them without logging into the container.

Syntax : docker logs  CONTAINER

root@Docker:~# docker logs centos
[root@6d65e303381c /]# uptime
 04:07:13 up  1:02,  0 users,  load average: 0.02, 0.01, 0.00
[root@6d65e303381c /]# hostname
6d65e303381c
[root@6d65e303381c /]# exit
root@Docker:~#

Docker will look at the container's specific log file from /var/lib/docker/containers/<Container ID> and show the result.

For help with the docker logs use --help

Working with Docker Container - Returning low level information about a container

Thursday, January 18, 2018 0

Returning low-level information about a container

While doing the debugging, automation, and so on, we will need the container configuration details. Docker provides the inspect command to get those easily.

To inspect a container/image, run the following command:
Syntax:
docker inspect [-f|--format="" CONTAINER|IMAGE [CONTAINER|IMAGE...]
We'll start a container and then inspect it:

root@Docker:~# docker run -id centos /bin/bash
c5f6ce3b5d2f82bb7a2bbc82b0b71bc2130ceb4caf163afbf5883cfbb150f256

root@Docker:~# docker inspect c5f6ce3b5d2f82bb7a2bbc82b0b71bc2130ceb4caf163afbf5883cfbb150f256

[
    {
        "Id": "c5f6ce3b5d2f82bb7a2bbc82b0b71bc2130ceb4caf163afbf5883cfbb150f256",
        "Created": "2018-01-02T12:18:18.232317934Z",
        "Path": "/bin/bash",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,

    .........
    .........
}]

Docker will look into the metadata and configuration for the given image or container and present it.

With the -f | --format option we can use the Go (programming language) template to get the specific information. The following command will give us an IP address of the container:

root@Docker:~# docker inspect --format='{{.NetworkSettings.IPAddress}}' c5f6ce3b5d2f82bb7a2bbc82b0b71bc2130ceb4caf163afbf5883cfbb150f256
172.17.0.4

The following command will give us an Hostname Path of the container:

root@Docker:~# docker inspect --format='{{.HostnamePath}}' c5f6ce3b5d2f82bb7a2bbc82b0b71bc2130ceb4caf163afbf5883cfbb150f256
/var/lib/docker/containers/c5f6ce3b5d2f82bb7a2bbc82b0b71bc2130ceb4caf163afbf5883cfbb150f256/hostname

For help with the docker inspect use --help

Tuesday, January 16, 2018

Working with Docker Container - Getting privileged access inside a container

Tuesday, January 16, 2018 0

Getting privileged access inside a container

Linux divides the privileges traditionally associated with superuser into distinct units, known as capabilities (run man capabilities on a Linux-based system), which can be independently enabled and disabled. For example, the net_bind_service capability allows nonuser processes to bind the port below 1,024. By default, Docker starts containers with limited capabilities. With privileged access inside the container, we give more capabilities to perform operations normally done by root. For example, let's try to create a loopback device while mounting a disk image.

Syntax : docker run --privileged [ OPTIONS ]  IMAGE[:TAG]  [COMMAND]  [ARG...] 

root@Docker:~# docker run --privileged -i -t centos /bin/bash
[root@89f516205250 /]#
[root@89f516205250 /]# dd if=/dev/zero of=disk.img bs=1M count=10 &> /dev/null
[root@89f516205250 /]# mkfs -t minix disk.img &> /dev/null
[root@89f516205250 /]# mount disk.img /mnt/
[root@89f516205250 /]# mount | grep -i disk
/var/lib/docker/aufs/diff/72bebd0aff7bf4dbbd74495a41884d3113f2dedbfcffa3c82256abced73b0b21/disk.img on /mnt type minix (rw,relatime)
[root@89f516205250 /]# df -hP | grep -i /mnt
/dev/loop0                   9.9M  1.0K  9.9M   1% /mnt
[root@89f516205250 /]# cd /mnt/
[root@89f516205250 mnt]# echo "This is docker test" > test
[root@89f516205250 mnt]# cat test
This is docker test

This mode causes security risks as containers can get root-level access on the Docker host. With Docker 1.2 or new, two new flags --cap-add and --cap-del have been added to give fine-grained control inside a container. For example, to prevent any chown inside the container, use the following command:

 docker run --cap-drop=CHOWN [ OPTIONS ]  IMAGE[:TAG]  [COMMAND]  [ARG...]
root@Docker:~# docker run --cap-drop=CHOWN  -i -t centos /bin/bash
[root@5c536ea0d181 /]# chown root:adm mnt
chown: changing ownership of 'mnt': Operation not permitted

For help with the docker run use --help

Monday, January 15, 2018

Working with Docker Container - Deleting a container

Monday, January 15, 2018 0

Deleting a container

We can delete a container permanently, but before that we have to stop the container or use the force option. In this recipe, we'll start, stop, and delete a container.
Syntax : docker rm [ OPTIONS ] CONTAINER [ CONTAINER ]

Let's first start a container, stop it, and then delete it using the following commands:

root@Docker:~# id=`docker run -d -i centos /bin/bash`
root@Docker:~# docker stop $id
e62286794466459f2dd08d5f1cec0749187247ffabb5224d6b6b3aae334d4bf8
root@Docker:~# docker rm $id
e62286794466459f2dd08d5f1cec0749187247ffabb5224d6b6b3aae334d4bf8

To forcefully delete a container without an intermediate stop, use the -f option.

To delete all the containers, we first need to stop all the running containers and then remove them. Be careful before running the commands as these will delete both the running and the stopped containers:

root@Docker:~# docker stop `docker ps -q`
root@Docker:~# docker rm `docker ps -q`

For help with the docker rm use --help

Sunday, January 14, 2018

Working with Docker Container - Stopping a container

Sunday, January 14, 2018 0

Stopping a container

We can stop one or more containers at once. In this recipe, we will first start a container and then stop it.
Syntax : docker stop [-t|--time[=10]] CONTAINER [CONTAINER...]


root@Docker:~# docker stop fb83a04222c7
fb83a04222c7


This will save the state of the container and stop it. It can be started again, if needed.

To stop a container after waiting for some time, use the --time/-t option.

To stop all the running containers run the following command:

root@Docker:~# docker stop `docker ps -q`
5d950a3835d6
32238eabfac4

For help with the docker stop use --help

Thursday, January 11, 2018

Working with Docker Container - Listing containers

Thursday, January 11, 2018 0

Listing containers

We can list both running and stopped containers.

Syntax: docker ps [ OPTIONS ]
The Docker daemon can look at the metadata associated with the containers and list them down. By default, the command returns:

The container ID
The image from which it got created
The command that was run after starting the container
The details about when it got created
The current status
The ports that are exposed from the container
The name of the container

root@Docker:~# docker ps -a




To return just the container IDs of all the containers, use the -aq option as follows:
root@Docker:~# docker ps -aq
b613fbc39be2
e3bfe67aa175
8813d555d0dc
6d65e303381c
5d950a3835d6
32238eabfac4
d38d94f8b88c
ce065a026516
13b10b8f321d
40f35204ce70
1b6ecbd6a091
d1c871755f38

To show the last created container, including the non-running container, run the following command:
root@Docker:~# docker ps -l
CONTAINER ID   IMAGE   COMMAND    CREATED       STATUS       PORTS  NAMES
b613fbc39be2      centos    "/bin/bash"     8 minutes ago  Up 8 minutes             sad_shannon

For help with the docker ps use --help

Monday, January 8, 2018

Working with Docker Container - Starting a container

Monday, January 08, 2018 0

Starting a container

Listing images

We can list the images available on the system running the Docker daemon. These images might have been pulled from the registry, imported through the docker command, or created through Docker files.
root@Docker:~# docker images
REPOSITORY                     TAG                 IMAGE ID               CREATED            SIZE
nskselvan/nsk                      latest              b2f0c17eed23        2 months ago        197MB
centos-latest                        latest              2083898799b1       2 months ago        197MB
wordpress                            latest              224b7eef6944        3 months ago        408MB
mysql                                   5.7                  b4e78b89bcf3        3 months ago        412MB
registry                                 2                    28525f9a6e46         3 months ago        33.2MB
localhost:5000/reg               latest              28525f9a6e46         3 months ago        33.2MB
centos                                  latest             196e0ce0c9fb          3 months ago        197MB
localhost:5000/centos-ka     latest             196e0ce0c9fb          3 months ago        197MB

For help with the docker images use --help

Once we have images, we can use them to start the containers. In this recipe, we will start a container with the fedora:latest image and see what all things happen behind the scene.
syntax  : docker run [ OPTIONS ]  IMAGE[:TAG]  [COMMAND]  [ARG...]

root@Docker:~# docker run -i -t --name=centos centos /bin/bash
[root@6d65e303381c /]# uptime
 04:07:13 up  1:02,  0 users,  load average: 0.02, 0.01, 0.00

Here,
The -i option starts the container in the interactive mode
The -t option allocates a pseudo-tty and attaches it to the standard input

So, with the preceding command, we start a container from the centos:latest image, attach pseudo-tty, name it centos, and run the /bin/bash command. If the name is not specified, then a random string will be assigned as the name.
Also, if the image is not available locally, then it will get downloaded from the registry first and then run. Docker will run the search and pull commands before running the run command.

Under the hood, Docker:

Will merge all the layers that make that image using UnionFS.
Allocates a unique ID to a container, which is referred to as Container ID.
Allocates a filesystem and mounts a read/write layer for the container. Any changes on this layer will be temporary and will be discarded if they are not committed.
Allocates a network/bridge interface.
Assigns an IP address to the container.
Executes the process specified by the user.
Also, with the default Docker configuration, it creates a directory with the container's ID inside /var/lib/docker/containers, which has the container's specific information such as hostname, configuration details, logs, and /etc/hosts.

To exit from the container, press Ctrl + D or type exit. It is similar to exiting from a shell but this will stop the container.
The run command creates and starts the container. With Docker 1.3 or later, it is possible to just create the container using the create command and run it later using the start command, as shown in the following example:

root@Docker:~# ID=$(docker create -t -i centos bash)
root@Docker:~# docker start -a -i $ID
[root@8813d555d0dc /]#

The container can be started in the background and then we can attach to it whenever needed. We need to use the -d option to start the container in the background:
root@Docker:~# docker run -d -i -t centos /bin/bash
e3bfe67aa1759c5ccd0e8e061595b8214881673c4df599f9652a58369df7d948

The preceding command returns the container ID of the container to which we can attach later, as follows:
root@Docker:~# docker attach e3bfe67aa1759c5ccd0e8e061595b8214881673c4df599f9652a58369df7d948
[root@e3bfe67aa175 /]#

The --read-only option of the run command will mount the root filesystem in the read-only mode:
root@Docker:~# docker run --read-only -d -i -t centos /bin/bash
b613fbc39be2b46cbc485d76bae3f2e7f1781c570e32b6509dab5345d45b6e04

Note: this option just makes sure that we cannot modify anything on the root filesystem, but we are writing on volumes
For help with the docker run use --help

Wednesday, January 3, 2018

Working with Docker Containers

Wednesday, January 03, 2018 0

Working with Docker Containers

We need an image to start the container. Let's see how we can search images on the Docker registry. A registry holds the Docker images and it can be both public and private. By default, the search will happen on 
the default public registry, which is called Docker Hub and is located at https://hub.docker.com/

Docker client and server version
root@Docker:~# docker version
Client:
 Version:        17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:             Tue Sep 26 22:42:18 2017
 OS/Arch:       linux/amd64

Server:
 Version:        17.09.0-ce
 API version:  1.32 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:             Tue Sep 26 22:40:56 2017
 OS/Arch:       linux/amd64
 Experimental: false

Listing/searching for an image
Format : docker search TERM
root@Docker:~# docker search centos

For help with the Docker search, run the following command:
docker search --help

Pulling an image
Format: docker pull NAME[:TAG]

root@Docker:~# docker pull fedora
Using default tag: latest
latest: Pulling from library/fedora
a8ee583972c2: Downloading [=========>                                         ]  16.75MB/86.82MB
..
Image tags group images of the same type. For example, CentOS can have images with tags such as centos5, centos6, and so on. For example, to pull an image with the specific tag, run the following command:
$ docker pull centos:centos7

By default, the image with latest tag gets pulled. To pull all images corresponding to all tags, use the following command:
$ docker pull --all-tags centos

For help with the docker pull use --help