sefidian.com Open in urlscan Pro
157.90.105.112 Public Scan

Back to summary

URL:
http://sefidian.com/2021/10/18/guide-to-different-types-of-docker-volumes/
Submission: On December 13 via api (December 13th 2022, 3:27:02 pm UTC) from AR — Scanned from DE

Form analysis
1 forms found in the DOM

GET http://sefidian.com/

<form method="get" id="searchform" action="http://sefidian.com/">
  <i class="icon_search icon-search-fine"></i>
  <a href="#" class="icon_close"><i class="icon-cancel-fine"></i></a>
  <input type="text" class="field" name="s" placeholder="Enter your search">
  <input type="submit" class="display-none" value="">
</form>

Text Content

* Home
* Blog

*
*

Show all
88

GUIDE TO DIFFERENT TYPES OF DOCKER VOLUMES

Categories
* Docker
* Linux
* Software Engineering

Tags

0
0
0

6 mins read

The data generated and used by containers are not persisted after we restart or
remove containers. So, we can use Docker volumes and bind mounts to manage data
in Docker containers to solve this issue. We can use it to persist data in a
container or share data between containers. From this post, you will learn how
to use Docker volumes and bind mounts in your project.

WHAT IS A VOLUME?

THE DOCKER FILE SYSTEM

A docker container runs the software stack defined in an image. Images are made
of a set of read-only layers that work on a file system called the Union File
System. When we start a new container, Docker adds a read-write layer on the top
of the image layers allowing the container to run as though on a standard
Linux file system. So, any file change inside the container creates a working
copy in the read-write layer. However, when the container is stopped or deleted,
that read-write layer is lost.

We can try this out by running a command that writes and then reads a file:

1
2
$ docker run bash:latest \
bash -c "echo hello > file.txt && cat file.txt"

The result is:

1
hello

But if we run the same image with just the command to output the file’s
contents:

1
2
$ docker run bash:latest bash -c "cat file.txt"
cat: can't open 'file.txt': No such file or directory

The second run of the container runs on a clean file system, so the file is not
found.

SETUP

Docker uses the following types of volumes and bind mounts to persist data. For
this setup, I’m using macOS.

1. Anonymous volumes
2. Named volumes
1. Bind mounts

For this post, we will run a MySQL server and execute some commands. By default,
MySQL will store its data files inside /var/lib/mysql directory in the
container, and Docker volumes will help us to persist that data.

We have three docker-compose.yml files to demonstrate volumes and bind mounts.
To start these files, you will need to use the following command.

1
docker compose up

Once our container is running, we can use the following commands to create a
table inside our container for testing purposes.

1
2
3
4
5
6
7
8
9
10
11
# Access the container
docker exec -it mysql_db_1 bash# Connect to MySQL server
mysql -uroot -proot# Run MySQL commands
USE test_db;
SHOW TABLES;
CREATE TABLE users (
user_id int NOT NULL AUTO_INCREMENT,
name VARCHAR(20),
PRIMARY KEY (user_id)
);
SHOW TABLES;

1. ANONYMOUS VOLUMES

If we run the following docker-compose.yml file, an anonymous volume will be
created. If we restart our container, the data will be visible, but not after we
remove the container. Also, it’s not accessible by other containers. It is
helpful if we want to persist data temporarily. These volumes are created
inside /var/lib/docker/volume localhost directory.

1
2
3
4
5
6
7
8
9
10
11
12
version: '3.8'
services:
db:
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: test_db
ports:
- "3306:3306"
volumes:
- /var/lib/mysql

As we can see, we don’t have to specify the host directory. We just need to
specify the directory inside the container.

If we remove the volume instruction from the docker-compose.yml file, the
container will create an anonymous volume by default because it’s specified
inside the MySQL Dockerfile. So, the MySQL image ensures that we can still
access the data if we don’t provide any volume information.

1
VOLUME /var/lib/mysql

Now, we have an anonymous volume with a random identifier.

1
2
3
docker volume ls
DRIVER VOLUME NAME
local 4e679725b7179e63e8658bc157a1980f320948ab819f271fd5a44fe94c16bf23

Let’s inspect our Docker container.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
docker inspect mysql_db_1
.
.
."Mounts": [
{
"Type": "volume",
"Name":
"4e679725b7179e63e8658bc157a1980f320948ab819f271fd5a44fe94c16bf23",
"Source":
"/var/lib/docker/volumes/4e679725b7179e63e8658bc157a1980f320948ab819f271fd5a44fe94c16bf23/_data",
"Destination": "/var/lib/mysql",
"Driver": "local",
"Mode": "",
"RW": true,
"Propagation": ""
}
],
.
.
.

We can use the following command to remove the container and its associated
anonymous volume.

1
docker rm -v mysql_db_1

If we don’t remove the anonymous volume and the container together, it becomes a
dangling volume.

1
docker rm mysql_db_1

We can list and remove all the dangling volumes using the following commands.

1
2
docker volume ls -qf dangling=true
docker volume rm $(docker volume ls -qf dangling=true)

2. NAMED VOLUMES

Named volumes can persist data after we restart or remove a container. Also,
it’s accessible by other containers. These volumes are created
inside /var/lib/docker/volume localhost directory.

1
2
3
4
5
6
7
8
9
10
11
12
13
version: '3.8'
services:
db:
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: test_db
ports:
- "3306:3306"
volumes:
- db_data:/var/lib/mysqlvolumes:
db_data:

Here, the first field is a unique name of the volume on a host machine. The
second part is the path in the container. Moreover, if we remove the container
using the following command, we will still have the volume, unlike anonymous
volumes.

1
docker rm -v mysql_db_1

3. BIND MOUNTS

Bind mounts can persist data after we restart or remove a container. As we can
see, named volumes and bind mounts are the same, except the named volumes can be
found under a specific host directory, and bind mounts can be any host
directory.

1
2
3
4
5
6
7
8
9
10
11
12
version: '3.8'
services:
db:
image: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: test_db
ports:
- "3306:3306"
volumes:
- $PWD/data:/var/lib/mysql

Here, we are mounting a host folder. The first part is the path in the host
machine. The second part is the path in the container.

COMMANDS

Now, let’s list all the available commands for the volume instruction.

1
2
3
4
5
6
7
8
docker volume --help
Commands:
create Create a volume
inspect Display detailed information on one or more volumes
ls List volumes
prune Remove all unused local volumes
rm Remove one or more volumes

We can use these commands to manage anonymous volumes and named volumes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# Create a volume
docker volume create test-vol
# test-vol# Inspect a volume
docker inspect test-vol
# [
# {
# "CreatedAt": "2021-07-17T07:23:25Z",
# "Driver": "local",
# "Labels": {},
# "Mountpoint": "/var/lib/docker/volumes/test-vol/_data",
# "Name": "test-vol",
# "Options": {},
# "Scope": "local"
# }
# ]# List all volumes
docker volume create test-vol-2
docker volume ls
# DRIVER VOLUME NAME
# local test-vol
# local test-vol-2# Remove all volumes
docker volume prune
# WARNING! This will remove all local volumes not used by at least one
container.
# Are you sure you want to continue? [y/N] y
# Deleted Volumes:
# test-vol
# test-vol-2# Remove volumes
docker volume create test-vol-3
docker volume rm test-vol-3
# test-vol-3docker volume create test-vol-4
docker volume create test-vol-5
docker volume rm test-vol-4 test-vol-5
# test-vol-4
# test-vol-5

I hope you have a clear understanding of Docker volumes and bind mounts. It will
help you to persist data for your Docker projects.

References:

https://www.baeldung.com/ops/docker-volumes

https://towardsdatascience.com/the-complete-guide-to-docker-volumes-1a06051d2cce

AMIR MASOUD SEFIDIAN

Machine Learning Engineer

2022-10-15

SETTING UP APACHE AIRFLOW USING DOCKER-COMPOSE

--------------------------------------------------------------------------------

sefidian.com Open in urlscan Pro 157.90.105.112 Public Scan

Form analysis 1 forms found in the DOM

GET http://sefidian.com/

Text Content

sefidian.com Open in urlscan Pro
157.90.105.112 Public Scan

Form analysis
1 forms found in the DOM