[Docker memorandum] DockerFile, docker-compose.yml, Docker command

What is Docker: Container-based virtualization technology

• Container-based virtualization technology: Docker achieves virtualization (starting a virtual computer inside a computer) using containers, which are lighter and more efficient than traditional virtual machines.
• The concept of Docker is to make the process of developing, testing, and deploying applications more efficient and consistent. This can greatly reduce problems caused by differences in environments (such as “It works on my machine but not on another machine”).

Docker image

• Application blueprint : A Docker image is a static file that contains an application and all the files, libraries, dependencies, and environment settings needed to run it.
• Layered structure : Images are made up of multiple layers, which makes them easy to reuse, share, and update

Docker container

• Runnable instance : A Docker container is a runtime environment generated from a Docker image. Containers are launched from images and provide an isolated environment for running applications.
• Light and flexible : Containers are lightweight and can be easily started, stopped, moved, and deleted as needed.

Tag: Image version

• If no tag name is specified, the “latest” tag will be used automatically.
• The image name and tag name are separated by : and are written as follows.

nginx:latest
nginx:1.14-perl

For more information on Docker concepts, see below.

Dockerfile

Create when you want to dockerize and start an application.

A text file (recipe) for creating a Docker image.

Example of python flask application

Externalizing a Flask application: host=’0.0.0.0′

Dockerization means verifying an application by making it accessible from the outside to a unique environment called a docker container, so the application must be made available to the outside world. host='0.0.0.0' Specify the executable file .

host='0.0.0.0'If not specified, the application will not accept external connections and 127.0.0.1will only be accessible from (localhost)

# app.py
from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello, Docker!'

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')

Creating a Dockerfile

A Dockerfile is a blueprint for a Docker image. Docker cannot be started without a Dockerfile.

# 基本イメージとしてPythonを使用
FROM python:3.8-slim

# 作業ディレクトリを設定
WORKDIR /app

# 必要なファイルをコンテナにコピー
COPY . /app

# 必要なパッケージをインストール
RUN pip install -r requirements.txt

# アプリケーションの実行コマンド
# コンテナが起動する時に、app.pyを実行することを意味する。
CMD ["python", "app.py"]

WORKDIR

• WORKDIRThe directive sets the working directory (current directory) within the container. All subsequent instructions are executed with respect to this directory.

COPY

• COPYThe command copies files and directories from the host machine into the container.
• <source>is the path of the file or directory on the host machine, and <destination>is the path within the container. is based on the directory specified <destination>inWORKDIR

COPY <source> <destination>

For example, if you have the following flask application and the executable file (routes.py in this case) is under app/, copy all the files and subdirectories in the app/ directory into the container.

COPY app/ /app/

Regarding COPY relative path and absolute path description

When specifying a path in a Dockerfile, relative paths and absolute paths are distinguished by whether or not a “/” is added at the beginning.

If you don’t add “/” before the copy destination directory , it means specifying a relative path, and if you add “/”, it means specifying an absolute path.

app/: WORKDIRA relative path to the directory set to . WORKDIRIf not set, it will be the root directory of the container.

/app/: Absolute path from the container root directory.

It’s getting complicated, but if the executable file of the flask application in the local environment is under app/ and you want to reproduce it in app/ of the docker container,

1) If WORKDIR is not set

The container’s root directory becomes the default working directory.

WORKDIRIf is not set, means essentially the same operation COPY app/ /app/. COPY app/ app/Both instructions copy the host’s app/directory to the container’s directory./app/

Docker automatically creates the necessary directories, so app/you won’t get an error if the directory doesn’t exist. When specifying COPY with a relative path, if workdir is not set, there is no app/ directory in the first place, so I feel like an error will occur saying that the copy destination directory has not been created, but that can be fixed by Yoshina. Willing to.

COPY app/ /app/ #コンテナのルートディレクトリからの絶対パスで指定
COPY app/ app/　#相対パス。WORKDIRを設定していないのでルートディレクトリからのパスになる

2) When setting WORKDIR

If workdir is written as /app

If you specify an absolute path, it is the same as not setting workdir, and of course there is no change.

On the other hand, if you specify a relative path, the current directory is /app, so set it to “.”.

WORKDIR /app
COPY app/ /app/ #コンテナのルートディレクトリからの絶対パスで指定
COPY app/ .　#相対パス。/appに作成したいので、カレントディレクトリ「.」

By the way , if you set it to , WORKDIRyou need to take this value into account when specifying the copy destination path when copying a file with a command . First of all, I don’t think so./testCOPYWORKDIR

WORKDIR /test
COPY app/ /test/

becomes. If you run this dockerfile, the following file structure on the host will be:

app/
  app.py
  static/
    style.css

Inside the Docker container

/test/
  app.py
  static/
    style.css

becomes.

requirements.txt

Continuing from the previous copy, in the case of the folder structure of the flask application as shown below, if you copy the app/ directory to the docker container’s app/ directory, requirements.txt is not under app/, so naturally it will not be copied.

Therefore, you need to copy it separately.

If you want to use the same folder structure as the local folder structure, enter the absolute path root directory “/” below.

COPY requirements.txt /

# 使用するPythonのベースイメージ
FROM python:3.10-slim

# 作業ディレクトリを設定
WORKDIR /app

# 依存関係をインストール
COPY requirements.txt /
RUN pip install --no-cache-dir -r requirements.txt

# アプリケーションのコードをコピー
COPY app/ /app/

# コンテナ内で実行するコマンド
CMD ["python3", "routes.py"]

• -rFlag: stands for “requirements file” and piptells to read a list of package names from the given file and install them.
• --no-cache-dirOptional: Install packages listed in requirements.txt without caching. dir is an abbreviation for directory.

pipWhen installing a Python library using , the downloaded library is usually saved as a cache. When creating a Docker image, the cache is also stored within the image, increasing the size of the image.

--no-cache-dirIf you disable caching using the option, the library will be installed but not cached, resulting in a smaller image size.

However, disabling the cache requires pip to download the necessary packages from the internet each time, which increases Docker container build time.

requirements.txtCreate a file that describes Flask and other necessary libraries .

flask==1.1.2
requests>=2.24.0
numpy

-u option: disable buffering

buffering

• Buffering is the process of temporarily storing data
• In Python, standard output (output from print statements, etc.) is “buffered” by default. That is, the output is held until a certain amount of data accumulates before it is actually displayed on the screen, or until the output buffer is flushed (emptied).

To output python logs in real time, you need to disable this buffering, and to do that you -uneed to run the python script with options.

CMD ["python", "-u", "script.py"]

-uOptional -uis not an abbreviation, just an optional flag. In the official Python documentation, -uis used simply to mean “unbuffered binary stdout and stderr”.

Building a Docker image

Build a Docker image using Dockerfile.

If you want to create an image named .using the Dockerfile located in the current directory (flask-app

• The -t option stands for “tag” and is used to give a name (tag) to the image being built.
• flask-app is the name given to the image. This name is optional and will be used to help identify the image later. For example, it is common to use a meaningful name, such as the application name or version.
• The dot . is the directory where the Dockerfile is located. In this case, the current working directory (the directory where you are running the command). If the Dockerfile is in a different directory, you must specify the path to that directory.

docker build -t flask-app .

Running a Docker container

Start a container from the built image.

• Map port 8080 to port 5000 on your local machine and start the container from the flask-app image. This will allow you to access your Flask application at http://localhost:8080.

docker run -p 8080:5000 flask-app

When specifying environment variables

Now you can build and run your Docker container and access your site.

However, with this, you have to specify port mapping and environment variables with the docker run option command each time, which is a hassle. Therefore, the concept of docker-compose.yml appears.

Docker-compose.yml does not necessarily mean that Docker cannot be started without it, but it is used to make docker operations more efficient.

docker-compose.yml

version: '3'
services:
  web:
    build: .
		image: flask-app
    ports:
      - "8080:5000"
    environment:
      - FLASK_ENV=development
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    volumes:
      - ./app:/app

• version: '3': This docker-compose.ymlspecifies the version of the file. Version 3 has a feature set that is supported by specific versions of Docker Compose.
• services: This section defines the service (container) to be managed. A service is a running container.
• web: Name of the defined service. In this example, the service is named “web,” which is the container that runs the Flask application.
• build: .: This .tells Docker Compose to use the current directory ( ) to build the container image. docker build -t flask-app . Refers to the part added at the end of the command.
• ports: port mapping. means mapping the host machine’s 8080 port to the container’s 5000 port
• environment: Set environment variables in the container. In this example, we set it FLASK_ENVto development, which means Flask will run in development mode. It is also OPENAI_API_KEYread from the environment variables and set as environment variables within the container.
• volumes: Configure file system mounting between host machine and container. In this example, a directory in the host machine’s current directory is mounted insideapp the container . When you edit a Python function (or other file) in a directory /appon the host machine , those changes are immediately reflected in the file in the container. appIf you change a file while the container is running, those changes will immediately affect applications running inside the container.

If you want to communicate between multiple Docker containers, set external:true.

How Docker automatically generates image names

imageRegarding keys

• docker-compose.ymlIt is possible to execute imagewithout writing the key in the file . docker-compose upIn this case, Docker Compose automatically generates an image name and assigns it to the built image.
• imageIf no key is specified, Docker Compose generates an image name using default naming conventions.

<ディレクトリ名>-<サービス名>

<ディレクトリ名>is docker-compose.ymlthe name of the directory where the file resides

<サービス名>is the name of a service defined in a section docker-compose.ymlwithin the file . servicesAdditionally, Docker Compose automatically tags images latest.

For example, if docker-compose.ymla file is located in a directory, Docker Compose automatically generates and tags an image named . This identifies the built image as . If you need to specify a specific image name, use a key to explicitly set the name.myprojectmyproject-weblatestmyproject-web:latestimage

Docker command

Once you create the docker-compose.yml file, you will be able to operate Docker (build, run, stop, delete) using the docker-compose command.

Build

docker-compose build

docker-compose.ymlBuild an image for the service defined in . This will create a Docker image for each service based on the specified Dockerfile .

Run (Up/Run)

docker-compose.ymlTo start all services (containers) defined in a file

docker-compose up

When starting specified services (containers) individually and executing arbitrary commands

SERVICE: Name of the service to be executed

docker-compose run [options] SERVICE [COMMAND] [ARGS...]

Force build at runtime: up —build

docker-compose upThe command normally performs a new build if there is no built image. If there is an already built image, start the container without rebuilding it. If you want to force a build, --builduse options.

docker-compose up --build

logs

Real-time display of logs: -f (abbreviation for follow)

If you want to view logs for a specific service, specify the service name. example: docker-compose logs -f web.

Stop (Down)

docker-compose down

docker-compose upStop the services started in , and also delete resources such as containers, networks, and volumes. Images are not deleted. If you want to remove an image, add an option (abbreviation for remove image) docker-compose downto the command .--rmi

Image deletion: —rmi (remove image)

docker-compose down --rmi all #全てのイメージ削除
docker rmi [イメージ名またはイメージID] #特定のイメージ削除

Individual service stop (Stop)

When you want to stop all services, use down, but when you want to stop individual services, use stop.

The stop command stops the container for the specified service from running, but does not remove the container.

docker-compose stop service_name  # service_name：停止したいサービス名

Command execution within the container (Exec: execute)

docker-compose exec -it service_name bash

Exec stands for execute

-itoption

• Used to enable interaction and terminal (TTY) attachment
• -i: --interactiveAbbreviation. -t:-Abbreviation for tty. Virtual terminal (TTY)
• To start an interactive shell for a container that is already running, use options docker execin the command:-it

docker execuses the container name or ID, and docker-compose execuses the service name

docker exec -it my_container bash #コンテナ名 or コンテナID
docker-compose exec my_service bash #サービス名

• Specifies the command to be executed within the container
• bashNot all containers have it installed. (shell) or other command line interfaces may be used bashinstead , especially for lightweight images.sh

Container list display: ps (abbreviation for process status)

docker ps

Display a list of currently running containers. This allows you to easily see which Docker containers are running on your system.

Output a table containing information like the following:

• CONTAINER ID : Unique ID of the container.
• IMAGE : The image used to create the container.
• COMMAND : Command executed when the container starts.
• CREATED : Time elapsed since the container was created.
• STATUS : The state of the container (running, stopped, paused, etc.).
• PORTS : Ports opened by the container and corresponding ports on the host machine.
• NAMES : Name of the container.