Docker Containers and Containerization: A Practical Guide for Developers


Docker containers have changed how developers build, package, test, and deploy software. This practical guide explains containerization from first principles, shows how Docker images and containers work, and walks through Dockerfiles, Docker Compose, persistent storage, networking, security, and production operations. You will also learn when containers are useful, how to avoid common mistakes, and how to build a repeatable workflow for applications ranging from a small API to a multi-service system.
Software is developed and executed in environments made up of operating system libraries, language runtimes, application dependencies, configuration, and system services. When those environments differ between a developer's computer, a test server, and production, an application can behave differently even when its source code has not changed. This is the familiar problem behind the phrase "it works on my machine."
Containerization addresses much of this inconsistency by packaging an application and its user-space dependencies into a standardized image. A container is created from that image and runs as an isolated process on a host system. Developers can build an image once, store it in a registry, and run the same image in different environments that support the required container runtime and platform.
Docker is a widely used platform for building, distributing, and running containers. Its command-line interface, image format, build system, networking, storage, and Compose tooling make it possible to manage application environments using repeatable configuration rather than relying entirely on manual setup.
A useful distinction is that Docker is not the same thing as containerization itself. Containerization is an operating-system-level approach to isolating processes and their resources. Docker provides tools and workflows for working with containers. Other ecosystems and runtimes, including containerd, CRI-O, Podman, and Kubernetes-based platforms, also work with container images and containers.
Containers are valuable because they make software environments easier to reproduce and move between machines. A backend service can run with a specified Python or Node.js runtime, a database can run in a separate container, and a test environment can use the same dependency versions as the application.
Common benefits include:
Containers do not automatically make an application scalable, secure, or fault tolerant. Those qualities still depend on application design, infrastructure, configuration, operational practices, and testing.
Containers and virtual machines both provide isolation, but they do so at different layers.
A virtual machine uses a hypervisor to virtualize hardware and run a guest operating system. Each VM has its own operating system kernel and system services. A container isolates application processes while sharing the host operating system kernel. This usually makes containers quicker to start and more resource-efficient than running a separate full operating system for every service.
| Characteristic | Containers | Virtual machines |
|---|---|---|
| Isolation layer | Operating-system process and resource isolation | Virtualized hardware and guest operating system |
| Kernel | Shares the host kernel | Runs a guest kernel |
| Startup | Usually fast, often seconds or less | Typically slower because a guest OS must boot |
| Resource overhead | Generally lower | Generally higher |
| Operating-system flexibility | Must be compatible with the host kernel and runtime | Can run a different supported guest OS |
| Common use | Application packaging, microservices, repeatable development | Stronger OS separation, legacy systems, different operating systems |
Neither approach replaces the other in every situation. Virtual machines are useful when a workload needs a separate operating-system kernel, a different operating system, or a particular isolation boundary. Containers are useful when teams want to package and operate applications consistently with less per-workload overhead.
Containers are not simply lightweight virtual machines. Because they share the host kernel, a kernel vulnerability or unsafe runtime configuration can affect the host. Container isolation should be combined with appropriate permissions, security controls, and host maintenance.
Docker uses a client-server architecture. Understanding the components makes it easier to troubleshoot builds, image pulls, container startup, and network or storage problems.
The Docker client is the command-line interface commonly invoked with the docker command. It accepts commands such as docker build, docker pull, docker run, and docker compose up, then communicates with a Docker daemon or compatible endpoint through the Docker API.
The client and daemon do not have to run on the same machine. A developer can configure a Docker context to target a remote Docker Engine, provided the connection is secured and access is appropriately controlled.
The Docker daemon, dockerd, manages Docker objects such as images, containers, networks, and volumes. It listens for API requests from the client and coordinates the work needed to create and manage containers.
Docker Engine includes the daemon, API, and supporting components. The daemon commonly requires elevated host privileges in a traditional installation, which is one reason access to the Docker socket must be treated as sensitive.
A Docker image is an immutable template containing a filesystem and metadata needed to create a container. Images are built from layers and can be tagged, stored in registries, and reused across environments.
A container is an instance created from an image. It has a writable layer for runtime changes, process configuration, networking, and optional mounts for persistent or shared data. Removing a container normally removes its writable layer, but named volumes and bind-mounted host data can persist independently.
A container registry stores and distributes images. Docker Hub is a commonly used public registry, while organisations may use private registries provided by cloud platforms or self-hosted services.
A registry is not the same as a running container. Pulling an image downloads it to a local image store; running a container creates an instance from that image.
BuildKit is Docker's modern build backend. It improves build performance and capabilities through features such as concurrent build steps, more efficient transfer of build context, advanced caching, and secret or SSH mounts.
Docker Buildx is a Docker CLI plugin for using BuildKit builders. It supports multi-platform builds, cache import and export, and more advanced build workflows. Modern Docker installations use BuildKit for the standard build experience, although exact capabilities depend on the builder and Docker version.
Docker Engine uses lower-level components to manage container lifecycle and execution. containerd manages container lifecycle and image-related operations, while runc is a low-level runtime that creates and runs containers according to the Open Container Initiative (OCI) runtime specification.
Developers do not normally need to invoke these components directly for everyday Docker workflows, but knowing that Docker builds on a broader container ecosystem helps explain why the same OCI images can run on different container platforms.
The correct installation method depends on your operating system and whether you need a local development environment or a server runtime.
Docker Desktop provides a packaged development environment for macOS and Windows. It includes Docker Engine components, the Docker CLI, Compose, and a graphical interface. On Windows, it commonly uses the Windows Subsystem for Linux 2 (WSL 2) backend for Linux containers.
Check current system requirements, licensing, and organisation-use terms before installing Docker Desktop. Its availability and commercial licensing conditions are not identical to those of the open-source Docker Engine components.
Linux developers can install Docker Engine using the official repository instructions for their distribution. Follow the current documentation for Ubuntu, Debian, Fedora, or another supported distribution rather than copying installation commands from an old tutorial. Repository setup, package names, and supported versions can change.
Official installation instructions: Install Docker Engine.
Avoid running an unreviewed installation script with administrator privileges simply because it appears in a quick-start guide. Prefer the official installation documentation and understand what repository and packages are being installed.
After installation, check that the client and daemon are available:
docker --version
docker version
docker info
The first command displays the client version. The second reports client and server information, while the third provides details about the Docker environment.
Run a test container:
docker run hello-world
Docker downloads the image if it is not already available locally, creates a container, runs the test program, and displays a confirmation message. If the command fails, check that the daemon is running, that your user has the required permissions, and that network access to the registry is available.
On many Linux installations, access to the Docker daemon is controlled through a Unix socket. Adding a user to the docker group can make Docker commands available without sudo, but membership in that group can effectively grant root-level control of the host. Do not treat it as an ordinary low-privilege permission. Use a trusted account and review rootless mode if a non-root daemon is appropriate.
You do not need to memorise every Docker command at once. Start with the commands used to inspect images, create containers, view logs, and clean up resources.
| Command | Purpose |
|---|---|
docker version |
Display client and server versions |
docker info |
Inspect daemon and environment information |
docker pull nginx |
Download an image |
docker images |
List local images |
docker build -t my-app:1.0 . |
Build an image from a Dockerfile |
docker run --name web -p 8080:80 nginx |
Create and start a container with a published port |
docker ps |
List running containers |
docker ps -a |
List all containers, including stopped ones |
docker logs web |
Display container logs |
docker exec -it web sh |
Open a shell in a running container when the image includes one |
docker stop web |
Stop a running container |
docker start web |
Start an existing stopped container |
docker rm web |
Remove a stopped container |
docker rmi nginx |
Remove an image when no longer in use |
docker system df |
Show Docker disk usage |
A typical interactive run command might look like this:
docker run --name demo-web -d -p 8080:80 nginx
docker ps
docker logs demo-web
docker stop demo-web
The -d flag runs the container in detached mode. The -p option publishes a host port to a container port. In this example, requests to port 8080 on the host are forwarded to port 80 inside the container.
Be careful with cleanup commands. docker system prune and related commands can remove unused objects, and options such as --volumes can remove data you intended to keep. Inspect what will be affected and back up important data before pruning.
Images are the units developers build, test, publish, and deploy. A reliable image workflow starts with a known base image, a reproducible build, meaningful tags, and a registry that can be accessed by the environments that need it.
Download an image from a registry with:
docker pull nginx:1.27
docker image ls
docker image inspect nginx:1.27
The tag identifies a particular image reference. Tags such as 1.27 or an application release number are easier to reason about than a floating latest tag. A tag is mutable: a publisher can update what it points to. For stronger deployment reproducibility, record and deploy an image digest where appropriate.
After building an image, tag it with a registry and repository name:
docker tag my-app:1.0 registry.example.com/team/my-app:1.0
docker login registry.example.com
docker push registry.example.com/team/my-app:1.0
Authenticate using the registry's recommended credential mechanism. Do not place passwords or access tokens directly in shell history, Dockerfiles, or source control. In automated pipelines, use the platform's secret store and short-lived or narrowly scoped credentials where possible.
A base image influences image size, compatibility, patch availability, and the tools available during runtime. Official images and images from trusted vendors are useful starting points, but they still need to be updated and assessed.
Minimal images can reduce the number of installed packages and potential attack surface. However, a smaller image is not automatically more secure: maintainers, update cadence, included libraries, and your application's runtime requirements all matter. Alpine-based images use musl libc rather than glibc, which can affect compatibility with some prebuilt binaries and language packages. Test the application in the actual target image instead of choosing a base solely by compressed size.
A Dockerfile describes how to build an image. It is a version-controlled recipe that lets a developer or CI pipeline reproduce the environment without manually repeating installation steps.
Consider a simple Python application with a requirements.txt file and an app.py entry point:
from http.server import BaseHTTPRequestHandler, HTTPServer
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
body = b"Hello from a Docker container"
self.send_response(200)
self.send_header("Content-Type", "text/plain")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
if __name__ == "__main__":
HTTPServer(("0.0.0.0", 8000), Handler).serve_forever()
A simple Dockerfile could be:
FROM python:3.12-slim
WORKDIR /app
COPY app.py .
EXPOSE 8000
CMD ["python", "app.py"]
Build and run the image:
docker build -t hello-container:1.0 .
docker run --rm -p 8000:8000 hello-container:1.0
Open http://localhost:8000 in a browser. The application listens on all interfaces inside the container, and Docker publishes the container's port 8000 on the host.
This example is intentionally small. A real application should pin and manage dependencies, run as a non-root user, use an appropriate health and shutdown strategy, and avoid embedding secrets or development-only files in its image.
The most common Dockerfile instructions define the base image, build directory, files, environment, ports, and startup command.
| Instruction | What it does | Typical use |
|---|---|---|
FROM |
Selects a base image or previous build stage | Choose a runtime or build environment |
WORKDIR |
Sets the working directory for later instructions | Keep application files in a predictable location |
COPY |
Copies files from the build context or another stage | Add source code and required assets |
ADD |
Copies files and supports additional source handling | Use only when its extra behaviour is needed |
RUN |
Executes a command during image construction | Install packages or build application assets |
ENV |
Sets environment variables in the image | Define non-secret runtime defaults |
ARG |
Defines a build-time argument | Parameterise build instructions |
EXPOSE |
Documents a port the application listens on | Describe the intended container port |
USER |
Selects the user for subsequent instructions and runtime | Run the application without unnecessary privileges |
ENTRYPOINT |
Defines the executable for the container | Create an image with a primary executable |
CMD |
Supplies default command or arguments | Set the default process or entrypoint arguments |
HEALTHCHECK |
Defines a container health check | Detect whether an application process is healthy |
VOLUME |
Declares a mount point for external data | Indicate data intended to live outside the image |
CMD provides a default command or arguments. ENTRYPOINT configures the executable that the container runs. They can be combined: an entrypoint defines the program, while CMD provides default arguments. Prefer the exec form with a JSON array for application processes so the process receives signals correctly and can handle shutdown.
For example:
ENTRYPOINT ["python"]
CMD ["app.py"]
This runs python app.py by default. Supplying an alternative command argument when starting the container changes the CMD arguments while retaining the entrypoint.
EXPOSE documents the port an application listens on inside the container. It does not publish that port on the host. To make the service reachable from the host, use -p with docker run or a ports mapping in Compose.
ARG values are available during image build and are not automatically persisted as runtime environment variables. ENV values are stored in image configuration and are available to the running container. Neither should be treated as a secure way to store credentials. Build secrets should use BuildKit secret mounts, while runtime secrets should be supplied through an appropriate secret-management mechanism.
Docker images are assembled from layers. Many Dockerfile instructions create filesystem changes that can be cached and reused in later builds. When an instruction and its dependencies have not changed, the builder may reuse its previous result rather than repeat the work.
This makes Docker builds more efficient, but only when the Dockerfile is structured thoughtfully.
Dependencies usually change less often than application source code. Copying dependency manifests and installing packages before copying frequently changed source files can preserve the dependency installation cache.
For example:
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "app.py"]
If only app.py changes, Docker can often reuse the dependency installation layer. If requirements.txt changes, the dependency layer must be rebuilt.
The build context is the set of files made available to the builder. Sending unnecessary files can slow builds and accidentally include secrets, local environments, test output, or version-control metadata.
A .dockerignore file excludes files and directories from the build context:
.git
.venv
__pycache__
*.pyc
.env
.env.*
node_modules
dist
coverage
.pytest_cache
Adapt the list to the project. Do not exclude files required by the build, and do not assume that a file excluded from the image is automatically protected if it is still sent to another process outside the build context.
BuildKit supports cache mounts that preserve package-manager or compiler caches between builds without copying those caches into the final image. This can speed up repeated builds.
For example, a Python build might use a cache mount for pip's download cache:
# syntax=docker/dockerfile:1
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
pip install -r requirements.txt
Cache mounts are an optimisation, not a correctness requirement. The build must still work if the cache is empty or has been removed. For CI systems that use fresh runners, configure an external build cache when it is worthwhile.
Do not pass credentials through Dockerfile ARG or copy secret files into an image. Those values may be exposed through image layers, build metadata, logs, or cache records.
BuildKit provides secret mounts that expose a secret only to the build step that needs it:
# syntax=docker/dockerfile:1
FROM alpine
RUN --mount=type=secret,id=api_token \
TOKEN="$(cat /run/secrets/api_token)" && \
./download-private-dependency.sh
Pass the secret at build time using the supported Buildx or Docker build option. The secret mount is designed not to persist the secret in the resulting image. Review the build process itself to ensure commands do not print or copy the secret elsewhere.
A development or build environment often contains compilers, headers, package managers, test tools, and other utilities that are unnecessary at runtime. A multi-stage Dockerfile separates building from execution so the final image can contain only the files and runtime dependencies needed to run the application.
For example:
# syntax=docker/dockerfile:1
FROM python:3.12-slim AS builder
WORKDIR /build
COPY requirements.txt .
RUN pip wheel --no-cache-dir --wheel-dir /wheels -r requirements.txt
FROM python:3.12-slim AS runtime
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /app
COPY --from=builder /wheels /wheels
COPY requirements.txt .
RUN pip install --no-cache-dir --no-index --find-links=/wheels -r requirements.txt \
&& rm -rf /wheels
COPY app.py .
RUN useradd --create-home --uid 10001 appuser
USER appuser
EXPOSE 8000
CMD ["python", "app.py"]
The first stage builds dependency wheels. The runtime stage installs those wheels and copies only the application files it needs. A production project may need additional system libraries or a different dependency strategy, so test the final image rather than assuming the example fits every Python package.
Multi-stage builds are useful for compiled languages, frontend applications, and services with large build dependencies. They can reduce image size, lower the number of tools present at runtime, and make the boundary between build-time and runtime dependencies clearer.
Modern applications commonly depend on more than one process. A web API may need a database and cache; a frontend may need an API; a development environment may include a message broker and a test service. Docker Compose lets you describe these services, networks, and volumes in one YAML file.
Compose is useful for local development, automated testing, demonstrations, and some single-host deployments. It does not by itself provide the multi-host scheduling, high availability, and cluster management features of a full orchestration platform.
The following configuration starts a Python API and a PostgreSQL database:
services:
api:
build:
context: .
ports:
- "8000:8000"
environment:
DATABASE_URL: postgresql://app:local-dev-password@db:5432/appdb
depends_on:
db:
condition: service_healthy
db:
image: postgres:17
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: local-dev-password
POSTGRES_DB: appdb
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
interval: 5s
timeout: 3s
retries: 10
volumes:
postgres_data:
This is a local-development example, not a production secret-management pattern. Replace example credentials with securely supplied values, and do not commit real passwords. The API connects to the database using the service name db because Compose creates a network where services can resolve one another by service name.
The health check asks PostgreSQL whether it is ready to accept connections. The depends_on condition delays API startup until the database reports healthy. Application code should still handle connection failures and retries because dependencies can become unavailable after startup.
Use the modern Compose plugin syntax:
docker compose up --build
docker compose up -d
docker compose ps
docker compose logs -f api
docker compose exec api sh
docker compose down
The first command builds required images and starts services in the foreground. The detached option runs them in the background. docker compose down removes the project containers and network, but named volumes are retained unless you explicitly request their removal. This is important when the database contains data you want to keep.
Compose supports multiple configuration files, profiles, environment substitution, and other features that help separate development and deployment concerns. A common approach is to keep a base configuration and use an additional development override for source-code mounts, debug settings, or development-only services.
Keep configuration readable. A Compose file should describe the services and their relationships, not become a place to hide complex application logic or hard-coded production credentials.
Official reference: Docker Compose documentation.
Containers need network connectivity to communicate with one another and with external clients. Docker provides network drivers and configuration options that define how containers reach services.
A bridge network is commonly used for containers on a single Docker host. Docker creates a default bridge network, but user-defined bridge networks are generally preferable for application stacks because they provide automatic DNS resolution between connected containers and allow better separation.
Create a network and attach containers:
docker network create app-network
docker run -d --name api --network app-network my-api:1.0
docker run -d --name worker --network app-network my-worker:1.0
docker network inspect app-network
Containers on the same user-defined bridge network can communicate using their container or service names and the ports on which their applications listen. Publishing a port is not necessary for container-to-container communication on that network.
A port mapping exposes a container port through the host:
docker run -d --name web -p 127.0.0.1:8080:80 nginx
Binding to 127.0.0.1 makes the published port available through the host's loopback interface rather than all host interfaces. This can be a safer choice for local development. Publishing to all interfaces may expose the service to other machines depending on host firewall and network configuration.
Docker supports additional network drivers for different requirements:
Do not choose a network driver simply because it sounds more advanced. Start with a user-defined bridge network for a typical single-host application, then introduce other drivers when the deployment architecture requires them.
A container's writable layer is not a reliable place for data that must survive container replacement. If a database writes its files only into the container layer, removing the container can remove that data. Docker provides mounts to store data outside the container's writable layer.
Named volumes are managed by Docker and are a common choice for persistent application data:
docker volume create app-data
docker run -d --name database \
-v app-data:/var/lib/postgresql/data \
postgres:17
docker volume ls
docker volume inspect app-data
The volume can be mounted into a replacement container, allowing the data to outlive an individual container. A volume is not a backup. Backups should be created, stored separately, protected, and tested through restoration exercises.
A bind mount maps a specific host path into a container. It is useful during development when source-code changes should be visible inside the running container:
docker run --rm -it \
-v "$PWD":/app \
-w /app \
python:3.12-slim \
python app.py
Bind mounts expose host files to the container. Use read-only mounts where appropriate and avoid mounting sensitive host directories unnecessarily. File ownership and permission behaviour can also differ between Linux, macOS, and Windows environments.
A tmpfs mount stores temporary data in memory on supported Linux environments. It can be useful for short-lived files that should not be written to the container layer or persistent storage. It is not suitable for data that must survive a container stop or host restart.
Choose the mount type according to the data's lifetime, sharing needs, performance requirements, and security implications.
Applications often need different settings in development, testing, staging, and production. Database connection strings, service URLs, feature flags, and logging levels should be supplied through environment-specific configuration rather than hard-coded into the image.
Docker supports environment variables through the -e or --env options, Compose's environment and env_file settings, and other runtime configuration mechanisms.
For example:
docker run --rm \
-e APP_ENV=development \
-e LOG_LEVEL=debug \
my-app:1.0
A Compose file can use variable substitution so that values are supplied by the environment or an appropriate environment file. Remember that environment variables are not automatically secret: they may be visible to users or processes with sufficient access to inspect the container or host.
Use a dedicated secrets manager or the platform's supported secret mechanism for sensitive values. Avoid committing populated environment files to Git, and make sure logs and crash reports do not expose credentials.
One of Docker's most practical uses is creating a development environment that every contributor can start in a consistent way. Instead of asking each developer to install a database, cache, and message broker manually, a project can define those dependencies in Compose.
A typical local workflow might include:
docker compose up --build.Containers are also useful in automated tests. CI pipelines can start database and cache services, build the application image, execute tests against those services, and publish an image only after the required checks pass.
A containerised test environment does not guarantee identical production behaviour. Production may differ in network policies, storage, resource limits, identity, secrets, and external dependencies. Use staging and deployment checks to validate the real target environment.
A production container workflow involves more than running docker run on a server. Teams need a repeatable way to build images, identify releases, store them, configure runtime services, manage persistent data, and roll back when a deployment fails.
Build an image in CI, run tests and security checks, then publish it to a trusted registry. Deploy the same immutable image digest to staging and production rather than rebuilding separately for each environment. This reduces the chance that the tested artifact differs from the released artifact.
Use meaningful release tags, such as a semantic version or commit identifier, and record the digest. Avoid relying on latest as a production release identifier because it can point to different content over time.
A process being started does not necessarily mean that the application is ready to serve traffic. Use health checks and application-level readiness signals where the platform supports them. Applications should handle termination signals, stop accepting new work, finish or safely abandon in-flight operations, and close connections within the configured shutdown window.
Health checks should test meaningful application readiness without creating excessive load or depending on unrelated external systems unnecessarily.
Treat containers as replaceable application instances. Store persistent data in managed databases, volumes, or storage services designed for the workload. Back up databases and files independently of the container image, define retention, and test recovery.
Docker images can run on a single Linux server, a managed container service, or an orchestrated cluster. Examples include Amazon ECS, Azure Container Apps, Google Cloud Run, and Kubernetes.
Compare platforms based on networking, persistent storage, scaling, health checks, background jobs, secrets, deployment strategy, observability, regional availability, and cost. A small application may not need a cluster. A managed container platform can reduce operational work, while a self-managed cluster gives more control at the cost of greater responsibility.
A CI/CD pipeline can build an image, run unit and integration tests, scan dependencies and image layers, publish the artifact, deploy to staging, and promote it to production after approval. Keep credentials scoped to the workflow that needs them and protect production deployment environments.
For a broader discussion of delivery workflows, see our guide to CI/CD for machine learning and deployment strategies.
Containers provide useful isolation, but they should not be treated as a complete security boundary. A secure deployment combines trusted images, minimal privileges, host protection, network controls, patch management, and careful handling of build and runtime secrets.
Many images default to running as root inside the container. Create a dedicated user for the application and select it with the Dockerfile USER instruction. Grant only the filesystem permissions and capabilities the process requires.
Running as a non-root user limits the impact of some application compromises, but it does not replace host security or other container controls. Verify that the application can still read required files and write only to intended directories.
Choose images from trusted publishers, review their provenance and update cadence, and rebuild regularly to incorporate security fixes. Pinning a tag helps control versions, while digest pinning can identify exact image content. Establish a process to update pinned images when security patches become available.
Use vulnerability scanning to identify known issues in base images and application dependencies. A scan is a point-in-time assessment, not a guarantee that an image is safe.
Avoid --privileged unless there is a specific, reviewed requirement. It grants broad access and weakens isolation. Keep the default capability restrictions, drop capabilities that the application does not need, and use read-only root filesystems where practical.
Use seccomp, AppArmor, or SELinux policies as appropriate to the host platform. Avoid exposing the Docker daemon socket to application containers: access to that socket can grant extensive control over the host.
Never store passwords, tokens, private keys, or certificates in a Dockerfile or source repository. Use BuildKit secret mounts for build-time access and a runtime secret store or orchestrator-supported mechanism for deployed applications.
If a credential is accidentally committed or included in an image, revoke or rotate it. Deleting the line from a later commit does not make the original credential safe.
Publish only the ports that need to be reachable. Separate services into user-defined networks according to their communication requirements. Apply host firewall rules and platform network policies as appropriate.
Set CPU and memory limits based on measured workload requirements. Resource limits can reduce the risk of one service exhausting host capacity, but limits that are too low can cause throttling, restarts, or failed requests.
Rootless mode runs both the Docker daemon and containers as a non-root user inside a user namespace. It can reduce the impact of some daemon and container vulnerabilities, although it has prerequisites and networking or storage considerations. Evaluate compatibility with your workload before adopting it.
Container security depends on the host kernel, Docker Engine, containerd, runc, BuildKit, base images, and application dependencies. Track official release notes and security advisories, apply updates according to risk, and rebuild images when their underlying packages need patching.
As of November 5, 2026, the Docker Engine 29 release notes list version 29.9.3, dated November 3, 2026. That release includes security fixes involving crafted RSA key parsing, zstd decompression, Swarm overlay protections, SwarmKit certificate handling, and BuildKit. Check the official Docker Engine 29 release notes and the relevant advisories for the versions you operate; do not assume every environment is running the latest release.
Containers are easier to operate when teams can inspect their state, read logs, measure resource use, and identify failures quickly.
Useful commands include:
docker ps
docker ps -a
docker inspect my-container
docker stats
docker top my-container
docker ps shows running containers, while docker ps -a includes stopped containers. docker inspect returns detailed configuration and runtime information. docker stats displays resource usage, and docker top shows processes running in a container.
Use:
docker logs my-container
docker logs --follow --tail 100 my-container
Applications should generally write logs to standard output and standard error so the container runtime can collect them. For production systems, configure appropriate log rotation, retention, access control, and centralised collection. Avoid logging credentials, access tokens, or sensitive customer data.
| Symptom | Possible cause | What to inspect |
|---|---|---|
| Container exits immediately | Application error, wrong command, missing configuration | Container logs, exit code, entrypoint and CMD |
| Port is unreachable | Port not published, wrong bind address, firewall, application not listening | Port mapping, application bind address, host firewall |
| Container cannot reach another service | Wrong network, hostname, port, or service readiness | Network membership, DNS, service logs |
| Data disappears after recreation | Data stored only in container writable layer | Volume and bind-mount configuration |
| Image build is unexpectedly slow | Large build context, poor cache ordering, missing build cache | Dockerfile, .dockerignore, BuildKit output |
| Container is out of memory | Memory limit too low or application growth | Resource limits, metrics, application profiling |
| Permission denied | Container user cannot access mounted files | UID/GID ownership, mount permissions, USER instruction |
Troubleshoot one layer at a time. Check the container state, inspect its logs, verify the image and configuration, then examine networking and storage. Avoid changing several settings simultaneously because that makes the root cause harder to identify.
For broader observability, teams can integrate container workloads with Prometheus, Grafana, and OpenTelemetry.
Docker and Kubernetes are often discussed together, but they solve different problems.
Docker provides tools for building images and running containers. Docker Compose defines and runs related services, commonly on one host. Kubernetes is a container orchestration platform that manages workloads across a cluster of machines. It provides scheduling, service discovery, declarative deployment, health management, and mechanisms for scaling and replacing application instances.
| Capability | Docker Engine and Compose | Kubernetes |
|---|---|---|
| Build images | Yes, using Docker build tooling | Usually uses an external image builder |
| Run containers | Yes | Yes, through a container runtime |
| Local multi-service setup | Compose is designed for this | Possible, but generally more involved |
| Multi-node scheduling | Not the purpose of standalone Compose | Core capability |
| Declarative workload management | Compose configuration | Kubernetes API objects |
| Service discovery and cluster networking | Host or Swarm networking options | Cluster networking and Services |
| Operational complexity | Often lower for single-host use | Higher; requires cluster operations |
Kubernetes does not require Docker Engine as its container runtime. It uses the Container Runtime Interface (CRI), commonly with containerd or CRI-O. Docker-built images can still be used because OCI-compatible images are supported by the broader ecosystem.
Start with Docker and Compose when learning container concepts or building a small multi-service application. Learn Kubernetes when you need its scheduling, multi-node orchestration, and operational capabilities and are prepared to manage the additional complexity.
For a deeper introduction, read our Kubernetes guide.
The latest tag is convenient for experiments but does not identify a stable release. Pin image versions or digests in repeatable deployments and update them deliberately.
A root process inside a container can increase the impact of a compromise. Use a dedicated non-root user where possible and grant only required permissions.
Dockerfiles, image layers, build arguments, and repository history can expose credentials. Use dedicated build and runtime secret mechanisms, and rotate any credential that has been exposed.
Containers are replaceable. Store persistent data in volumes or external storage, and create tested backups. Do not confuse a named volume with a backup strategy.
Large contexts slow builds and may include local files that do not belong in the image. Use a carefully maintained .dockerignore file.
A container's internal port is not automatically available from the host. Understand the difference between service-to-service networking, port publishing, and host binding.
Compose is excellent for local development and some single-host deployments. It does not automatically provide cluster scheduling, multi-host failover, or all production operations. Choose deployment infrastructure based on actual availability and scaling requirements.
A container image can contain outdated operating-system packages or application dependencies. Rebuild and scan images regularly, and monitor the security advisories for the components you use.
The following example combines a Dockerfile, Compose, and a persistent PostgreSQL service. It illustrates how the pieces fit together in a development workflow.
A typical project might contain:
my-api/
├── app/
│ └── main.py
├── requirements.txt
├── Dockerfile
├── compose.yaml
└── .dockerignore
A dependency file might include FastAPI and an ASGI server:
fastapi
uvicorn[standard]
A simple application entry point:
from fastapi import FastAPI
app = FastAPI()
@app.get("/health")
def health():
return {"status": "ok"}
A Dockerfile:
# syntax=docker/dockerfile:1
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app ./app
RUN useradd --create-home --uid 10001 appuser \
&& chown -R appuser:appuser /app
USER appuser
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
A Compose configuration:
services:
api:
build: .
ports:
- "8000:8000"
environment:
DATABASE_URL: postgresql://app:local-dev-password@db:5432/appdb
depends_on:
db:
condition: service_healthy
db:
image: postgres:17
environment:
POSTGRES_USER: app
POSTGRES_PASSWORD: local-dev-password
POSTGRES_DB: appdb
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
interval: 5s
timeout: 3s
retries: 10
volumes:
postgres_data:
The password is deliberately an example for a local tutorial only. Use a secure configuration mechanism for real deployments and do not commit production secrets.
Create a .dockerignore file:
.git
.venv
__pycache__
*.pyc
.env
.pytest_cache
Start the environment:
docker compose up --build
Check the API at http://localhost:8000/health. Inspect service status and logs:
docker compose ps
docker compose logs -f api
docker compose logs -f db
Stop the services:
docker compose down
The named PostgreSQL volume remains unless it is explicitly removed. This lets the database retain local development data across container recreation. For a real application, add database migrations, automated tests, connection retry logic, secure secret handling, and a backup and restore procedure.
This example is a foundation rather than a complete production template. A production image should use a tested dependency lock, a deliberate base-image update policy, an appropriate health and shutdown strategy, and a release process that promotes the same tested image between environments.
Docker is particularly useful when an application has dependencies that are difficult to install consistently, when several services must run together, or when development and production environments need a reproducible packaging process.
Use containers when:
Docker may be unnecessary for a very small static website, a simple script with no complex dependencies, or a project where the deployment platform already provides a simpler and reliable build environment. Containers introduce their own concepts: images, registries, networking, volumes, build contexts, and runtime permissions. Use them when those capabilities solve a real problem rather than adding them automatically to every project.
Docker is a platform that helps developers package applications and their dependencies into images, then create and run containers from those images. It makes application environments easier to reproduce across development, testing, and deployment systems.
Containerization is a method of isolating applications and their processes while sharing the host operating system kernel. An application is packaged with its user-space dependencies so it can run in a consistent environment.
An image is a read-only template containing the filesystem and configuration used to create an application environment. A container is a running or stopped instance created from an image, with its own runtime configuration and writable layer.
No. A traditional container shares the host operating system kernel, while a virtual machine runs a guest operating system on virtualized hardware. Containers are generally lighter, but they do not provide the same kernel-level separation as a virtual machine.
Docker Engine and many Docker ecosystem components are open source, but Docker Desktop has separate licensing terms and some Docker products and services are commercial. Review the current Docker subscription and licensing information and the terms that apply to your organisation before adopting a product.
Docker Compose defines and runs multi-container applications using a YAML configuration file. It is commonly used to start an API, database, cache, and other dependent services together in a repeatable local development environment.
A Dockerfile is a text file containing instructions for building a Docker image. It specifies a base image, working directory, files, dependencies, environment settings, and the default process to run.
A Docker volume is persistent storage managed by Docker and mounted into a container. It allows data to outlive the lifecycle of an individual container. Volumes are useful for databases and other stateful services, but they still need independent backups.
Docker provides isolation and security controls, but security depends on how the daemon, host, images, containers, networks, and credentials are configured. Use trusted images, run processes as non-root users where possible, avoid unnecessary privileges, protect the Docker socket, and keep components updated.
No. Docker Engine and Docker Compose can run containers without Kubernetes. Kubernetes becomes useful when a system needs cluster scheduling, multi-node orchestration, declarative service management, and related operational capabilities.
Yes. Docker Desktop provides a development environment on supported Windows and macOS systems. Linux containers run using a Linux environment or virtualisation layer where required. Exact platform requirements and supported features depend on the Docker Desktop version.
Docker is a broader platform with tools for building and managing containers. containerd is a lower-level container runtime component focused on managing container lifecycle and image operations. Docker Engine uses containerd as part of its architecture, and other platforms can use containerd independently.
A container normally runs the process defined by its entrypoint and command. If that main process exits, the container stops. Inspect the logs, exit code, command, environment variables, and application configuration to identify why the process ended.
Store persistent data in a named volume, bind mount, or external storage service rather than relying on the container's writable layer. Back up important data and test restoration; a volume alone is not a backup.
Docker and containerization make it easier to package, test, distribute, and operate applications in consistent environments. By understanding images, containers, Dockerfiles, BuildKit, Compose, networking, and volumes, developers can build workflows that are easier to reproduce and maintain.
Start with a small application, write a Dockerfile, build an image, and run it locally. Add Compose when the application depends on multiple services. As the project grows, introduce multi-stage builds, automated tests, image scanning, controlled deployments, monitoring, and a deliberate backup strategy.
The most effective container workflow is not the one with the most tools or the largest cluster. It is the simplest approach that meets the application's requirements while remaining secure, repeatable, observable, and maintainable. Learn the fundamentals first, apply them to a real project, and add operational complexity only when the system needs it.

Secure healthcare management REST API with authentication and role-based access control for medical records.

Learn how Docker images and containers work, build and run an application, manage storage and networking, use Compose, optimize images, and secure containerized workloads.
Read
Learn how to design structured application logs, choose severity levels, correlate requests, protect sensitive data, and build reliable observability for production systems.
Read
Learn how environment variables configure applications across development and production, with practical examples for Python, Node.js, Docker, Kubernetes, cloud platforms, validation, and secure secrets management.
Read