Garbage collector for Docker

February 28, 2018

If you are heavily using Docker on your local environment, it’s common to have a lot of dangling images, stopped containers, orphans volumes, etc. And if you have multiple environments, it can be challenging to clean only useless things without impacting what you want to keep, like databases, for example.

Before executing something, please be sure to understand what it implies!

Manual cleaning

It’s possible to remove images manually, containers and volumes.

1# Remove all dangling images
2docker rmi $(docker images --filter="dangling=true" --quiet)
3
4# Remove all stopped containers
5docker rm $(docker ps --filter="status=exited" --quiet)
6
7# Remove all dangling volumes
8docker volume rm $(docker volume ls --filter="dangling=true" --quiet)

You can even use the built-in version of these three commands.

1# Remove all unused data
2docker system prune --volumes

But what if you want to remove everything except some specific elements?

Advanced cleaning

Spotify teams provide a garbage collector that can run as a scheduled task with native whitelist support. Furthermore, it can run through a Docker container! You can find all details about the usage on GitHub. The only thing missing from that tool, in my opinion, is the volumes cleanup.

That’s why I decided to create a small script to implement a workaround until that feature is officially supported (maybe once this pull request will be merged). The script is only composed of four main steps.

1. Check if there are exclusion patterns
2. Request confirmation before going further
These two steps would prevent any unwanted loss if we forgot to configure exclusion patterns or if we accidentally call the script.

3. Remove all unused images and containers
The source code of the Spotify garbage collector is downloaded from GitHub, and the corresponding Docker image is built from it. After that, a container is run with all exclusion patterns passed as environment variables. And all unused images and containers are removed, except those whitelisted.

4. Remove all dangling volumes
All dangling volumes, except those whitelisted, are removed. That last part is only executed if the -v argument is passed to the script; otherwise, the treatment is skipped.

And finally, here is my custom version of the Docker garbage collector.

 1#!/usr/bin/env bash
 2set -euo pipefail
 3
 4# ==========================================================================================================
 5# Custom implementation of a garbage collector for Docker images, containers and volumes.
 6#
 7# Usage:
 8# bash docker-gc.sh
 9#
10# In order to clean Docker volumes, the argument "-v" must be provided to the script.
11#
12# Three additional files are needed.
13#     .gc-excludes-images
14#     .gc-excludes-containers 
15#     .gc-excludes-volumes
16#
17# Those configuration files must be placed into the $HOME/.docker directory. They describe which elements
18# must be preserved by the garbage collector. They must contain "grep" compliant values, but can be empty.
19# ==========================================================================================================
20
21# Define output formats
22question=$(tput bold)
23warning=$(tput setaf 3)
24error=$(tput setaf 1)
25info=$(tput setaf 4)
26success=$(tput setaf 2)
27reset=$(tput sgr0)
28
29# Check whether exclusion patterns are configured
30exclude_files=(
31    "$HOME/.docker/.gc-exclude-images"
32    "$HOME/.docker/.gc-exclude-containers"
33    "$HOME/.docker/.gc-exclude-volumes"
34)
35for file in "${exclude_files[@]}"; do
36  if [[ ! -f "${file}" ]]; then
37    echo "${warning}${file} is missing, potential side-effects may occurred!${reset}"
38    exit 1
39  fi
40done
41
42# Request a confirmation going further
43echo "This action is irreversible. Are you sure you want to continue? Type 'yes' to approve."
44read -r -p "${question}Enter a value:${reset} " response
45if [[ "${response}" != "yes" ]]; then
46    echo "${error}Cleaning cancelled.${reset}"
47    exit 2
48fi
49
50echo ""
51echo "${info}==> Cleaning containers and images...${reset}"
52
53# Remove all unused images and containers
54rm -rf /tmp/docker/spotify-gc && \
55git clone git@github.com:spotify/docker-gc.git /tmp/docker/spotify-gc && \
56docker build -t spotify/docker-gc /tmp/docker/spotify-gc && \
57docker run --rm \
58    -e "EXCLUDE_FROM_GC=$HOME/.docker/.gc-exclude-images" \
59    -e "EXCLUDE_CONTAINERS_FROM_GC=$HOME/.docker/.gc-exclude-containers" \
60    -v /var/run/docker.sock:/var/run/docker.sock -v /etc:/etc spotify/docker-gc
61
62# Remove all dangling volumes
63if [[ $# -eq 1 && $1 == "-v" ]]; then
64    echo ""
65    echo "${info}==> Cleaning volumes...${reset}"
66
67    dangling_volumes=$(docker volume ls --quiet \
68        --filter="dangling=true" | grep -vf \
69        $HOME/.docker/.gc-exclude-volumes || true)
70
71    if [[ ! -z ${dangling_volumes} ]]; then
72        docker volume rm -f $(echo ${dangling_volumes})
73    fi
74fi
75
76echo ""
77echo "${success}==> Cleaning completed.${reset}"