Hobyte

Deploy Hugo With Flatcar

This blog was running on a debian server with unattended upgrade enabled and caddy for serving the build hugo site with tls. This worked nicely, but the unattended updates feel a bit brittle and my deploy process wasn’t the best either. It consisted of this GitLab CI job building the hugo page and pushing it to the server with rsync. Works, but doesn’t feel reliable.

stages:
  - build
  - deploy

build:
  stage: build
  image: registry.gitlab.com/hobyte/container/hugo:main
  script:
    - hugo version
    - hugo --minify (1)
  artifacts:
    paths:
      - public
  only:
    - main

deploy:
  stage: deploy
  image: debian:12.11
  script:
    (2)
    - apt-get update -y
    - apt-get install openssh-client rsync -y
    - eval $(ssh-agent -s)
    - chmod 400 "$BLOG_KEY"
    - ssh-add "$BLOG_KEY"
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - rsync -rlvz -e 'ssh -o StrictHostKeyChecking=no' --delete public/ gitlab@hobyte.eu:/var/www/blog/ (3)
  dependencies:
    - build
  only:
    - main

  1. build the blog

  2. import the ssh key for the server from the CI Variables

  3. copy the blog to the server with rsync

Something better

I wanted a setup, where the OS would update itself and the blog would be updated when I change something. All without me doing anything. So for running the blog, I quickly decided to use docker. As Caddy served me well in my old setup, the Idea is to build a docker image with Caddy and the Blog and deploy it automatically to a server. Then I only need a OS that would update itself automatically. For this I choose flatcar linux, as it seemed easy and straightforward to use.

The Plan

plan.drawio

I already use a git repository for my blog, so only the right side of the plan is new. I wan to build a docker image with caddy and the blog with the GitLab CI Pipeline. This image is pushed to the GitLab container registry. The server is running flatcar linux, which updates itself. On top of that, the container from the container registry is running. I just have to find a way to update it when it changes or on a timer.

Building the Docker Image

The Dockerfile for building the caddy image uses a two stage build. First, a custom container with hugo is used to build and minify the site. The second stage uses the official caddy image an copies the build blog to the right directory:

# build blog with hugo
FROM registry.gitlab.com/hobyte/container/hugo:main AS builder
WORKDIR /app
COPY . .
RUN hugo --minify

# build Caddy image
FROM docker.io/caddy:2.10
COPY Caddyfile /etc/caddy/Caddyfile
COPY --from=builder /app/public /srv

To automate this build, a simple GitLab CI/CD Pipeline builds this image and pushes it to the GitLab container registry. To make this easer, I used the to be continuous templates. They automatically build, scan and push the image. Additionally, I added scanning for credentials, just for good measure:

# included templates
include:
  # Docker template
  - component: "$CI_SERVER_FQDN/to-be-continuous/docker/gitlab-ci-docker@8.0"
    inputs:
      trivy-disabled: true
      prod-publish-strategy: "auto"
  # Gitleaks template
  - component: "$CI_SERVER_FQDN/to-be-continuous/gitleaks/gitlab-ci-gitleaks@2.8"

Flatcar Linux

There are multiple Linux Distributions that can automatically update themselves. I choose flatcar linux over CoreOS and openSUSE MicroOS, as it looked more mature and stable to me.

Flatcar uses two partitions to update the os in the background and allow a rollback, if the update has an error. Additionally, it can be configured with config file that is applied on the first boot. This allows for an easy installation with automatic configuration.

Following the flatcar systemd getting started guide, I configured a systemd Service to pull and start the caddy container from gitlab. to update it to a newer version, a timer restarts this service daily, forcing a pull of the caddy container. Additionally, I set the hostname and configured a user with a public ssh key for authentication. This allows me to connect with ssh and troubleshoot the setup if it’s ever needed (hopefully not).

variant: flatcar
version: 1.1.0

storage:
  files:
  - path: /etc/hostname (1)
    contents:
      inline: "hobyte.eu"

passwd:
  users:
    - name: core (2)
      ssh_authorized_keys:
        - "ssh-rsa xxxxxxx...."
systemd:
  units:
    - name: caddy.service (3)
      enabled: true
      contents: |
        [Unit]
        Description=Caddy service
        [Service]
        TimeoutStartSec=0
        ExecStartPre=-/usr/bin/docker rm --force caddy
        ExecStart=/bin/sh -c '/usr/bin/docker login registry.gitlab.com -u flatcar-blog -p secret && /usr/bin/docker run --name caddy --pull=always --cap-add=NET_ADMIN -p 80:80 -p 443:443 -p 443:443/udp registry.gitlab.com/hobyte/blog:main'
        ExecStop=/usr/bin/docker stop caddy
        Restart=always
        RestartSec=5s
        [Install]
        WantedBy=multi-user.target
    - name: caddy-restart.timer (4)
      enabled: true
      contents: |
        [Unit]
        Description=Restart Caddy service once a day
        [Timer]
        OnCalendar=daily
        Unit=caddy.service
        [Install]
        WantedBy=multi-user.target

  1. Set the hostname

  2. Create a user and add a public ssh key for authentication

  3. Systemd service to run the caddy container

  4. Systemd timer to restart the caddy service daily

With this setup complete, I used a rescue system on the server to install flatcar linux with the install script. This was needed as the provider I’m using is not supported by flatcar out of the box.

Summary

I migrated my blog from debian with caddy to flatcar linux with a prebuilt docker image. Both the OS and the container are updated automatically, making this a practical 0-ops deployment. This allows me to focus on my project an writing blog posts instead of managing and updating the infrastructure for my blog (which is also boring).