MiniPoD a MINImal POdman Deployment

MiniPoD is a collection of small scripts that wire git, Podman, duplicity, Linux, SSH and systemd in a minimal deployment together.

MiniPoD:

• Provides minipods. A minipod is a script, which creates a Podman pod for a service. All provided minipods cleanly separate the container it's configuration and persistent storage.
• Registers each minipod with systemd for robustness.
• Runs each deployment rootless in its own user space. This gives security against hackerz and podman pods stop --all.
• Makes a clean and complete backup with a systemd service and duplicity.

Requirements:

• Your deployment target is a Linux host. Preferably Ubuntu 22.04 but other distributions should work.
• For continuous deployment you will need GitLab and SSH.
• podman
• duplicity
• age

A GIF showing the greatest feature.

[[TOC]]

Quick start

For testing MiniPoD locally or setting a deployment up without keeping your deployment configuration in a repository, or a GitLab CI job, which automatically deploys when the configuration is updated:

1. Install dependencies:

For Ubuntu 22.04:

bash echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/unstable/xUbuntu_22.04/ /' | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:unstable.list curl -fsSL https://download.opensuse.org/repositories/devel:kubic:libcontainers:unstable/xUbuntu_22.04/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/devel_kubic_libcontainers_unstable.gpg > /dev/null sudo apt update sudo apt install podman duplicity age

For Arch Linux:

bash sudo pacman -S podman crun duplicity age

1. Clone MiniPoD:

bash git clone git@gitlab.com:infai/minipod.git

1. Create configs:

bash cd minipod export ENVIRONMENT=NAME_FOR_YOUR_ENVIRONMENT find . -name env-minipod-test -type f -execdir cp {} $ENVIRONMENT \;

1. Modify minipod's config:

bash vi $ENVIRONMENT

1. Deploy all the minipods:

bash ./minipod enable $ENVIRONMENT

1. Reach each minipod under $SERVICE.$MINIPOD_HOST:$MINIPOD_PORT, for example ckan.localhost:7000.

Setup

A deployment targets a single host. A deployment can have multiple environments. An environment is defined by a set of environment config files.

The following steps should be repeated for each environment:

1. Fork MiniPoD: Everything what you need to deploy and configure an environment is inside the MiniPoD repository. For versioning your own deployment you have to fork MiniPoD.
2. Create your environment config files.
3. Create environment users for your deployment.
4. Setup GitLab environment.

VERY IMPORTANT!

MiniPoD has the convention that in each deployment environment the following names:

• environment config files
• environment user
• GitLab environment

MUST be identical.

Fork MiniPoD

1. Go to MiniPoD's repository.
2. Click on Fork in the upper left corner.
3. Fill out the form and press Fork project.
4. Clone your fork locally and add upstream MiniPoD as a remote:

bash git clone YOUR_FORK cd YOUR_FORK git remote add -f -t main minipod git@gitlab.com:infai/minipod.git

Create environment config files

1. A working start configuration is given by the env-minipod-test files. Copy and edit them to create a configuration for your ENVIRONMENT.

bash cd YOUR_FORK export ENVIRONMENT=NAME_FOR_YOUR_ENVIRONMENT find . -name env-minipod-test -type f -execdir cp {} $ENVIRONMENT \;

1. At least you should set MINIPOD_HOST, MINIPOD_PORT_HTTP and MINIPOD_PORT_HTTPS inside the environment config file in the root directory.
2. You can enable and disable minipods by modifying the MINIPOD_CREATE_SCRIPTS variable.

Create environment user

Create user, configure ssh access and add your public key to the authorized keys file:

1. SSH into deployment server as root.
2. Create environment user user and login as this user:

bash environment=NAME_FOR_YOUR_ENVIRONMENT sudo adduser $environment su - $environment

1. Create an SSH key pair:

bash mkdir ~/.ssh ssh-keygen -b 4096 -t rsa -f .ssh/id_rsa -C $(whoami) cat .ssh/id_rsa.pub >> .ssh/authorized_keys cat .ssh/id_rsa # PRIVATE_DEPLOY_KEY cat .ssh/id_rsa.pub # PUBLIC_DEPLOY_KEY

1. Add your public SSH key to .ssh/authorized_keys, to be able to log in to the environment_user with SSH.

2. In your local shell:

   bash cat .ssh/id_rsa.pub

3. In environment_user's shell:

   bash echo "RESULT FROM LAST COMMAND (4.i)" >> .ssh/authorized_keys

4. Create an GitLab CI environment for this user in your fork's GitLab repository: Go to Deployments -> Environments and create a New Environment with the same name as environment_user.

5. Add SSH_PRIVATE_KEY variable to your forks GitLab repository:

Go to Settings -> CI/CD -> Variables -> Add variable

• Key: SSH_PRIVATE_KEY
• Value: The result from cat .ssh/id_rsa
• Protected: Uncheck
• Environment scope: environment_user

Set up GitLab CI job

1. Add your ENVIRONMENT to .gitlab-ci.yml.

Encrypt secrets

1. Create a secrets file:

```bash environment=NAME_FOR_YOUR_ENVIRONMENT cp $environment "$environment"-secrets vi $environment # Remove non secret variables

```

1. Encrypt secrets file:

bash ./minipod encrypt env-minipod-test

Deploy

After you did the steps described under the setup. You can push your changes to the main branch of your repository.

git add --all
git commit -m "Setup env"
git push

Go to YOUR_FORK -> Deployments -> Environments. To view your deployment.

Backup

Configuration

You can change your backup destination by modifying the BACKUP_DESTIATION variable inside the environment config file in the root directory. Possible backup backend are described in the duplicity documentation under URL FORMAT.

Backups are create automatically with a systemd service. Modify how the backup is run by editing the create-backup script and the minipod_backup.timer template.

Restore a backup

./minipod disable ENVIRONMENT
./restore-backup
./minipod enable ENVIRONMENT

FAQ

Privileged Ports

sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80

Quotas exceeded

For example, the following error indicates, that the kernel key limit is set to small:

join keyctl `d34906332721de9a96f7b4fdd6e771ad23af513edaf07e6ee75438992c850bef`: Disk quota exceeded

sudo sysctl -w kernel.keys.maxkeys=2000
sudo sysctl -w kernel.keys.maxbytes=2000000

Also check out for further limits: Production setup - LXD documentation

Updating

TODO

Deploy script

TODO

Creating a new Minipod

TODO

1. Should have create.sh
2. use template
3. Parameters that wont change inside create script
4. named volumes must be unique prefix "$APP_NAME"
5. Header
6. Should have nuke.sh
7. Template

Project owners

@fbrei @BonaBeavis @Kibubu

Acknowledgements

This project is developed with funding from the MaterialDigital support program (13XP5119F).