Minimal CI project using gitlab pipeline

Updated: 2025-06-15

In this exercise you will:

• deploy a python app to Gitlab,
• run tests on it, and if they succeed,
• deploy to a server in DigitalOcean.

This project is based on https://gitlab.com/nanuchi/gitlab-cicd-crash-course

Nana's project is a little too much for a 50 minutes lesson. In this minimalist project, just create - python server - python unit test

Then create a pipeline (defined by a .gitlab-ci.yml file) that will run the unit-tests, and if they pass, deploy to your internet server

Step 0: Preparations to be done BEFORE arriving to the lesson

1. create account in gitlab
   1.1 fork https://gitlab.com/EladKal6/minimalci-2025 . You will be working in your forked repository for this exercise.

   What does it mean to "fork" a repository?
   Forking a repository means creating your own personal copy of someone else's project.
   - You can modify it without affecting the original
   - Later, you can propose your changes via a merge request
   Think of it as cloning a project into your own workspace so you can experiment or contribute.

You cannot fork directly from the command line, Click the "Fork" button at the top right of the https://gitlab.com/EladKal6/minimalci-2025 page

Step 0.5: Making your Repo CI-CD safe

Protecting the main branch

To ensure only CI-green code reaches main, follow these steps in GitLab:

1. Protect the branch
2. Go to Settings > Repository > Protected branches
3. Under Protect a branch, select main
4. Set Allowed to push to No one (or only Maintainers)
5. Set Allowed to merge to Developers + Maintainers
6. Click Protect

7. 

8. Require passing pipelines

9. Go to Settings > General > Merge requests
10. Enable Pipelines must succeed
11. 

Now: - Direct pushes to main are blocked
- Merge requests into main only complete if the pipeline passes

Step 1: Create the python http server

1. Clone your forked repo to your computer.

2. IMPORTANT: Create a new branch and work there, when you are done, merge it back to main (You cannot push directly to main anymore remember?)

3. Open a terminal (CMD on Windows, Terminal on macOS/Linux) inside the project folder.

4. Create a virtual environment: python -m venv venv

This creates a virtual environment — a clean place to install packages just for this project (so nothing conflicts with other Python projects).

1. Activate it and install Flask:

Windows (CMD) (run these two commands one after the other): To enter the virtual enviorment you created (you will only work from here) venv\Scripts\activate To install flask on it pip install flask

macOS/Linux: To enter the virtual enviorment you created (you will only work from here) source venv/bin/activate To install flask on it pip install flask

You’ll know it worked if you see (venv) at the beginning of the command line.

1. Run the server:

cd src (src is a folder you forked on your computer remember?)

python main.py

1. Check that it works: Open http://localhost:5000 in your browser.

Step 2: Write some tests

1. Install pytest: pip install pytest

2. Create your test file at (you might need to add some functionalities to main.py to test): src/test_basic_server.py

3. Run tests with the command (in the venv of course): pytest src

NOTE: There are two styles of testing Flask servers: If the test code uses Flask's test client, then the server does not need to run. Otherwise, run the server in one terminal, and the test in a second terminal.

1. Using Flask’s test client (recommended for most unit tests)

2. No need to run the server separately.

3. Best when you want to test your Flask routes internally, quickly and in isolation.

Example:

from src.main import app

def test_homepage():
    with app.test_client() as client:
        response = client.get('/')
        assert response.status_code == 200
        assert b"Hello" in response.data

1. Using real HTTP requests (e.g. with requests.get)

2. Requires the server to be running in a separate terminal.

3. Use this when testing full integration or simulating real client behavior.

Example:

import requests

def test_homepage_live():
    response = requests.get('http://localhost:5000/')
    assert response.status_code == 200
    assert "Hello" in response.text

When to use which: - Use test client for fast, isolated testing of logic and endpoints. - Use real HTTP requests for integration tests to ensure the live server behaves correctly in practice.

Step 3: commit + push

Add a requirements.txt file and populate it. (What "pip installs" did you make for this proj to run?)

Edit .gitlab-ci.yml in the root folder, which is not only a non working skeleton. Add to it the statements to run the tests. (This implementation will change based on what method you used to run the tests)

Repeat this step until the 'git push' to your branch trigger the full flow of "build - test" and it completes with success

After a push, you will be able to see the tests are running in gitlab.

This is what a successful push looks like.

NOTE: You don't need to do any more work to activate your tests other than add, commit and push. The tests will run automatically when you do that thanks to .gitlab-ci.yml

NOTE 2: MAKE SURE TO NOT ADD THE (VENV) folder to the commit (you can put it in the .gitignore for example)

NOTE 3: It doesn't matter if you do git commands from the (venv) or not, the (venv) only controls what python version is used.

NOTE 4: This is where you will spend most of the time!

WARNING: If the stage passed, is it because ALL tests passed, or NONE of them was run? How can you verify?

debug tip: in the gitlab view of the repo, find the pipeline triggered by the commit and click it. You will see the log messages of the whole run

NOTE: Even gitlab runs your project through docker!

Step 4: Add a hook (unrelated to what we have done so far)

Why?
This hook runs automatically before every commit and stops you from accidentally committing any file that contains the string “Noam.” It’s a simple safety check to enforce project naming rules locally.

1. Create the hook file
   In your project folder, open (or create)
   .git/hooks/pre-commit

2. Paste this script into pre-commit: ```sh #!/bin/sh # Block any staged file containing "Noam"

for file in $(git diff --cached --name-only); do [ -f "$file" ] || continue if grep -q "Noam" "$file"; then echo "Commit aborted: '$file' contains the forbidden string 'Noam'." exit 1 fi done

exit 0 ```

This is what will happen when you try and commit this file:

The hook is local to your computer. It is not seen by gitlab (or anyone outside your computer)

Step 5: Branch passes the pipeline successfully! Time to merge with main!

Request for your branch to merge with main.

1. On GitLab, click Create merge request for your branch.
   
2. In the MR form, check Delete source branch when merge request is accepted.
3. also check Squash commits to combine all your work into one clean commit.
4. Click Submit merge request and then Merge once the CI pipeline passes. Make sure you are merging the correct branches!!

You should see your request passed the pipeline!

Accept the request you made

Step 6 (BONUS): Deploy to your server (In DigitalOcean)

1. signup and create Digital Ocean VM: https://www.digitalocean.com/

New to DigitalOcean? See an explanation how to create a VM (called droplet) in the annex below

To deploy:

The gitlab pipeline needs to connect using SSH to your server.

Important: The SSH connection to your server will originate from the GitLab CI/CD runner, not your local computer.

See below for instructions.

You need to update the .gitlab-ci.yml

Create gitlab variable that contains your SSH key

This is the key you use to connect to your target VM. You must have it by now and the VM must allow it.

In gitlab web: Settings - CI/CD - Variables

Add Variable; choose

type == File
key == DEPLOY_SSH_KEY
value == the actual private ssh key 

The SSH key will look like:

-----BEGIN OPENSSH PRIVATE KEY-----
b3Bl**************
**** example *****
******************
-----END OPENSSH PRIVATE KEY-----

LEAVE AN EMPTY LINE BELOW THE 'END OPENSSH'

Leave all default values unchanged.

Assume the target server is already running

TIP: while developing the deploy stage, you can skip the test stage by adding when: manual key to the test stage. Remember to remove it!

TIP: I found it is easier to write all the installation commands in a file "install-server.sh", debug it locally, then upload it from the runner to the deployment machine, and run it there (use scp and ssh).

TIP: While debugging shell script, add #!/bin/bash -eux. It will bring you good luck. Trust me.

TIP: During debugging of the ssh/scp commands, add -v to the arguments.

The updated .gitlab-ci.yml needs to have a 'deploy' stage of the pipeline , where we: 1. connect using ssh to the target VM 2. stop the current server if it runs 3. install new venv and activate it 4. install packages 5. start the server as a daemon

The updated .gitlab-ci.yml took a while to get working in my case, don't give up easily and don't be shy from looking for help online/through LLMs!

In a nutshell: how do you specify the remote user name/ host name/ deploy path ? All must be env variables stored in gitlab.

Step 7: Enjoy the full flow!

Change "hello, world" to "hello cicd" in the server's endpoint.

commit + push. Wait a few minutes.

Refresh the browser window. You should see the updated string.

This means the full pipeline worked!

(Commit-->Push-->Pipeline Testing Passed-->Changes were uploaded to production --> The users can see the changes!)

Summary

In this lab, you created a small server and tests for it. You created a pipeline to test and deploy the code.

The difference between this and real life

• many more tests
• do not use hardcoded values
• add code to launch the tested server (and shut it down before!)
• add code to configure and create the target VM (e.g. Terraform)
• add stage to test the deployed server

... and add 3 ideas of your own

Have fun and break things!

Lastly, don't forget to clean your DigitalOcean account

Retrospection

What were the most time consuming parts? the most confusing?

Will it be different the next time you do it?

Annex: Creating a droplet (a VirtualMachine) in DigitalOcean (BONUS)

In this annex you will quickly setup a new VM.

first signup at https://cloud.digitalocean.com/login

follow the instructions in https://www.youtube.com/watch?v=g1-nQ9pvbxc

On 2025-06-15, I chose: - Region = New york - DataCenter = default - image = default (Ubuntu 24.10) - Droplet type = Basic - CPU options = Regular, $6/month - Authentication method = default (SSH key)
--> "Add SSH key" and follow the instructions and write down the name of the SSH key as it appears in DigitalOcean” - Tags = "Noam course"

Finally, press "CREATE".

After a minute, the droplet is ready with its public IP

Warning: You will still be billed for a turned off Droplet. To end billing, destroy the Droplet instead

Once the droplet is up, connect from your terminal e.g. ssh root@104.236.119.165

The private key is stored at ~/.ssh/your-key-name
This is the file you need to copy into gitlab and save as DEPLOY_SSH_KEY key