Simplicité CI/CD Demo with gitlab

Project showing an example of a Simplicité project leveraging GitLab's CI/CD tooling.

Quick Access : dev | staging | prod

General requirements

• gitlab.com account
• Google Cloud Platform (GCP) account, with one project with following activations (https://docs.gitlab.com/ee/topics/autodevops/quick_start_guide.html#enable-auto-devops-optional)
  • billing account
  • Google Kubernetes Engine API
  • Cloud Resource Manager API
  • Cloud Billing API

Although a GCP Account is not required to build and run parts of the pipeline, it's required for some fundamental steps, as running the UI tests and the unit tests.

1) Developement (manual)

This steps happens outside of GitLab. Simply run a Simplicité instance and design your app, single or multi-module, as studied in the Simplicité Tutorials.

2) Commit (manual)

On a machine capable of executing bash scripts, with the jq and curl CLI utilities properly installed, the tools/export.sh script is executed in the ./modules/ directory, based on what is configured in the simap.json file.

{
    "name": "AppName",
    "version": "1",
    "instances": {
        "dev": "https://devinstance.mydomain.com"
    },
    "modules": [
        { "name": "Module1" },
        { "name": "Module2" }
    ]
}

This step will export your modules and generate a Dockerfile and an importspec. Some extra information can be found about this step in the Simplicité Forum

Important : The EXPORT_MODULE_ARTIFACTS must be set to yes on the dev instance (the module's pom.xml are needed)

Important : The order of the modules in this file determines how they are imported. This should be carefully thought if there are dependencies

3) Build

In this step, we leverage the AutoDeploy feature of GitLab, which detects the Dockerfile at the root of the repository, to build a new image and push it to the project's registry. This image will be deployed on our K8s in later steps.

Changelog:

• Simplicité has no Buildpack published, so the only AutoBuild option is to use a Dockerfile
• A custom respository structure is used for this project instead of Simplicité's internal module repository feature
  • because:
    • GitLab expects some files & directories containing configuration (Dockerfile, .gitlab/auto-deploy-values.yaml, .gitlab-ci.yml), and Simplicité's internal repos don't offer that much flexibility
  • as a consequence:
    • a custom bash script is used to update module's configuration, by exporting it from some dev instance with an export.sh script configured in simap.json
• Managed gitlab.com provides shared runners => no need to install gitlab Runner. Use a Docker In Docker runner to build docker image and push to registry

4) Code analysis

4.1 Code Quality, Container Scanning & Secret detection

Those jobs are part of GitLab's AutoDevops feature and are under review for Simplicité's context.

4.2 SonarCloud

A SonarCloud account is bound to GitLab. The project has been configured as a maven project on SonarCloud. The pom.xml at the root of the repo indicates the URL of the sonar instance and the paths to the modules, so sonar cana analyse the code.

4.3 Unit testing with AutoTest

Problem: Simplicité's JUnit tests can't be run outside of Simplicité. Makes standard testing kind of useless Recommended method: deploy a test instance and call io service : https://community.simplicite.io/t/unit-test-io-triggering/3893

Changelog:

• add EXPORT_MODULE_ARTIFACTS => yes on instance
• autotest fails becaus "unable to select a buildpack":
  • suspect absence of pom.xml at root (https://devcenter.heroku.com/articles/java-support)
  • manually adding a parent pom.xml (#1)
    • fails because default module's pom.xml has no <simplicite.url>. Inherit from parent pom fails, so it's hardcoded => automatically modified by export.sh
    • fails because "Fatal error compiling: invalid target release: 11". Suspect Heroku defaut java version. Specify Java 11 (https://devcenter.heroku.com/articles/java-support#supported-java-versions#specifying-a-java-version). (#3)

5) Deploy to staging environement

This step leverages GitLab's AutoDeploy to deploy the image on the K8s cluster configured on GCP.

Changelog - added a standard docker healthcheck to the Dockerfile, but the next stage of the pipeline is still hit before the app is ready (#7)

6) Testing

6.1 UI Tests

In this step, a job runs a Docker image with Maven and a headless chrome browser. A test dataset is imported (via curl) on the testing environement and the ./uitest/ Java project (based on Selenium / Selenide) is then executed, taking control of the browser and runing the test suite on the staging instance. A report is generated with debugging information: logs, screenshots, etc.

6.2 Unit tests

[ Not yet implemented ]

https://docs.simplicite.io/documentation/01-core/unit-testing.md#io

Changelog

• added a uitest stage & adapted HELM config to disable SSL on K8 ( (#4) )
• added dataset import automation (temporary)

7) Deploy to production environement

See step #5

Additionnaly, this step requires a manual deployement (simple click), so the app can be verified by a human before going to prod.