GitLab CI: Difference between revisions
No edit summary |
No edit summary |
||
Line 1: | Line 1: | ||
= Pipeline Configuration | = CI/CD Pipeline Configuration = | ||
GitLab CI/CD build for a repository are controlled placing a file named '''.gitlab_ci.yml''' in the root folder of the repository. The expected content of the file is specified on GitLab's | GitLab CI/CD build for a repository are controlled placing a file named '''.gitlab_ci.yml''' in the root folder of the repository. The expected content of the file is specified on GitLab's | ||
Line 85: | Line 85: | ||
Note that the '''pull_policy''' option was explicitly set to "never" instead of going with the default "always". This prevents GitLab Runner from searching for the specified Docker image on [https://hub.docker.com Docker Hub] and directs the query to the local docker engine! | Note that the '''pull_policy''' option was explicitly set to "never" instead of going with the default "always". This prevents GitLab Runner from searching for the specified Docker image on [https://hub.docker.com Docker Hub] and directs the query to the local docker engine! | ||
== Troubleshooting == | |||
* Some CI/CD pipelines will fail due to the presence of ''clear_console'' in '''/home/gitlab-runner/.bash_logout'''. Delete or comment out the contents of that file. | |||
= How | = How to create a Product Webpage = | ||
<br /> | |||
= How to create Product Releases with Binaries = | |||
The best way to add binaries to project releases on GitLab, is to create a job which uploads the release assets as a [https://docs.gitlab.com/ee/user/packages/generic_packages generic package], then creates a [https://docs.gitlab.com/ee/user/project/releases/index.html release] by referring to the uploaded assets. | |||
* In the project root folder, add a ''.gitlab-ci.yml'' file with the contents based on the example below. Note that the upload script makes use of **CI_JOB_TOKEN**, which GitLab passes to the build runner for use in build scripts that need to authenticate themselves when sending files to GitLab.<br/> | |||
<pre class="code"> | |||
stages: | |||
- upload | |||
- release | |||
variables: | |||
PRODUCT_NAME: "MyProject" | |||
ASSET_FILE_NAME: "${PRODUCT_NAME}-${CI_COMMIT_TAG}.dmg" | |||
ASSET_FILE_SOURCE_PATH: "${HOME}/binary_products/${ASSET_FILE_NAME}" | |||
PACKAGE_REGISTRY_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${PRODUCT_NAME}/${CI_COMMIT_TAG}/" | |||
upload: | |||
stage: upload | |||
rules: | |||
- if: $CI_COMMIT_TAG | |||
script: | |||
- | | |||
curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file ${ASSET_FILE_SOURCE_PATH} ${PACKAGE_REGISTRY_URL}/${ASSET_FILE_NAME} | |||
release: | |||
stage: release | |||
image: registry.gitlab.com/gitlab-org/release-cli:v0.4.0 | |||
rules: | |||
- if: $CI_COMMIT_TAG | |||
script: | |||
- | | |||
/usr/local/bin/release-cli create --name "Release $CI_COMMIT_TAG" --tag-name $CI_COMMIT_TAG \ | |||
--assets-link "{\"name\":\"${ASSET_FILE_NAME}\",\"url\":\"${PACKAGE_REGISTRY_URL}${ASSET_FILE_NAME}\"}" | |||
</pre> | |||
where '''release-cli''' is a tool (written in Go) that instructs GitLab to create a release for the given Git tag and with the given binary assets. | |||
* In VirtualBox/Ubuntu register a GitLab Runner instance for the project. This runner will not build the product. It will only be used to upload the binary assets. | |||
* In VirtualBox/Ubuntu install ''release-cli'' by downloading from [https://gitlab.com/gitlab-org/release-cli here] and running | |||
<pre class="terminal"> | |||
sudo apt install golang | |||
cd release-cli-v0.8.0 | |||
go build cmd/release-cli/main.go | |||
cp main /usr/local/bin/release-cli | |||
</pre> | |||
* Create the binary to be released and place it in the folder expected by the upload script. You may want to set up a VirtualBox shared folder. | |||
* In GitLab, create a tag for the version to be released. This will trigger the execution of the upload job in GitLab Runner. | |||
* To refer the user to a download page containing all releases, use the following URL: | |||
<pre class="code"> | |||
https://gitlab.com/<project group name>/<project name>/-/releases | |||
</pre> |
Revision as of 2021-06-23T19:16:20
CI/CD Pipeline Configuration
GitLab CI/CD build for a repository are controlled placing a file named .gitlab_ci.yml in the root folder of the repository. The expected content of the file is specified on GitLab's gitlab_ci.yml Documentation page.
GitLab Runner
Installing
Follow the instructions on this GitLab documentation page, summarized below, where you set up a user account for GitLab Runner and make it launch automatically when your system starts.
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64" sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner sudo gitlab-runner start
You can check the status of the service by typing
sudo gitlab-runner status
Using with Docker
If GitLab Runner is to build your software inside of Docker containers, it needs to able to issue Docker commands. Therefore, make sure Docker is installed and the user gitlab-runner is a member of the docker group:
sudo usermod -aG docker gitlab-runner
which, in virtualized Linux environments, may require a reboot to take effect.
The integration of GitLab Runner with Docker goes deeper than just instantiating a container:
Before launching the build scripts from .gitlab_ci.yml, GitLab Runner will clone the repository
into a build folder inside the container! The default path is /build/ followed by the project path
on the GitLab server. The build scripts simply leave it to GitLab Runner to provide them with the
correct commit from the correct branch.
Registering the Runner
After GitLab Runner is installed, you will need to register the runner with the GitLab code repository. Typing
sudo gitlab-runner register
launches the interactive registration, which will prompt you for each of the configuration parameters. You can also supply some or all of the parameters directly in the command:
sudo gitlab-runner register \ --url "https://" \ --registration-token <secret token> \ --description "docker-swift:5.4" \ --executor "docker" \ --docker-image swift:5.4
where <secret token> is the token obtained from GitLab project settings → CI/CD → Runners → Specific Runners.
Registration finishes by generating the /etc/gitlab-runner/config.toml file similar to this:
concurrent = 1 check_interval = 0 [session_server] session_timeout = 1800 [[runners]] name = "ubuntu-at-home" url = "https://gitlab.com/" token = <secret token> executor = "docker" [runners.custom_build_dir] [runners.cache] [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure] [runners.docker] tls_verify = false image = "keytree" privileged = false disable_entrypoint_overwrite = false pull_policy = "never" oom_kill_disable = false disable_cache = false volumes = ["/cache"] shm_size = 0
after which, the gitlab-runner service will notice that the configuration file has changed and applies the new configuration.
Note that the pull_policy option was explicitly set to "never" instead of going with the default "always". This prevents GitLab Runner from searching for the specified Docker image on Docker Hub and directs the query to the local docker engine!
Troubleshooting
- Some CI/CD pipelines will fail due to the presence of clear_console in /home/gitlab-runner/.bash_logout. Delete or comment out the contents of that file.
How to create a Product Webpage
How to create Product Releases with Binaries
The best way to add binaries to project releases on GitLab, is to create a job which uploads the release assets as a generic package, then creates a release by referring to the uploaded assets.
- In the project root folder, add a .gitlab-ci.yml file with the contents based on the example below. Note that the upload script makes use of **CI_JOB_TOKEN**, which GitLab passes to the build runner for use in build scripts that need to authenticate themselves when sending files to GitLab.
stages: - upload - release variables: PRODUCT_NAME: "MyProject" ASSET_FILE_NAME: "${PRODUCT_NAME}-${CI_COMMIT_TAG}.dmg" ASSET_FILE_SOURCE_PATH: "${HOME}/binary_products/${ASSET_FILE_NAME}" PACKAGE_REGISTRY_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${PRODUCT_NAME}/${CI_COMMIT_TAG}/" upload: stage: upload rules: - if: $CI_COMMIT_TAG script: - | curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file ${ASSET_FILE_SOURCE_PATH} ${PACKAGE_REGISTRY_URL}/${ASSET_FILE_NAME} release: stage: release image: registry.gitlab.com/gitlab-org/release-cli:v0.4.0 rules: - if: $CI_COMMIT_TAG script: - | /usr/local/bin/release-cli create --name "Release $CI_COMMIT_TAG" --tag-name $CI_COMMIT_TAG \ --assets-link "{\"name\":\"${ASSET_FILE_NAME}\",\"url\":\"${PACKAGE_REGISTRY_URL}${ASSET_FILE_NAME}\"}"
where release-cli is a tool (written in Go) that instructs GitLab to create a release for the given Git tag and with the given binary assets.
- In VirtualBox/Ubuntu register a GitLab Runner instance for the project. This runner will not build the product. It will only be used to upload the binary assets.
- In VirtualBox/Ubuntu install release-cli by downloading from here and running
sudo apt install golang cd release-cli-v0.8.0 go build cmd/release-cli/main.go cp main /usr/local/bin/release-cli
- Create the binary to be released and place it in the folder expected by the upload script. You may want to set up a VirtualBox shared folder.
- In GitLab, create a tag for the version to be released. This will trigger the execution of the upload job in GitLab Runner.
- To refer the user to a download page containing all releases, use the following URL:
https://gitlab.com/<project group name>/<project name>/-/releases