GitHub CI
Getting started with self-hosted runners for GitHub CI on ExCL systems.
Last updated
Was this helpful?
Getting started with self-hosted runners for GitHub CI on ExCL systems.
Last updated
Was this helpful?
If you don’t want to run the runner as service then you can follow the steps posted at to create a self-hosted runner in ExCL.
If you do want to register the runner as a service, the easiest way is to use systemd user services. To set this up follow the steps below.
Notes:
If you are trying this on a system which doesn’t already have a /scratch
folder the command will fail. Please send an email to to create a folder for local storage.
If you setting up a second runner, the ln
command will fail if the link already exists. Ensure that the link is a valid link pointing to scratch before continuing with these instructions.
The steps are similar to that posted at with some changes. You will need to create one folder per machine and per repo so I recommend the following structure.
~/github-runners/<node>-<repo>
Once you create this directory and enter it, you will then download and configure the runner. The steps are reproduce below, but you should follow the instructions from the “add new self-hosted runner” page after clicking on “New self-hosted runner”.
Apply this patch to modify the directory to use user systemd modules.
Use this command to enable linger for your user.
This allows your user-level systemd services to run when you are not logged into the system and auto-start when the system is rebooted.
Note: Use loginctl disable-linger
to remove linger and ls /var/lib/systemd/linger
to view the users with linger set.
svc.sh
script to install and manage the runner service.Install service
Start service and check status.
Note: The above install adds the service to auto start on reboot. If you want to disable or enable this auto starting of the service run.
or
To stop the service run
To uninstall the service run
Trigger on issue_comment: this is the event that triggers the CI pipeline. The types: [created]
ensures that the pipeline is triggered only when a new comment is made and not when an existing comment is edited.
NOTE: in GitHub Actions PRs are issues, so the
issue_comment
event is used to trigger the pipeline when a PR comment is made.
Verify Actor: and "actor" is any user writing a comment on the PR. This step verifies that the actor is an authorized user to trigger the CI pipeline. The following is an example of how to verify the actor in the workflow yaml file. ACTOR_TOKEN
puts the current "actor" within the delimiter and checks if it is in the list of authorized users. If it is, it triggers the pipeline. If not, it skips all subsequent steps.
Create PR status: this step creates a status check on the PR extracting information from the json information generated in the previous step. This steps allows for seamless integration with the typical checks interface for a PR along with other CI workflow. The status check is created as a "pending" status and the URL is linked to the current pipeline run before the actual tests run.
Run tests: the following steps continue the pipeline tests and they are specific to each workflow reusing these steps.
Report PR status: this step reports the status of the pipeline to the PR. The status is updated to "success" if the tests pass and "failure" if the tests fail. The URL is linked to the current pipeline run to update the PR status created in step 4.
See for more information.
After this patch is applied the svc.sh
script works as documented in . However you don’t need to specify a username since it now defaults to the current user. The commands are reproduced below.
GitHub Actions the use of self-hosted runners for public repos. However, if you want to use an ExCL self-hosted runner for a public repo, you can use the following steps to create a secure CI pipeline that is triggered by an authorized user in a PR comment. This will prevent unauthorized users from running arbitrary code (e.g. attacks) automatically on ExCL systems from any PRs.
We follow the resulting workflow yaml file in the JACC.jl as an example that can be reused across repos.
Select authorized users: those who can trigger the pipeline and store it in a in your repo using the following format: CI_GPU_ACTORS=;user1;user2;user3;
and store another secret TOKENIZER=;
to be used as a delimiter (it can be any character). Users should have a strong password and 2FA enabled.
Request PR info: since the event triggering the pipeline is a issue_comment
the pipeline needs to retrieve information for the current PR. We use the official octokit/request-action
to get the PR information using the GITHUB_TOKEN
available automatically from repo . This is stored in a json format and available for future steps.
NOTE: in GitHub Actions statuses are different from checks, see for a better explanation. The statuses generated by this pipeline get reported and stored in the Actions, and not in the PR checks tab. The important part is that the status from this workflow gets reported to the PR, users can see the status of the pipeline and admins can make these statuses mandatory or optional before merging.