Development

Required tools

The following tools are required for development:

Once just is installed, you can install adjacent golang dependencies using the following command:

just install-dev-deps

The following just rules are helpful for local development:

just test-server
just test-agent
just test-cli
just test-server-datastore # test datastore / database related code like migrations

just lint
just lint-backend
just lint-frontend

Building images locally

If you want to build images for dev purposes, the following just rules can be used:

# server
env PLATFORMS='linux|amd64;linux|arm64' just image-server-alpine TAG=<tag> REGISTRY_USER='<user>' REGISTRY='<registry>' IMAGE='<image slug>'

# agent
env PLATFORMS='linux/amd64,linux/arm64' just image-agent-alpine <tag> REGISTRY_USER='<user>' REGISTRY='<registry>' IMAGE='<image slug>'

# cli
env PLATFORMS='linux/amd64,linux/arm64' just image-cli-alpine <tag> REGISTRY_USER='<user>' REGISTRY='<registry>' IMAGE='<image slug>'

The PLATFORMS env var must be prepended as it will be used dynamically by other sub-rules to build the proper arch-specific components.

VSCode debug/test configuration

.vscode/launch.json holds a complete configuration for starting all Crow components locally. Use the "Crow CI" configuration and then access localhost:8000 in your browser.

To make this all work, some env vars need to be provided in a local .env file.

export CROW_OPEN=true
export CROW_ADMIN=<your forge username>

export CROW_HOST=http://localhost:8000

# forge config (forgejo) (see /docs/administration/forge/overview for other forges)
export CROW_FORGEJO=true
export CROW_FORGEJO_URL=https://codeberg.org
export CROW_FORGEJO_CLIENT=<id>
export CROW_FORGEJO_SECRET=<secret>

# agent
export CROW_SERVER=localhost:9000
export CROW_AGENT_SECRET=<agent secret>
export CROW_MAX_WORKFLOWS=1

# enable if you want to develop the UI
export CROW_DEV_WWW_PROXY=http://localhost:8010

# if you want to test webhooks with a forge this address needs to be set and accessible from public server
export CROW_EXPERT_WEBHOOK_HOST=

# disable health-checks while debugging (normally not needed while developing)
export CROW_HEALTHCHECK=false

# (optional) when running the vite dev server in a container
# export VITE_DEV_SERVER_HOST=0.0.0.0

export CROW_LOG_LEVEL=debug

Notes:

• Workflows will run on your local machine using the docker backend.
• The forge-specific settings (CROW_FORGEJO_URL, CROW_FORGEJO_CLIENT, CROW_FORGEJO_SECRET) must be set if you want to test against existing repositories of a forge. We recommend to create a new OAuth2 app on the forge and use it solely for local testing purposes. If you use an existing one, the issued tokens will be split across multiple instances and lead to invalid tokens on the respective other one.
• Crow will use SQLite in this scenario, stored at ./crow.sqlite in the workspace root (and gitignore'd by default). Any backend action you do (e.g. adding secret or enabling a repo) will be persisted in this database. It is recommended to keep this SQLite DB at the repo root, gitignore'd.

• CROW_EXPERT_WEBHOOK_HOST sets the URL which incoming webhooks should target, i.e. the local development server. By default this one runs on localhost and is accessible from the public. To make this work, you need to expose the local Crow server to the outside world. To do this, you can either

  • run the local development server on 0.0.0.0 and expose the port directly to the outside world (not recommended without https if you're working with an oauth2 app)

  • run a reverse proxy locally which listens for incoming requests, does SSL termination and forwards the requests to the local development server. This is recommended if you're working with an oauth2 app.

Usually, the easiest way is to not use CROW_EXPERT_WEBHOOK_HOST and work with event: manual instead. This event trigger is not dependent on remote webhooks but is triggered from within the Crow server itself.

Documentation

When working on the documentation, the following commands can be used:

# inits and populates the virtual environment
just docs-venv
just docs-serve

Crow uses the "insiders" build of mkdocs-material, which is only available through a private repository which must be accessed via a token. The Justfile rules automatically accounts for this and will download the free version of mkdocs-material if the token is not available. Most features for development do not rely on the "insiders" build.

Versioning

A new docs version is published to the gh-pages branch via mike for every release and push to main (="dev"). (mike does not allow to use a different branch name than gh-pages as it is heavily focused on GitHub Pages.) From this branch, the assets will be synced to S3.