Skip to content
Crow CI
Search limited to v5.6. Switch to the latest version for up-to-date results.

Logging

Logging configuration variables.

LOG_LEVEL

Section titled “LOG_LEVEL”
  • Name: CROW_LOG_LEVEL
  • Description: Logging level. Possible values are trace, debug, info, warn, error, fatal, panic, and disabled.
  • Default: none

LOG_FILE

Section titled “LOG_FILE”
  • Name: CROW_LOG_FILE
  • Description: Output destination for logs. stdout and stderr can be used as special keywords.
  • Default: stderr

LOG_STORE

Section titled “LOG_STORE”
  • Name: CROW_LOG_STORE
  • Description: Log store to use (database, file, or s3).
  • Default: database

Choosing a pipeline log store

Section titled “Choosing a pipeline log store”

Pipeline logs can grow quickly, especially with many active repositories. The three backends have different trade-offs:

database (default)

Section titled “database (default)”

Stores log entries as rows in the log_entries table.

  • Pros: Simplest setup — no extra infrastructure needed. Supports live log streaming in HA mode (the distributed logger polls the database for new entries).
  • Cons: Database size grows proportionally with log volume. On busy instances this can significantly increase backup times, disk usage, and query latency for unrelated tables. Databases like SQLite are particularly affected since the entire file grows.
  • Best for: Small to medium instances, or when database size is not a concern.

file

Section titled “file”

Stores one NDJSON file per pipeline step on the server’s local filesystem.

  • Pros: Moves log data out of the database entirely. Fast local I/O with append-only writes. Easy to inspect logs manually.
  • Cons: Logs are tied to a single server’s disk — not suitable for HA setups with multiple servers unless using a shared filesystem (e.g. NFS). You are responsible for managing disk space and backups.
  • Best for: Single-server deployments where you want to keep the database lean.

s3

Section titled “s3”

Stores logs in any S3-compatible object storage (AWS S3, MinIO, Ceph, DigitalOcean Spaces, etc.).

  • Pros: Decouples log storage from both the database and local disk. Scales to very large volumes without affecting database performance. Works well for HA setups since all servers can access the same bucket. Object storage is typically cheaper than database storage at scale.
  • Cons: Higher latency per read compared to local disk or database — each log view requires listing and fetching objects from S3. Writes create one small object per batch (~1 second of log output), so a long-running step will produce many objects. Not suitable for the distributed logger’s live streaming poll (live streaming still uses the in-memory multiplexer regardless of log store).
  • Best for: Large or multi-server deployments where database size is a concern and you have S3-compatible storage available.

LOG_STORE_FILE_PATH

Section titled “LOG_STORE_FILE_PATH”
  • Name: CROW_LOG_STORE_FILE_PATH
  • Description: Directory used for file based log storage.
  • Default: none

LOG_STORE_S3_ENDPOINT

Section titled “LOG_STORE_S3_ENDPOINT”
  • Name: CROW_LOG_STORE_S3_ENDPOINT
  • Description: S3-compatible endpoint for log storage (e.g. s3.amazonaws.com or minio.example.com:9000).
  • Default: none

LOG_STORE_S3_BUCKET

Section titled “LOG_STORE_S3_BUCKET”
  • Name: CROW_LOG_STORE_S3_BUCKET
  • Description: S3 bucket name for log storage.
  • Default: none

LOG_STORE_S3_ACCESS_KEY

Section titled “LOG_STORE_S3_ACCESS_KEY”
  • Name: CROW_LOG_STORE_S3_ACCESS_KEY
  • Description: S3 access key for log storage.
  • Default: none

LOG_STORE_S3_SECRET_KEY

Section titled “LOG_STORE_S3_SECRET_KEY”
  • Name: CROW_LOG_STORE_S3_SECRET_KEY
  • Description: S3 secret key for log storage.
  • Default: none

LOG_STORE_S3_SSL

Section titled “LOG_STORE_S3_SSL”
  • Name: CROW_LOG_STORE_S3_SSL
  • Description: Use SSL for S3 log storage connection.
  • Default: true

LOG_STORE_S3_PATH_STYLE

Section titled “LOG_STORE_S3_PATH_STYLE”
  • Name: CROW_LOG_STORE_S3_PATH_STYLE
  • Description: Use path-style addressing for S3 log storage (required for MinIO and some S3-compatible providers).
  • Default: false

LOG_STORE_S3_PREFIX

Section titled “LOG_STORE_S3_PREFIX”
  • Name: CROW_LOG_STORE_S3_PREFIX
  • Description: Key prefix for log objects in the S3 bucket.
  • Default: logs

Log Rotation (Alpine only)

Section titled “Log Rotation (Alpine only)”

LOGROTATE_ENABLED

Section titled “LOGROTATE_ENABLED”
  • Name: CROW_LOGROTATE_ENABLED
  • Description: Enable automatic log rotation (Alpine images only). Set to auto to enable when CROW_LOG_FILE points to a file, true to always enable, or false to disable.
  • Default: auto

LOGROTATE_SCHEDULE

Section titled “LOGROTATE_SCHEDULE”
  • Name: CROW_LOGROTATE_SCHEDULE
  • Description: Cron schedule for log rotation (Alpine images only). Uses standard cron syntax (minute hour day month weekday).
  • Default: 0 0 * * * (daily at midnight)

LOGROTATE_RETAIN_DAYS

Section titled “LOGROTATE_RETAIN_DAYS”
  • Name: CROW_LOGROTATE_RETAIN_DAYS
  • Description: Number of days to retain rotated logs before deletion (Alpine images only).
  • Default: 7