CI gating

Block PRs that blow your LLM budget.

Calcis runs in your CI as a GitHub Action. It tokenises every prompt change, predicts the cost, and fails the build if the predicted spend crosses your threshold.

Add the ActionSee the API

How it works

Three steps. Sixty seconds.

  1. 1

    Drop the action into your workflow

    One uses: line in any pull_request workflow. Pin to v1 so a future major can't silently change the gate behaviour.

  2. 2

    Calcis tokenises every prompt change in the PR diff

    We diff the prompt files in the PR against the base branch, run the same tokeniser the provider uses (tiktoken, Anthropic counter, sentencepiece), and predict per-request cost on the configured model.

  3. 3

    Block, comment, or both

    Above the configured fail-above-usd threshold the action exits non-zero so the required check turns red. A PR comment lands either way with the predicted cost, the model, and the per-route delta against main.

What you can gate on

Four signals. Pick the ones that matter.

Per-route cost

Set a per-request cost ceiling for each prompt template the action discovers. Useful for high-volume routes where a 2x regression compounds.

Per-PR delta

Fail when the PR pushes total predicted spend more than N% above the base branch. Catches threshold tweaks (RAG top-k, retrieval rerank, prompt expansion) that quietly multiply context.

Monthly projected spend

Multiply per-request cost by your declared monthly request volume. Fails when the PR pushes the projection past your budget, even if individual requests look cheap.

Model regression

Fail when a PR routes a step from a cheaper model to a more expensive one without an explicit override comment. Catches the Haiku-to-Opus copy-paste before it ships.

Setup in 60 seconds

One YAML file. One secret.

Drop the snippet below into .github/workflows/cost-guardrail.yml. Add a CALCIS_API_KEY repository secret. Done.

.github/workflows/cost-guardrail.ymlv1 · pinned
name: LLM Cost Guardrail
on:
  pull_request:
    types: [opened, synchronize]

jobs:
  guardrail:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: rc397/calcis-action@v1
        with:
          api-key: ${{ secrets.CALCIS_API_KEY }}
          model: claude-sonnet-4-6
          # Fail the build when predicted per-request cost
          # crosses the threshold. Defaults to 0 (warn-only).
          fail-above-usd: 0.05

Get an API key from your dashboard. Required for the action; the dashboard is free to create.

Works with your CI

GitHub-first, but not GitHub-only.

  • GitHub ActionsFirst-class. The rc397/calcis-action repository on the GitHub Marketplace.
  • GitLab CIRun the calcis CLI as a step. Same flags, same exit codes, no extra dependency.
  • CircleCIDrop the calcis CLI into any orb-free job. The CLI is a single npm install away.
  • JenkinsSame CLI, same exit codes. Use the npm-cli plugin or a plain shell stage.
  • Generic CLI exit codesExit 0 when predicted spend is below the threshold, exit 2 when above, exit 1 on a real error. Standard enough to wire into anything that reads $?.

Ship the gate today.

Free plan covers the action. Pro adds the higher-confidence predictor; Max adds the API quota the action needs at scale.

Add the ActionSee the tiers