PORTLANEDocs

Developer setup

portlane-pipelines repo, flow authoring, and dev/prod deployment model

Portlane separates platform (engines, Control Tower, Postgres) from your integration code (portlane-pipelines). You version flows in git; CI deploys transforms and wiring to each on-prem instance.

Two repositories

RepositoryWhat you edit
Portlane bundleInstalled once per environment — Docker engines, ops config
portlane-pipelinesFlow identity cards (flows/*.flow.yaml), Python transforms, tests, CI

After installation, the bundle includes a starter portlane-pipelines/ folder. Connect it to your own Git remote and treat it as your long-lived integration repo.

Authoring a flow

  1. Declare a flow card — ingress, routing, target, transform name (flows/<id>.flow.yaml).
  2. Implement a pure Python transform(canonical, config) -> dict, no network I/O in transform code.
  3. Register an entry-point in pyproject.toml and add a golden test.
  4. Run portlane-flow validate locally.

Business secrets (target URLs, API tokens) use {{ env.VAR }} placeholders in the card. They are never committed in YAML.

From code to a running flow

Two things travel together on every deploy: the transform code (baked into a container image) and the flow wiring (synced into the platform's routing configuration). Neither is ever loaded at runtime from your git repo — the running engine only ever executes code from its own image and reads its configuration from the platform database.

git push (flow card + Python transform)


   CI: validate + build worker image ──► push to registry (immutable tag)


   self-hosted runner on your server: pull image, restart worker, health-check


   same run: publish flow card ──► platform routing configuration


   worker picks up matching messages, resolves the transform by name,
   applies it, and hands the result to egress

Because the build step and the publish step run in the same deploy, the code and its wiring can never drift apart — a flow that references a transform name always ships with a worker image that contains it.

Dev and prod deployment

BranchTypical target
devDevelopment Portlane instance
mainProduction instance

Everything that runs on a Portlane server is delivered as a container image pulled from a private registry (GHCR). Deployment is pull-based:

  1. On push, CI validates your flow cards, builds a worker image with your transforms baked in, and pushes it to GHCR with an immutable per-commit tag.
  2. A self-hosted runner on each server (one for dev, one for prod) pulls that image, restarts only the worker, and health-checks it.
  3. The runner then runs portlane-flow publish against the local Control Tower, so flow wiring and worker code always move together.

Because servers only pull, no inbound access is ever needed — no SSH from CI, no open ports. Outbound HTTPS to GitHub and GHCR is the entire network requirement. Platform engine images follow the same model: signed, pinned to exact versions, and only updated when you install a new release bundle. Rolling back means redeploying a previous per-commit image tag.

Where secrets live (no GitHub sync)

Because the runner executes on the same machine as Portlane, publish secrets stay on the instance:

FilePurpose
deploy/publish.envResolves {{ env.* }} in flow cards (target URLs, tokens)
deploy/.envOperator credentials, GHCR pull, database passwords

When you change a target URL or rotate a token, edit deploy/publish.env on that server. The next CI deploy picks up the new values automatically — you do not duplicate secrets into GitHub.

GitHub only needs an environment variable per instance, e.g. PORTLANE_DEPLOY_PATH=/opt/portlane.

Server and hosting

For recommended OS (Ubuntu 24.04 LTS), Hetzner Cloud instance types (CPX32–CPX52), and indicative monthly costs (dev vs prod), see Hosting and sizing.

For installing the core bundle from the client portal (before /docs is available), see Instance bootstrap.

What stays in the customer bundle

Detailed runbooks (runner install, workflow YAML, dry-run checks) ship in the on-prem documentation served from your Control Tower (/docsCI et déploiement des flux). This page is the public overview.

Next steps

For a walkthrough on your systems, book a demo.

On this page