Developer setup
portlane-pipelines repo, flow authoring, and dev/prod deployment model
Portlane separates platform (engines, Control Tower, Postgres) from your integration code (portlane-pipelines). You version flows in git; CI deploys transforms and wiring to each on-prem instance.
Two repositories
| Repository | What you edit |
|---|---|
| Portlane bundle | Installed once per environment — Docker engines, ops config |
| portlane-pipelines | Flow identity cards (flows/*.flow.yaml), Python transforms, tests, CI |
After installation, the bundle includes a starter portlane-pipelines/ folder. Connect it to your own Git remote and treat it as your long-lived integration repo.
Authoring a flow
- Declare a flow card — ingress, routing, target, transform name (
flows/<id>.flow.yaml). - Implement a pure Python transform —
(canonical, config) -> dict, no network I/O in transform code. - Register an entry-point in
pyproject.tomland add a golden test. - Run
portlane-flow validatelocally.
Business secrets (target URLs, API tokens) use {{ env.VAR }} placeholders in the card. They are never committed in YAML.
From code to a running flow
Two things travel together on every deploy: the transform code (baked into a container image) and the flow wiring (synced into the platform's routing configuration). Neither is ever loaded at runtime from your git repo — the running engine only ever executes code from its own image and reads its configuration from the platform database.
git push (flow card + Python transform)
│
▼
CI: validate + build worker image ──► push to registry (immutable tag)
│
▼
self-hosted runner on your server: pull image, restart worker, health-check
│
▼
same run: publish flow card ──► platform routing configuration
│
▼
worker picks up matching messages, resolves the transform by name,
applies it, and hands the result to egressBecause the build step and the publish step run in the same deploy, the code and its wiring can never drift apart — a flow that references a transform name always ships with a worker image that contains it.
Dev and prod deployment
| Branch | Typical target |
|---|---|
dev | Development Portlane instance |
main | Production instance |
Everything that runs on a Portlane server is delivered as a container image pulled from a private registry (GHCR). Deployment is pull-based:
- On push, CI validates your flow cards, builds a worker image with your transforms baked in, and pushes it to GHCR with an immutable per-commit tag.
- A self-hosted runner on each server (one for dev, one for prod) pulls that image, restarts only the worker, and health-checks it.
- The runner then runs
portlane-flow publishagainst the local Control Tower, so flow wiring and worker code always move together.
Because servers only pull, no inbound access is ever needed — no SSH from CI, no open ports. Outbound HTTPS to GitHub and GHCR is the entire network requirement. Platform engine images follow the same model: signed, pinned to exact versions, and only updated when you install a new release bundle. Rolling back means redeploying a previous per-commit image tag.
Where secrets live (no GitHub sync)
Because the runner executes on the same machine as Portlane, publish secrets stay on the instance:
| File | Purpose |
|---|---|
deploy/publish.env | Resolves {{ env.* }} in flow cards (target URLs, tokens) |
deploy/.env | Operator credentials, GHCR pull, database passwords |
When you change a target URL or rotate a token, edit deploy/publish.env on that server. The next CI deploy picks up the new values automatically — you do not duplicate secrets into GitHub.
GitHub only needs an environment variable per instance, e.g. PORTLANE_DEPLOY_PATH=/opt/portlane.
Server and hosting
For recommended OS (Ubuntu 24.04 LTS), Hetzner Cloud instance types (CPX32–CPX52), and indicative monthly costs (dev vs prod), see Hosting and sizing.
For installing the core bundle from the client portal (before /docs is available), see Instance bootstrap.
What stays in the customer bundle
Detailed runbooks (runner install, workflow YAML, dry-run checks) ship in the on-prem documentation served from your Control Tower (/docs → CI et déploiement des flux). This page is the public overview.
Next steps
Hosting and sizing
OS, Hetzner, costs
ReadData flows
Triggers, transforms, egress
ReadArchitecture
Control plane vs data plane
ReadGetting started
Prerequisites and first flow
ReadFor a walkthrough on your systems, book a demo.