Data flows
The Portlane flow model — ingress, routing, transform, and egress
A flow describes the journey of a message from a source system to a target system. Each flow is authored as code — a small declarative identity card plus a pure Python transform — versioned in your own git repository.
The four steps
| Step | Engine | Purpose |
|---|---|---|
| Ingress | listener (webhooks) or pooling (scheduled pull) | Receive or fetch data from a source system |
| Routing | broker | Normalize the event and route it toward the right flow |
| Transform | worker | Apply your business logic — mapping, enrichment, business rules |
| Egress | distribute | Push the transformed message to the target system |
Authoring a flow
You declare a flow with two artifacts, reviewed like any code change:
- An identity card (YAML) — source, routing, target, and which transform to apply.
- A pure Python transform — no network calls, no database access, just data in and data out.
Publishing a flow synchronizes the card into the platform's routing configuration; the transform ships inside a versioned container image. Both move together on every deploy.
Typical patterns
Extract–transform–load (ETL) — Pull from a source system on a schedule, normalize in a transform, push to a warehouse or API.
Event-driven sync — React to a webhook, apply business rules, deliver to one or more downstream systems.
Versioning and rollback
Flow cards and transforms are versioned in git and deployed through your CI pipeline. Rolling back means redeploying a previous, already-tested image — no manual reconfiguration.
What this doc does not cover
This page describes the generic flow model. The flows actually configured in your instance are generated from your environment and shown live in the Control Tower console, not written here.