Scheduling & triggers
A pipeline runs either on a schedule or in response to incoming events. This guide covers both.
Scheduled pipelines
Set a schedule (a cron expression) when creating or updating a pipeline via the Pipelines endpoint. KMP runs the pipeline once per schedule tick, processing whatever new data is available from the source since the last run.
{
"schedule": "0 */4 * * *"
}Event-triggered pipelines
Omit schedule and the pipeline instead runs continuously, processing events as its source connector emits them — the right choice for sources like Kafka or a Webhook connector where near-real-time matters.
Use a schedule for sources that batch naturally (a nightly file drop to S3). Use event-triggered for anything where minutes matter.
Pausing and resuming
A paused pipeline skips scheduled ticks entirely and buffers nothing for event-triggered sources — resuming picks up new events going forward, not a backlog. To reprocess a time range instead, see the replay flow in Handling errors & retries.