Collect Module API Routes

Step-by-step tutorials for common API-based pipeline management scenarios.

Use cases

Scenario	Description	Guide
New column in the source	Add a column to an incremental pipeline and backfill historical data	Case 1
Duplicate records	Migrate from `incremental` to `incremental_with_qualify`	Case 2
Job failing with OOM	Increase memory for a job that fails due to lack of resources	Case 3
Customize table names	Customize raw and qualify table names in Snowflake	Case 4

Pipeline (postgres_vendas)
├── Job 0: clientes
├── Job 1: pedidos
└── Job 2: produtos

A pipeline contains one or more jobs. Each job synchronizes one table.

The jobId follows this format: {pipeline_id}-{job_index}.

Example: f47ac10b-58cc-4372-a567-0e02b2c3d479-0 (first job)

Note: in API responses, job_id may appear normalized with _ (underscores), but when calling the endpoints you should use - (hyphens).

Mode	Behavior
`full_load`	Extracts everything on every run
`incremental`	Extracts only new or changed records
`incremental_with_qualify`	Incremental extraction plus primary-key deduplication

Action	Endpoint
List pipelines	`GET /platform/pipelines`
Inspect a JDBC job	`GET /platform/jobs/jdbc/{jobId}`
Add columns	`PATCH /platform/jobs/{jobId}/input`
Migrate sync mode	`POST /platform/jobs/jdbc/{jobId}/sync-mode`
Change job memory	`PUT /platform/jobs/{jobId}/memory`
Change pipeline memory	`PUT /platform/pipeline/{pipelineId}/memory`
Reset state	`POST /platform/jobs/{jobId}/reset-state`
Execute pipeline	`POST /platform/pipeline/execute`
Monitor runs	`GET /platform/pipeline/{pipelineId}/runs`

For detailed endpoint documentation, see the API reference.