Connect REST API

Summary

Every Kafka Connect worker exposes a REST API, typically on port 8083. This API lets you deploy connectors, check status, pause and resume tasks, and monitor your data pipelines—all without restarting workers.

The REST API is available on every worker in a distributed cluster. You can send requests to any worker, and the result is the same because all workers share state through Kafka topics.

Because configuration is stored in Kafka topics, you can send a request to any worker and it will propagate to the entire cluster. This makes the API highly available and easy to use.

Common REST API Operations

List All Connectors

1curl http://localhost:8083/connectors

This returns a JSON array of all connector names currently deployed.

Get Connector Status

1curl http://localhost:8083/connectors/my-connector/status

This shows the connector's state, which tasks are running, and any errors.

Create a New Connector

1curl -X POST http://localhost:8083/connectors \
2  -H "Content-Type: application/json" \
3  -d '{
4    "name": "my-connector",
5    "config": {
6      "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
7      "tasks.max": "3",
8      "connection.url": "jdbc:mysql://localhost/mydb",
9      "mode": "incrementing",
10      "incrementing.column.name": "id",
11      "topic.prefix": "mysql-"
12    }
13  }'

This deploys a new connector with the given configuration. The cluster will create tasks and start moving data.

Update Connector Configuration

1curl -X PUT http://localhost:8083/connectors/my-connector/config \
2  -H "Content-Type: application/json" \
3  -d '{
4      "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
5      "tasks.max": "5",
6      "connection.url": "jdbc:mysql://localhost/mydb",
7      "mode": "incrementing",
8      "incrementing.column.name": "id",
9      "topic.prefix": "mysql-"
10  }'

This updates the configuration and triggers a restart with the new settings.

Delete a Connector

1curl -X DELETE http://localhost:8083/connectors/my-connector

This stops all tasks and removes the connector from the cluster.

Here's what happens internally when you create a connector via REST API. The request goes to one worker, which writes to the config topic, all workers read the new config, and the cluster distributes tasks.

This distributed coordination is why the REST API is so powerful. You don't manage individual workers—you manage the cluster as a whole, and tasks automatically distribute.

Advanced Operations

Beyond basic CRUD operations, the REST API supports several advanced management features:

Pause and Resume

1curl -X PUT http://localhost:8083/connectors/my-connector/pause
2curl -X PUT http://localhost:8083/connectors/my-connector/resume

Pausing stops data movement without deleting configuration. Resume picks up where it left off.

Restart Connector or Task

1curl -X POST http://localhost:8083/connectors/my-connector/restart
2curl -X POST http://localhost:8083/connectors/my-connector/tasks/0/restart

Useful when a connector or specific task encounters a transient error.

Validate Configuration

1curl -X PUT http://localhost:8083/connector-plugins/JdbcSourceConnector/config/validate \
2  -H "Content-Type: application/json" \
3  -d @config.json

Validates configuration before deployment to catch errors early.

Get Connector Topics

1curl http://localhost:8083/connectors/my-connector/topics

Shows which topics the connector is reading from or writing to.

A typical operations workflow uses the REST API for deployment, monitoring, and troubleshooting. You can integrate it with automation tools, CI/CD pipelines, or monitoring systems.

The REST API makes Kafka Connect easy to integrate into modern DevOps workflows. You can automate connector deployments, monitor health programmatically, and manage everything as code.