Kafka REST Proxy

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster. Use this when you really need a rest interface since it is usually more complex than using conventional kafka clients. You can check the API reference here.

Setup

Run Kafka REST Proxy:

docker compose --profile proxies up -d

Check the cluster information:

http kafka-rest:8082/v3/clusters

Create Topic

Payload:

{
  "topic_name": "proxy.rest",
  "partitions_count": 3,
  "replication_factor": 3
}

Hit rest proxy:

http kafka-rest:8082/v3/clusters/${CLUSTER_ID}/topics < kafka-rest/requests/create-topic.json

List topics:

http kafka-rest:8082/v3/clusters/${CLUSTER_ID}/topics | jq -r '.data[].topic_name'

Produce

Payload:

{
  "key": {
    "type": "STRING",
    "data": "ce59e9a6-aafd-44f4-bdc6-f285adfcc836"
  },
  "value": {
    "type": "AVRO",
    "schema": "{\"type\": \"record\", \"name\": \"User\", \"fields\": [{\"name\": \"name\", \"type\": \"string\"}]}",
    "data": {
      "name": "John Doe"
    }
  }
}

Send payload:

http kafka-rest:8082/v3/clusters/${CLUSTER_ID}/topics/proxy.rest/records < kafka-rest/requests/produce-avro-message.json

Consume

kafka-avro-console-consumer --bootstrap-server kafka1:9092 \
        --topic proxy.rest \
        --from-beginning \
        --property schema.registry.url=http://schema-registry:8081