Kafka Connect

It makes it simple to quickly define connectors that move large data sets into and out of Kafka.

• connect
• connect settings
• connect api reference
• jdbc connector plugin
• mongo connector plugin
• project location: kafka-connect
• plugins location: kafka-connect/plugins
• requests location: kafka-connect/requests
• connect port: 8083

Run Kafka Connect:

cd kafka-connect
docker compose up -d
cd ..

After a few seconds:

http :8083/connector-plugins

Docker Compose

services:
  kafka-connect:
    image: confluentinc/cp-kafka-connect:${VERSION}
    environment:
      CONNECT_BOOTSTRAP_SERVERS: kafka1:9092,kafka2:9092,kafka3:9092
      CONNECT_REST_ADVERTISED_HOST_NAME: localhost
      CONNECT_GROUP_ID: kafka-connect-sandbox
      CONNECT_PLUGIN_PATH: /usr/local/share/kafka/plugins
      CONNECT_KEY_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONNECT_CONFIG_STORAGE_TOPIC: kafka-connect.config
      CONNECT_OFFSET_STORAGE_TOPIC: kafka-connect.offsets
      CONNECT_STATUS_STORAGE_TOPIC: kafka-connect.status
    ports:
      - "8083:8083"
    restart: on-failure
    volumes:
      - ./plugins:/usr/local/share/kafka/plugins
    healthcheck:
      test: curl http://localhost:8083
      interval: 30s
      timeout: 30s
      retries: 5
      start_period: 30s

networks:
  default:
    external: true
    name: kafka-sandbox_network