Kafka ksqlDB
ksqlDB is a database that's purpose-built for stream processing applications.
⚠️ ksqlDB it is not a SQL database, it provides an extra layer for implementing kstream, ktable and connectors through a language (ksql) based on sql.
- ksqldb
- ksqldb settings
- ksqldb test runner
- project location: kafka-ksqldb
- ksqldb port:
8088
Create an alias for ksqldb-cli
:
⚠️ Run alias commands inside the root folder.
alias ksqldb-cli="docker run --rm -it --network kafka-sandbox_network --workdir /ksqldb -v $PWD/kafka-ksqldb/tests:/ksqldb/tests -v $PWD/kafka-ksqldb/statements:/ksqldb/statements -v $PWD/kafka-ksqldb-extensions/extensions:/ksqldb/extensions kafka-cli:latest "
To permanently add the alias to your shell (~/.bashrc
or ~/.zshrc
file):
echo "alias ksqldb-cli='docker run --rm -it --network kafka-sandbox_network --workdir /ksqldb -v $PWD/kafka-ksqldb/tests:/ksqldb/tests -v $PWD/kafka-ksqldb/statements:/ksqldb/statements -v $PWD/kafka-ksqldb-extensions/extensions:/ksqldb/extensions kafka-cli:latest '" >> ~/.zshrc
Run ksqlDB:
cd kafka-ksqldb
docker compose up -d
cd ..
After a few seconds:
http :8088/info
One line shell interaction:
ksqldb-cli ksql -e "SHOW STREAMS;" http://ksqldb:8088
Interactive ksqlDB shell:
ksqldb-cli ksql http://ksqldb:8088
SHOW STREAMS;
Docker Compose
services:
ksqldb:
image: confluentinc/cp-ksqldb-server:${VERSION}
environment:
KSQL_KSQL_SERVICE_ID: kafka-ksqldb.
KSQL_LISTENERS: http://0.0.0.0:8088
KSQL_BOOTSTRAP_SERVERS: kafka1:9092,kafka2:9092,kafka3:9092
KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
KSQL_KSQL_SCHEMA_REGISTRY_URL: http://schema-registry:8081
KSQL_KSQL_CONNECT_URL: http://kafka-connect:8083
KSQL_KSQL_EXTENSION_DIR: /ksqldb/extensions
ports:
- "8088:8088"
restart: on-failure
volumes:
- ../kafka-ksqldb-extensions/extensions:/ksqldb/extensions
healthcheck:
test: curl http://localhost:8088
interval: 30s
timeout: 30s
retries: 5
start_period: 30s
networks:
default:
external: true
name: kafka-sandbox_network