Kafka Integration Playground

Playground environment to evaluate integration appraches with Kafka.

This repository contains the following integration playgrounds:

• source-httpcastorm-avro: Kafka Connect - Castorm HTTP Source Connector with Avro

• source-httpcastorm-json: Kafka Connect - Castorm HTTP Source Connector with JSON

Quick Start

Ensure that platform with Kafka and Schema Registry is up and running

./dc-env.sh platform up -d

Run and play around with any of the integrations, e.g. source-httpcastorm-avro.

Finally, shut down the platform

./dc-env.sh platform down -v

Docker Images

The Docker images used for Kafka and Kafka Connect are custom builds of the Open Source version of Apache Kafka. The corresponding Dockerfiles are located in the kafka-images GitHub repository.

Detailed information about configuration of these images can be found at:

• Kafka Server Standalone Configuration

• Kafka Connect Standalone Configuration

The configuration is fully compatible with the Confluent Docker images.

The reason why we used the custom images is, that configuration during development is simplified.