Building Kafka with Confluent Training in Kuala Lumpur, Malaysia

This course provides an in-depth, practical guide to building scalable, real-time streaming applications using Apache Kafka and the Confluent Platform. Participants will learn how to set up Kafka clusters, develop producers and consumers, integrate Kafka Connect, leverage Confluent Schema Registry, manage Kafka with Confluent Control Center, and build robust data pipelines with Confluent’s ecosystem.

Kafka Foundations & Confluent Ecosystem

• Kafka Architecture Overview

  • Topics, partitions, brokers

  • Producers, consumers, offsets

  • ZooKeeper and Kafka metadata

• Introduction to Confluent Platform

  • Confluent components (Kafka, Connect, ksqlDB, Schema Registry, Control Center)

  • Differences between Apache Kafka and Confluent Kafka

• Cluster Setup and Configuration

  • Installing Kafka and Confluent Platform (local & cloud)

  • Configuration files and tuning parameters

  • Broker and topic management (retention, compaction)

Development with Kafka & Schema Registry

• Kafka Producers and Consumers (Java/Python)

  • Producer acknowledgments and retries

  • Consumer groups, partitions, and offset strategies

• Serialization and Schema Registry

  • Avro and JSON Schema formats

  • Registering and evolving schemas

  • Using Schema Registry in producers/consumers

• Kafka Connect

  • Source and Sink connectors

  • Installing, configuring, and scaling connectors

  • Integration with JDBC, Elasticsearch, S3, etc.

Monitoring, Security, and Best Practices

• Confluent Control Center

  • Monitoring topics, throughput, latency

  • Alerting, auditing, and managing consumer lag

• Kafka Security and Access Control

  • TLS/SSL encryption

  • SASL authentication and ACLs

  • Role-based access in Confluent

• Stream Processing and ksqlDB Introduction

  • Use cases and query types

  • Example transformations and aggregations

• Performance Tuning and Best Practices

  • Partitioning strategies

  • Producer/consumer tuning

  • High availability and replication

• Hands-on Exercises

• Summary and Conclusion