Course Introduction

Data Evolution Over Time

Big Data Overview

Big Data Analytics

Messaging System

Introduction and Features of Kafka

Kafka Use Cases

Kafka Terminologies

Kafka Components

Kafka Architecture

Kafka Clusters

Kafka Industry Use Cases

DEMO: Install Zookeeper

DEMO: Install Kafka

DEMO: Set up a Single Node Single Broker Cluster

DEMO: Set up a Multi-Node Single Broker Cluster

Specialization Introduction

Introduction and Syllabus

Big Data and Apache Kafka

Big Data Evolution

Kafka Basics

Zookeeper and Kafka Cluster Setup

Getting Started with Big Data and Apache Kafka

Overview of Producer and Architecture

Kafka Producer Configuration

Kafka Producer Optional Configuration

Kafka Producer Configuration Objects

DEMO: Create a Kafka Producer

Sending Messages

Serializers and Custom Serializers

DEMO: Creating a Custom Serializer

Serializers Challenges and Serializing Using Apache Avro

DEMO: Serializing Using Apache Avro

Partitions

DEMO: Set up Custom Partition

Kafka Producer

Configuring Kafka Producers

Working with Serializer

Kafka Partitions

Working with Kafka Producer

Overview of Kafka Consumer

Consumer Groups

Partition Rebalance and Creating a Consumer

Poll Loop and Its Functioning

Configuring Kafka Consumer: fetch.min.bytes fetch.max.wait.ms max.partition.fetch.bytes session.timeout.ms

Kafka Consumer Configuration: auto.offset.reset enable.auto.commit partition.assignment.strategy client.id max.poll.records receive.buffer.bytes send.buffer.bytes

DEMO: Create Kafka Consumer

Commit and Offset

Ways of Committing Offset - Automatic Offset

Ways of Committing Offset - Commit Current Offset

Ways of Committing Offset - Asynchronous Commit

Ways of Committing Offset - Synchronous and Asynchronous Offset

Ways of Committing Offset - Commit Specified Offset

Rebalance Listeners

Consuming Records with Specific Offset

Introduction to Deserializers

DEMO: Create and Use a Custom Deserializer

Course Synopsis

Kafka Consumer

Optimizing Kafka Consumers

Kafka Offset Handling

Kafka Deserialization

Working with Kafka Consumer