Kafka Development Environment Preparation

Kafka Introduction

Kafka is a distributed message release and subscription system. With features similar to JMS, Kafka processes active streaming data.

Kafka is applicable to message queuing, behavior tracing, operation & maintenance (O&M) data monitoring, log collection, streaming processing, event tracing, and log persistence.

Kafka features:

• High throughput
• Message persistence to disks
• Scalable distributed system
• Fault-tolerant
• Support for online and offline scenarios

Interface Type Introduction

APIs provided by Kafka can be divided into two types: Producer API and Consumer API. Both the types of APIs contain Java API. For details, see section Kafka Java APIs.

Concepts

• Topic

  A same type of messages maintained by Kafka.

• Partition

  One topic can be divided into multiple partitions, and each partition corresponds to an appendant and log file whose sequence is fixed.

• Producer

  Role in a Kafka topic to which messages are sent.

• Consumer

  Role that obtains messages from Kafka topics.

• Broker

  A node server in a Kafka cluster.