User Tools

Site Tools


kafka

This is an old revision of the document!


Kafka

Install Java with SDKman ( curl -s “https://get.sdkman.io” | bash ) and “sdk install java 25.0.3-tem” (if there's a newer version and this page is not updated, use “sdk list java” to get the list). Install Maven with “sdk install maven 3.9.16” and Gradle with “sdk install gradle 9.5.1”.

Extract the Kafka 4.2.0 tarball.

Configure kRaft

# config/kraft/server.properties
process.roles=broker,controller
node.id=1 # Change for every server
controller.quorum.voters=1@a.xyz:9093,2@b.xyz:9093,3@c.xyz:9093

#Default recommended is 6 (so each node gets 2 partitions)
num.partitions=3

#At least 2 nodes must be in-sync; change existing lines
offsets.topic.replication.factor=3
transaction.state.log.replication.factor=3
transaction.state.log.min.isr=2
#reinitialize the log storage after  changing the default partition no.
KAFKA_CLUSTER_ID="$(./bin/kafka-storage.sh random-uuid)"
echo $KAFKA_CLUSTER_ID

#re-use the UUID of the 1st server's storage on the 2nd & 3rd servers when formatting storage
./bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c ./config/kraft/server.properties
sudo touch /var/log/mykafka.log && sudo chown ubuntu. /var/log/mykafka.log

# Create a service file, this is for testing
nohup ./bin/kafka-server-start.sh ./config/kraft/server.properties > /var/log/mykafka.log 2>&1 &

</code>

kafka.1779193078.txt.gz · Last modified: by reddy

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki