Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Summary

Table of Contents
excludeSummary

Initial step: Initializing Kafka

Install Kafka according to these instructions: https://computingforgeeks.com/configure-apache-kafka-on-ubuntu/

Code Block
heon@ubuntu:$ kafka-start.sh 
● kafka.service - Apache Kafka Server
Loaded: loaded (/etc/systemd/system/kafka.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2020-12-10 18:23:22 EST; 4 days ago
Docs: http://kafka.apache.org/documentation.html
Main PID: 830 (java)
Tasks: 82 (limit: 14243)
Memory: 799.9M
CGroup: /system.slice/kafka.service
└─830 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt.headless=true -Xloggc:/opt/kafka/bin/../l>

.....

heon@ubuntu:$ ./kafka-create-topic.sh 
Created topic vivo-dataconnect-topic

heon@ubuntu:$ ./kafka-list-topic.sh 
vivo-dataconnect-topic

heon@ubuntu:$ ./kafka-describe-topic.sh
Topic: vivo-dataconnect-topic PartitionCount: 1 ReplicationFactor: 1 Configs: segment.bytes=1073741824
Topic: vivo-dataconnect-topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0

heon@ubuntu:$ ./kafka-show-numberof-message.sh 
0


4) Send to Kafka

Code Block
./kafka-start-producer.sh < ../onto_uqam.n3 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ... >>>>>>>>>>>>>>>>>>>>heon@ubuntu:$ 

heon@ubuntu:$ ./kafka-show-numberof-message.sh 
25171

5) Receive data

6) Send to VIVO