Hey,
I already installed and Managed, Confluent Kafka with ansible for 3 companies in Turkey, I previously have small experience with Data Lakes, but on Kafka site I believe we can handle the tasks.
- Here are few things I did in my previous 3 Confluent Kafka Deployments.
*Installed Confluent-Kafka and other components such as Schema Registry, Connector and Zookeeper of course with Ansible.
*Wrote a connector config to, migrate the topics and record to ElasticSearch Indexes.
*Installed HA cluster and both located in different sites, and sencronized, Topics with lowest delay possible.
*Even the hardware wasn't enought, and keep gaving OOM (out of memory) errors, i still managed to kafka to run healty.
*The OS wasn't supported from Confluent and still made it run.
*Installed UI for Confluent Kafka named AKHQ to make a better user experience.
*Kafka monitoring on Grafana.
Best.