In Progress

Create Spark Helm Chart

Hi,

what I need is the following:

- I need a helm script (kubernetes) which "installs" the latest spark version as yarn cluster on kubernetes

- This setup should provide Zeppelin

- All of this has to work with sparks latest structured streaming framework and should be able subscribe a kafka topic

- Confluent is used as kafka provider (and it's security mechanism - [login to view URL] = [login to view URL])

- Zeppelin has to deploy jobs to cluster and NOT execute them standalone

- The kubernetes cluster doesn't use level 4 load balancer, so we can't simply use the following helm chart ([login to view URL])

- Kubernetes Level 7 Loadbalancers are used

- Basic neccessary setup (Nodes needs to have the corresponding spark versions deployed)

- Acceptance criteria: I have a structured streaming script, which we can use to check if setup works, in the meantime you can use your script for development.

- Kubernetes Version is v1.16.3.

- I can't grant you access to the cluster nor to the confluent stream.

Thanks for your time.

Skills: Kubernetes, Docker, Spark, Hadoop, Yarn

See more: sample helm charts, unleash helm chart, spark operator helm, bitnami/spark, spark kubernetes, github chart helm, popular helm charts, spark 2.4 helm chart, create pie chart flash, charts dow jones chart, create simple chart vbnet using dundas, structure chart create user, structure chart create, create excel chart csv file, macro create dynamic chart excel 2003, create seating chart, create structured chart, create spark messenger plugin, hi i am looking for a developer who can adjust an existing website laravel 4, The paper will be five pages: (a) Describe the 7 Step SDLC - 2 pages, (b) Describe a different SDLC Model (4 step or 12 step) -

About the Employer:
( 0 reviews ) Frankfurt am Main, Germany

Project ID: #25391090

Awarded to:

vivmistry

I am a DevOps Consultant & Certified Kubernetes Administrator with 13+ years of experience specializes in implementing and maintaining DevOps Tools using different tools like Jenkins, Docker, Kubernetes, Python, Groov More

€200 EUR in 7 days
(0 Reviews)
0.0

9 freelancers are bidding on average €285 for this job

alonlong

● I have been in software development for 10+ years. ● I am a Golang developer with 5+ years of experience. ● I was responsible for many projects which is based on the microservice architecture and the design and deve More

€500 EUR in 15 days
(6 Reviews)
4.3
iwanthereds

Hi there. Thank you for your posting. As a senior cloud developer, I am familiar with k8s and spark. I've been operating several k8s clusters (managed clusters, and on-premise clusters) for several years. Writing helm More

€250 EUR in 7 days
(5 Reviews)
2.7
rks2020

Hi, I am a Technical Architect with 5 years of experience in Hadoop, Spark, Docker and Kubernetes. I understand this requirement is to set up Spark and Zeppelin in Yarn on top of Kubernetes cluster using Layer 7 load More

€200 EUR in 7 days
(1 Review)
3.0
Ayouma

Hi, I'm a software engineer with 5 years of experience in Bigdata technologies (Kubernetes, Docker, Spark, Kafka, Scala, Python,..) in Paris on different activity areas like banking and energy :). For your project, I More

€500 EUR in 10 days
(2 Reviews)
1.8
mwebdeveloper

HELM script Expert hi,I have gone through your description and got a pretty much idea of what you want. I can assure you that i can do this task. The skills and resources needed for this project are in my genes. I can More

€250 EUR in 7 days
(0 Reviews)
0.0
usamaahmadkhan

Hi I am a DevOps Engineer with hands on experience in managing Kubernetes cluster. I understand your problem and can modify the helm chart as per your use case. I have also worked with Kafka in K8s and can modify it as More

€280 EUR in 7 days
(0 Reviews)
0.0
€111 EUR in 5 days
(0 Reviews)
0.0
neerajgulia92

Certified Kubernetes Administrator and Helm expert. Created more than 50 Helm charts customized for multiple customers.

€278 EUR in 10 days
(0 Reviews)
0.0