Find Jobs
Hire Freelancers

Pyspark Data engineer for a test project in Spark

₹600-1500 INR

Closed
Posted about 3 years ago

₹600-1500 INR

Paid on delivery
Looking for a pySpark developer to help with a pyspark application for demonstration purpose. The goal is to read 3 small json files each less than 1 mb into a single data frame. Do preprocessing and then fetch some metrics based on regular expression search. ETL task is easy and I am able to achieve it. However the end goal is written below that is also the acceptability criteria for this project. The developer should be able to write code with following points in mind: Well structured, object-oriented and maintainable code. Unit tests for the different components. Documentation, comments and Proper exception handling. Solution is deployable and we can run it (locally and on a cluster) Config management. (separate folders and files like [login to view URL], [login to view URL] etc. rather than one python script). Logging and alerting. Data quality checks (like input/output dataset validation).
Project ID: 29872024

About the project

1 proposal
Remote project
Active 3 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
1 freelancer is bidding on average ₹1,050 INR for this job
User Avatar
Hi, I am working in MNC as Data Engineer and currently working on Big Data Fields using PySpark and Hadoop Frameworks. Having more than 4 years of experience in Big Data Field in production and certified pyspark developer as well as certified Hadoop developer. Requesting you to please share the details. Thanks Rahul.
₹1,050 INR in 7 days
4.4 (4 reviews)
2.2
2.2

About the client

Flag of INDIA
Pune, India
0.0
0
Member since Jul 8, 2019

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.