Data Acceleration

₹750-1250 INR / hour

Closed

Posted

almost 4 years ago

₹750-1250 INR / hour

Health Care Service Corporation (HCSC) is a not-for-profit corporation health insurance company in the United States. The current scope of Data Accelerator project is data ingestion and data processing. In this project we are ingesting data from different data sources to the data lake by applying the required business transformation rules and later analyzing the data for faulty records. This project was developed using Agile development methodology having Sprint of 3 weeks. Role and Responsibilities: As part of each Sprint, we were allocated respective target tables to which data should be loaded. Fixed length/Delimited flat files were loaded into the Source tables. According to the Mapping document we design the join criteria for the source tables. One or more source tables to be joined, applying different business rules and loading to Temp table. Writing Scala code to perform transformations joins on data frames and removing duplicates before loading to the target table. Developing code involved writing shell script, HQL’s, Spark-Scala code. Zena is used for job scheduling.

Apache Hadoop

Hive

Project ID: 26375393

About the project

2 proposals

Remote project

Active 4 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

2 freelancers are bidding on average ₹1,225 INR/hour for this job

@TechnicalGeeks

Hi, I am a bigdata developer and a module lead in reputed MNC.i an into the IT industry for more then 12 years. I have tonnes of experience in developing projects using Java,Apache Spark,Hive,Kafka,Scoop,Pig,Scala,aws,Spark streaming and managing cluster on aws cloud and Azure. I Have deployed multiple spark job on HDinsight and communication with data lake 1 and 2. I have delivered multiple lectures on setting up apache Spark on AWS EMR... I can be the best suitable person for your requirements I have also created multiple freelance project on AWS EMR,S3,REKOGNITION, RDS, and many more services Let me know your view Regards,

₹1,250 INR in 40 days

5.0

(1 review)

1.7

@jainrakesh1701

Hi, I have 10 years of IT experience and I have worked on SQL, Python, Shell scripting, Bigdata(Hadoop), Spark(Scala), Machine Learning models, NLP, Classification and Regression models. Since the last 7 years I am working in Bigdata and last 5 years in Spark(Scala). I have developed projects using Bigdata and Spark technologies. I have also done migration projects. I have also developed projects where data is fetched from Oracle and Teradata source in hadoop/hdfs. All my experience is in similar projects for clients like Apple, Johnsons and Johnsons and more. I have worked for 3 years in USA. I will be the best fit for this project. Let's discuss your requirement further.

₹1,200 INR in 40 days