Azure: Customize data factory for delta logic

€8-30 EUR

Closed

Posted

over 1 year ago

€8-30 EUR

Paid on delivery

Two new columns are required in the Postgres database for this purpose: Datahash LastModified A hash is stored in Datahash, which is calculated from all columns except for Datahash and LastModified. When transferring the data from Conceptum, the method should be adjusted: Calculate hash from new data Compare new hash with old If matching, data hash, and LastModified do not change Otherwise, insert a new hash into the data hash and set LastModified to the current time (now). Use new data from Conceptum

Project ID: 35355169

About the project

3 proposals

Remote project

Active 1 yr ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

3 freelancers are bidding on average €38 EUR for this job

@patilpk153

Experienced in Azure and Azure data factory, need to check on how to best achieve this requirement

€39 EUR in 4 days

4.9

(5 reviews)

3.2

@perfomancedev

Greetings! Thanks for your job posting. I read your project details and i understood all. I can help you with your project at a high level! I have rich experience in Azure developing. My skills and experience is a great match for your needs. I can start right now. Looking forward to hearing from you. Best regards. thanks.

€50 EUR in 2 days

0.0

(0 reviews)

0.0

@tirthodebnath

I can build a smooth ETL pipeline that can help extract data from any source system(eg: Hive Database, Relational database server) hosted in any cloud platforms(eg: GCP, AWS, AZURE) & on premice servers also. The pipelines can also handle transformations and modifications of data as per requirement, and lastly the filter data will be stored in the desired database / data warehouse. For transformation and modification I use Azure Data Bricks notebook, and for orchestrating and monitoring the ETL pipeline I use Azure Data Factory. And for visualization purpose I use Azure Synaps to connect with the any visualization tools(eg: Power BI, Tablue, MSTR). But I would like to know more about the project. If possible I am ready to discuss about this over a zoom or google meet. I work as a Data engineer where I have build various pipelines as per client requirements and delivered successfully to the clients.

€25 EUR in 7 days