Find Jobs
Hire Freelancers

Process a Large Amount of Data Files Using Python

$100-400 USD

Completed
Posted over 4 years ago

$100-400 USD

Paid on delivery
I have a large amount of data files that I’d like to have processed (and select information placed into an output CSV). The purpose of the project will be: 1 – Develop a python script that will process the 180 input files (each input file is ~150mb and is in [login to view URL] format) 2 – Incorporate multi-threading to take full advantage of the high-power server that will be used for the processing (96 processors and 192gb of memory) 3 – Run the developed script on the files and generate the [login to view URL] (you will be given access to the server which may be accessed using RDP) ** Please see the attached write-up that explains the exact details in-depth. ** Please include a brief description in your PM along with your bid so I can tell that you reviewed the attached document read and fully reviewed the project description instead of using an auto-bidder to bid on the project. Just a sentence or two will do. ** This is a rush project. It is a requirement that the project needs to be completed within 24 hours. Hence the budget I have allocated. Thank you!
Project ID: 21234666

About the project

8 proposals
Remote project
Active 5 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Hi, I have good experiences developing code in Python Programming language for scientific application and Machine learning projects more than 3 years . I read description files which have 5 parts. In first parts you need to find unique "IMDB ID" from input files and write them into one output file. After that in sequential 4 parts you need to find one attribute for each "IMDB ID" from part's input file(s) and add it to output file. For Multiprocessing in python i use multiprocessing module , but for this project we may can Dask package which can provide scalable pandas-like dataframe. I have one question. In part 2-5 you have only one input file? Thanks
$230 USD in 1 day
4.9 (27 reviews)
4.7
4.7
8 freelancers are bidding on average $226 USD for this job
User Avatar
Hello,sir. I have checked your project description. I have 7+years of experience software and data processing. I can implement your project with fully and completely. I can help you with my best. Thanks for your posting. I hope that you contact with me. If you need, i will send my live link to you.
$250 USD in 7 days
4.9 (22 reviews)
4.8
4.8
User Avatar
***** Scraping Expert ***** Hello. I have read your job description carefully. I have full experience in Web Scraping with PHP and Python. So I can do your project perfectly. I can start just now and I can deliver good result in time. I hope to discuss more via chatting. Thank you.
$250 USD in 2 days
5.0 (7 reviews)
4.3
4.3
User Avatar
Hi there, I am Python developer, having below given skills: Engineering professional with 10 years of experience in Software development. Mastering/Leading in the development of applications/tools using Python for 6 years. Worked on several python packages like numpy, scipy, pytables etc.. Having good experience in Hadoop Big Data processing. Expertise in developing the queries in Hive, Pig Mastering Web Application Development using html, JavaScript. Having Good Experience in Object Oriented Concepts with C , Java, Perl and Python. Experience in Design, Development, Testing, Automation, Enhancement and Maintenance of Applications and Tools. Good knowledge in various phases of SDLC Requirement Analysis, Design, Development and Testing on various Development and Enhancement Projects. Having good domain expertise in developing applications/tools using Perl and programming in C, C and Java. Having experience in developing GUIs using Tcl, Tk and Java. Good Experience in C/C development of multi-threaded applications in a Linux Environment. Good Experience in error and exceptional handling. Thanks
$250 USD in 7 days
4.8 (3 reviews)
3.5
3.5
User Avatar
⭐⭐⭐⭐ i am python developer. ⭐⭐⭐⭐⭐⭐ I read through the job details extremely carefully and I am absolutely sure that I can do the project very well. * 6+ years experience in web and mobile app development. If you want high quality of your project, please choose me. If you hire me, you will get all what you want, Timely delivery . Best quality of my work. 24 hours communication . Hope to get your early response . Thanks
$250 USD in 7 days
5.0 (6 reviews)
3.1
3.1
User Avatar
Hello Sir, Nice to meet you. How are you doing sir? I read your project description. I think I am able to complete your project. Kindly message me for discuss your project. Thank your sir, Best Regard, Md Abu Shalea
$100 USD in 7 days
2.2 (2 reviews)
0.0
0.0
User Avatar
The project sounds relatively straight forward. Taking info from files and populating a CSV into ordered, retrievable data. The program of which takes advantage of a high-power server by using multi-threading. I've written a number of scraping scripts and have researched multi-threading programs. I also have a network of mentors who can guide me if I hit obstacles. I'd love the opportunity to prove myself. My schedule is quite flexible and can get started when you need it. Thanks!
$200 USD in 3 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
New York, United States
5.0
115
Payment method verified
Member since Jul 14, 2011

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.