Airflow scala jobs
I am looking for a skilled Scala programmer to assist me in developing a new Scala program. This task involves writing code from scratch, with the specific focus on utilizing the Spark framework. Key Responsibilities: - Developing a new Scala program using Spark - Ensuring the code is optimized and efficient for the desired data processing application Ideal Skills: - Proficiency in Scala programming, with a strong understanding of Spark - Experience with writing code from scratch - Background in data processing applications would be a plus
I'm looking for a skilled programmer with robust knowledge of Scala, Hive and Spark to work on a data-related project. Key Points: - The project focuses on coding - The primary tools involved are Scala, Spark, Hive - No specific area of computer science is listed, just the programming languages Ideal candidate should be proficient in: - Scala, Spark, Hive - Data analysis and processing - Building scalable applications Previous experience in these areas would be a big plus.
As a client, I'm seeking a COMSOL MULTIPHYSICS expert who can conduct an in-depth analysis on the heat transfer and airflow rates of a vehicle brake disk. The primary goal of this project is to evaluate the performance of two different materials - cast iron and carbon ceramic - under varying conditions. Key responsibilities include: - Analyzing the heat transfer rate of the brake disk made from both cast iron and carbon ceramic. - Investigating the airflow rate on the brake disk for these materials. - Comparing and presenting the results of the analysis, particularly focusing on the performance differences between the two materials. - Providing insights and recommendations on the preferred material based on the analysis conducted. Ideal skills and experience for this ...
I'm in need of skilled machine learning developers with experience in SpringBoot and Scala. Your primary responsibility will be: - Training and tuning a clustering model. We are planning to apply this model for customer segmentation purposes. As such, the ideal candidate should: - Have previous experience in creating clustering models. - Understand the dynamics of customer segmentation. - Be proficient in data cleaning and preprocessing. - Have experience integrating and deploying machine learning models. You should be able to translate technical requirements into effective machine learning solutions and measure the effectiveness of the model's performance.
I have a project that requires extensive knowledge in Scala Java, focused on analyzing post metrics from social media. Responsibilities include: - Coding in Scala Java to retrieve and analyze social media post metrics - Performing data analysis in order to draw actionable insights The suitable freelancer for this job is someone with previous experience in Scala Java and data analysis, preferably involving social media data. The task requires in-depth knowledge in data sorting and categorization methods and a good understanding of social media platforms.
I'm looking for a freelancer who is an expertise of drying - particularly apron drying. The following designs are required for this project: Feeding mechanism Fan and airflow determination Apron design Controls and PLC Material to dry : potato paste
I'm in search of a seasoned DevOps consultant with strong expertise in Kafka and Akka. The role centers around the design, implementation, and monitoring of our cloud-based infrastructure on AWS. Strong scala programming skills and experience with Scala frameworks such as Akka. Efficient in designing and developing REST API’s. Experience with SQL and NoSQL databases. Proficiency in software design patterns and principles. Experience with version control tools such as Git. Good understanding of Agile methodologies and a collaborative mindset. Excellent problem- solving and analytical skills. Experience in unit testing (eg. JUnit/ Mockito). Experience in cloud- based environments such as AWS or Azure. Experience with Kafka and Elasticsearch. Experience : 5+ years ...
I am looking for an experienced Scala developer to implement a specific functionality in a data processing environment. The aim is to process text data more efficiently. Key Requirements: - Deep knowledge of Scala and spark - Proven experience in data processing, notably with excel, json and text data - Ability to deliver clean and efficient code The ideal candidate should be able to show previous projects where they successfully implemented or improved similar functionalities.
I'm seeking an expert in computational fluid dynamics who can help me understand how air flows through our stationary louvres, with a focus on improving ventilation efficiency. The goal is to work out the amount of airflow through the Louvres at different wind speed Ideally, you will have: - Extensive experience with CFD - Understanding of aerodynamics, specifically related to ventilation systems - Prior work with stationary louvres - Ability to interpret simulation data and provide actionable insights for design modifications to enhance ventilation efficiency.
... Ideal Candidate: - Proficient in Pyspark with hands-on experience in data storage and retrieval projects - Familiar with Data Warehousing concepts and best practices - Able to recommend and implement appropriate real-time processing solutions - Strong attention to detail and commitment to data quality. Specifically, I have a Jira ticket that consists of creating an application that runs on Airflow and connects to an API with survey metadata with values such as if it was opened, if it was answered, etc, then it should generate a zip file with all the data in jsons and save it in a S3 bucket. Once in the bucket you must save the information in a Hive table. All the code should be from Pyspark and there is a similar application that saves the raw survey data that you can take as ...
Looking for qualified performance engineer with experience using Gatling, Scala to work with
...best practices. • Stay updated with the latest trends and advancements in Azure data technologies to continuously improve our data solutions Key Skills :- • Bachelor's degree in Computer Science, Information Technology, or related field. • 4-5 years of professional experience as a Data Engineer with a focus on Azure Databricks and Azure Data Factory. • Strong proficiency in SQL, Python, and/or Scala for data manipulation and scripting tasks. • Experience working with data lakes, data warehouses, and ETL processes. • Familiarity with cloud platforms (preferably Azure) and related services such as Azure Blob Storage, Azure SQL Database, etc. • Excellent problem-solving skills and attention to detail. • Effective communication and col...
I'm looking to...comfortable, calming spot for unwinding and enjoying downtime. - Material: I prefer the patio to be constructed out of wood. It should complement the existing style of the house and provide a warm, natural aesthetic. - Design Element: I'd like to incorporate ceiling fans into this patio. These fans should provide both aesthetic appeal and practical function, helping to regulate the temperature and airflow, ideal for those warm, summer days. Experience with similar projects and a good eye for design are essential for this job. A keen understanding of structural integrity and the ability to seamlessly integrate the new construction with the existing house is crucial. Prior experience with wood construction and ceiling fan installations in outdoor spaces wou...
I'm seeking a skilled professional who is well-versed with Ansys Fluent to conduct airflow analysis on a single-flue chimney. The purpose of the project is to comprehensively assess the airflow within the structure. While it is crucial to accurately determine the chimney's airflow behavior, a basic assessment without considering turbulence considerations will suffice for this project. Ideal candidates will have a strong background in fluid dynamics simulations and hands-on experience using Ansys Fluent. Proficiency in interpreting and summarizing complex airflow data will also be beneficial.
...Real-time Sentiment Analysis for an Online Store 1. Overview: - Develop a web application for an online store where users can submit reviews for products. - Utilize Apache Kafka for real-time event streaming. - Implement Apache Airflow for workflow orchestration. - Use Apache Spark for data processing and sentiment analysis. - Store processed data in Cassandra DB for recording activities. - Store analyzed data in Amazon S3. 2. Workflow: - User submits a review through the web UI. - The review data is streamed to Kafka topics. - Airflow schedules tasks to ingest data from Kafka, process it using Spark, and store it in Cassandra. - Spark performs sentiment analysis on reviews. categorize it into positive , negative, neutral - Analyzed data...
I'm looking for an expert in GCP & Airflow who understands the intricacies of creating an Airflow DAG in Project-A (cloud composer) and triggering it via Project-B's service account. Work encompasses: - Crafting an Airflow DAG implementing Python script to create BigQuery tables. - Making the DAG operative within the confines of Project-A. - Leverage the service account of Project-B to trigger the DAG. - The BigQuery tables need to be created within Project-C. Ideal skills and experience: - Proficient in Python Scripting. - Experience in creating and managing workflows using Airflow. - Extensive knowledge of Google Cloud Platform. - Familiarity with BigQuery is a must. - Proven ability to script and manage service account interactions between ...
We are seeking a highly skilled and experienced Airflow and Heat Simulation Specialist to evaluate and optimize the ventilation systems of our crypto mining facilities. Each site encompasses an industrial unit under 200 square meters, requiring detailed analysis to ensure operational efficiency and safety. Responsibilities: Conduct detailed airflow and heat simulations to assess the current ventilation efficiency. Identify potential areas of improvement and provide actionable recommendations. Work closely with our on-site team to understand the specific configurations and requirements of each location. Deliver comprehensive reports summarizing findings and suggesting optimizations. Requirements: Proven expertise in airflow and heat simulation, specifically within in...
I need a proficient Apache Airflow and SQL consultant to provide expert-level support, specifically focused on complex query formation. - Specific Project Tasks - Implementing Apache Airflow - Creating complex SQL queries - Handling financial transaction data The ideal collaborator should have: - Proven experience working with Apache Airflow - Proficient in SQL, with an emphasis on generating complex queries - Prior experience managing financial transaction data is highly desirable. The objective is to streamline the handling, management, and processing of large volumes of financial transaction data, ensuring efficient and accurate results.
I need a proficient simulation expert who can model the airflow through Louvres in an industrial building setting. The key drive behind this project is to improve ventilation efficiency, minimizing energy waste, while remaining fully functional. Expected Deliverables: - Simulation of airflow within the identified industrial environment. Required skills and expertise: - Excellent understanding of airflow principles and simulations. - Proficient in utilizing simulation software. - Real-world experience with industrial ventilation systems is a plus. - Strong communication to effectively discuss simulation results. Preferably, the final output should be delivered as a video simulation, providing a dynamic view of the air circulation process. This project will greatly b...
I'm in need of a skilled designer to construct a fan shroud that adapts a 120mm fan to a 92mm radiator, with the main goal of maintaining optimal air flow. - The fan shroud is meant to: - adapt a 120mm fan to 92mm radiator - maintain optimal airflow - The specific dimensions for the fan and radiator are different The ideal freelancer for this project should have relevant experience in: - Designing similar components, preferably with a background in engineering or product design. - Working with plastic materials for construction. Looking forward to working with someone who can deliver a high-quality, efficient fan shroud.
As a client, I'm seeking a Python developer who that can help me trasforming data. The primary source of the data will be company-related information. Ideal Skills and Experienc...seeking a Python developer who that can help me trasforming data. The primary source of the data will be company-related information. Ideal Skills and Experience: - Proficiency in Python - Extensive experience in data analysis, particularly with telecomdata - Familiarity with APIs for data collection - Knowledge of telecom - Ability to understand Python application and python scripts -Know about Airflow , Spark, Kafka, Transaction Mysql Key Requirements: - Develop a Python program that can efficiently trasform non useful to useful data. - Strong understanding of telecom data -Spark sessions, ET...
As an ETL Expert Data Engineer with Airflow expertise, I require a professional who can take on a sophisticated data integration task. Key Responsibilities: - You will be responsible for data integration. - The data will be pulled from various APIs. - The processing logic required is advanced, aimed at facilitating analytics and machine learning. You should possess: - Extensive ETL experience. - Deep knowledge of Apache Airflow. - Proven track record in handling complex data integration tasks. - Capability in building data pipelines for advanced analytics. - Familiarity with APIs, ensuring efficient data extraction. - Strong problem-solving skills and attention to detail. I am looking for someone who can take this project and deliver high-quality results promptly.
...transparency. We're seeking an architect who will listen attentively to our needs, communicate effectively throughout the design process, and ultimately deliver results that exceed our expectations. Attention to Detail: From the layout of each room to the selection of fixtures and finishes, every aspect of the villas should reflect meticulous attention to detail. The design should optimize for good airflow and heat prevention. Experience with Luxury Residential Projects: Preference will be given to architects with experience in designing tropical villas, ideally in Thailand or Koh Phangan. We would like to see your project experience and portfolio beforehand. Consultative and Dedicated Approach: We are looking for an architect who acts as a consultant, communicates effect...
I am in urgent need of Hadoop/Spark developer who is proficient in both Scala and Python for a data processing task. I have a huge volume of unstructured data that needs to be processed and analyzed swiftly and accurately. Key Project Responsibilities: - Scrubbing and cleaning the unstructured data to detect and correct errors. - Designing algorithms using Scala and Python to process data in Hadoop/Spark. - Ensuring effective data processing and overall system performance. The perfect fit for this role is a professional who has: - Expertise in Hadoop and Spark frameworks. - Proven experience in processing unstructured data. - Proficient coding skills in both Scala and Python. - Deep understanding of data structures and algorithms. - Familiarity with data ...
I'm seeking a talented CFD software developer (using open source) to create a program that simulates airflow from an air tank through a pipe with outlets. The design of of a tank, pipe and nozzles will be based on parametric modelling. There should be input values that will be processed by the program and generate the required design, modelling and the simulation. The simulator must account for various parameters such as: - Tank pressure - Pipe characteristics - Nozzle sizes - Valve opening time - Air flow - Temperature - Porosity of cylinders - Size of pipe Additionally, the program should generate several outputs including the remaining tank pressure, pressure of the remaining air in the tank, the static pressure values created at different heights inside the cylinder, and ...
Help me in solving scala leetcode problems of data structures.
...of louvres under specific conditions, as per AS 4740:2000 standard. Our primary objective with the CFD simulation is to evaluate the performance of the louvres under variant conditions, more specifically: - The impact of wind and the associated pressure conditions. - The efficiency of airflow and ventilation through the louvres. The candidate must: - Have a deep understanding and an ability to execute CFD simulations. - Be familiar with the implications of wind impact, pressure, airflow, and ventilation in regards to louvres. - Have extensive knowledge of the AS 4740:2000 standard specifications. This project requires precision and a keen eye for detail to identify potential improvements and inform future design modifications. An individual with a histo...
The Data Engineer contractor role will be a project based role focused on migrating data pipelines from legacy infrastructure and frameworks such as Scalding to more modern infrastructure we support such as Spark Scala. This role will be responsible for: Analyzing existing data pipelines to understand their architecture, dependencies, and functionality. Working closely with data engineers to develop a migration strategy for converting pipelines from the current framework to the target framework. Designing, building, and testing new data pipelines in the target framework, ensuring they meet or exceed existing performance and reliability standards. Debugging and troubleshooting issues that arise during the migration process, working collaboratively with cross-functional teams to resol...
Hi, Please apply only individual. Agency can apply but budget should not be more than mentioned. Role : GCP Engineer (OTP) Exp : 7 + yrs SHIFT: IST Cloud Storage Buckets, BigQuery (SQL, Data Transformations and movement) Airflow (python, DAGs), DBT IAM Policies PyCharm Databricks (pySpark), Azure DevOps Clear and confident communication
The Data Engineer contractor role will be a project based role focused on migrating data pipelines from legacy infrastructure and frameworks such as Scalding to more modern infrastructure we support such as Spark Scala. This role will be responsible for: Analyzing existing data pipelines to understand their architecture, dependencies, and functionality. Working closely with data engineers to develop a migration strategy for converting pipelines from the current framework to the target framework. Designing, building, and testing new data pipelines in the target framework, ensuring they meet or exceed existing performance and reliability standards. Debugging and troubleshooting issues that arise during the migration process, working collaboratively with cross-functional teams to resol...
I am in need of a seasoned software engineer with intermediate expertise in Python. Your primary responsibility will be data processing using not only Python, but also complemented by Spark, Databricks, and Airflow. Therefore, prior experience with these is also expected. Ideal skills for this role include: - Intermediate to advanced skill level in Python - Experience with Spark, Databricks, and Airflow - Proficient in data processing tasks Your expertise in these areas will be integral to the success of this project and your problem-solving skills will be highly valued. I look forward to your proactive participation in this exciting project.
"Airflow pipeline I require a professional with experience in Airflow and dbt for my project. You will be handling an ETL/ELT integration specifically for data migration. This migration task involves moving specific tables from one MySQL database to another one. I'm trying to migrate data from mysql to mysql. We need to transfer incremental data on an hourly basis. We are looking to use open source ETL or ELT. In addition to airflow, you can use other open source ETL tools.".
I require a professional with experience in Airflow and dbt for my project. You will be handling an ETL/ELT integration specifically for data migration. This migration task involves moving specific tables from one MySQL database to another one. Your key responsibilities for this project will include: - Identifying the specific tables needed for migration, as indicated by me. - Developing and implementing an efficient migration process using Airflow and dbt. - Ensuring successful and accurate data migration. Ideal skills: - Incremental data must be transferred every hour. - Transfer speed is important. - Extensive experience in Airflow & dbt. - Proficient in MySQL databases. - Experience in data migration projects. - Thorough knowledge of ETL/ELT processes. - Att...
...and develop Airflow DAGs to schedule and manage ETL workflows. -Implement best practices for data engineering, including data modeling, data warehousing, and data pipeline architecture. -Monitor and troubleshoot ETL pipelines to ensure smooth operation. -Custom Python functions to handle data quality and validation. -Should have good understanding on S3,Cloud Formation, Cloud Watch, Service Catalog and IAM Roles -Perform data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes. -Should have good knowledge about Tableau, with creating Tableau Published Datasets and managing access. -Write PySpark scripts to process data and perform transformations.(Good to have) -Run Spark jobs on AWS EMR cluster using Airf...
Custom Table Design Specifications Request for Furniture Designer • I'm looking for a furniture designer who can design and provide detailed measurements for woodworkers to craft the required table. Table Dimensio...Tabletop Design • If the idea of the partition is difficult to design for a non-circular table, then please design the table with a 60 cm by 60 cm top. Extra Pieces A piece for fully closing the sphere from above (and if possible, it should include a cable management component). • 4 pieces that are 10 cm by 10 cm, 10 cm high, and 2 cm deep (these pieces are used to elevate a PC case for better airflow). • The table will be used for the Fractal Design Torrent PC case, so you may want to review the case dimensions for a better understanding o...
...balcony with a washing area. Key requirements: • Must incorporate a minimalistic architectural design, given my style preference • The building will be G + 4 stories • The steps and lift need to be positioned on the north side • The main gate must open to the south • Light and ventilation optimization are of high importance, the design should maximize natural light availability and ensure good airflow • A keen eye for space utilization, with a talent for creating practical and aesthetically pleasing designs Ideal skills and experience: • Extensive experience in residential design • Strong portfolio with minimalistic design samples • Proven track record in maximizing natural light and ventilation in living spaces • Expertise i...
I'm on the hunt for a freelancer proficient in working with plastic materials to design an efficient air duct system for my desktop computer. The purpose of this project is to optimize the airflow, particularly in the CPU and graphics card areas. Key Details: - Experience in creating air duct system for computers is essential. - Knowledgeable in working with plastic as the primary material. - Strong familiarity with the architecture and cooling requirements of desktop models is needed. As the end result, I expect functional and well-fitted air ducting that can significantly contribute to effective cooling of my machine's critical components. Looking forward to your proposals.
...detailed. Coding and smart contracts. This is a technical field, and many jobs require blockchain development experience. Ethereum developer jobs will need smart contract programming skills (Solidity). UX design and web development. At the front end, UX tools (HTML, CSS, JavaScript) will be helpful, without too much adaptation from what you’d need in other sectors. On the backend, Node.js, Go, Python, Scala and other languages will be valuable. Data analysis. The blockchain records a wealth of information, and if you have a background in data science you may be eligible for some of the top careers in cryptocurrency. Security and Cryptography. These skills will always be useful for any technical blockchain role. Experience in blockchain development, cryptography, and smart ...
I need an experienced Scala and Python developer to become an integral part of my project's implementation. The project will be centered around developing a robust, high-performance web application. Ideal Candidate: - Should possess deep knowledge and expertise in both Scala and Python. - Must show tangible experience with successful web application development using both languages. Skills and Experience: - Strong programming skills in Scala & Python - A portfolio of past projects proving your skills - Proficiency in web application development - Demonstrable experience in web application programming Your application will stand out if you: - Include examples of your previous work in Scala and Python. - Detail relevant achievements and specific roles ...
Searching for a Scala developer with a strong understanding and experience with Functional Programming. The project specifics are: Functionalities: - The exact functionality of the project was not specified. However, a Scala developer with general expertise in Functional Programming would be desirable. Timeline: - Project completion timeline was not provided at the moment. A scala developer who can be somewhat flexible with timeframes would be beneficial. Skills and Experience: - Proven track record as a Scala developer - Advanced knowledge and understanding of Functional Programming in Scala - Ability to interpret client needs and provide solutions - Flexible with timeframes - Strong problem-solving skills If you meet the criteria above, please don&#...
I need an experienced SWOOD for Solidworks designer who can help us in extending our library. Your work will revolve around: - Creating new 3D models with basic complexity. You'll be dealing with simple shapes and structures. - Adherence to industry-standard design p... - Adherence to industry-standard design principles is a must. An ideal freelancer would be proficient in SWOOD, with a firm understanding of 3D modeling and industry-standard design principles. Strong attention to detail, along with the ability to follow guidelines and meet deadlines is necessary. We need to import standard BLUM hinges in our library as SWOODboxes. So we can use the full scala of their products in our designs. If you now SWOOD up from version 2023 , you know there is import and than some ...
I'm seeking a proficient CFD expert to help me create a tri-manifold design. The primary objective of this project is to achieve uniform velocity, with a target of up to 80m/s. Your task will be to specifically design the manifold via custom aperture, allowing optimal control of airflow for the realization of my desired velocity. Key Requirements: - Proficient in FreeCAD software - Prior experience with tri-manifold design - Deep understanding of pressure distribution - Ability to deliver a digital 3D model This project demands someone who can not only produce superior model quality but also meet strict deadlines. Proven experience in similar projects will be a significant advantage. I'm looking forward to discussing my requirements in more detail and seeing your ...
I'm seeking a talented product design engineer with expertise in consumer electronics, to create a roof natural air inductor. This is intended to solve the issue of poor ventilation within a space less than 1000 sq ft. Here's what I'm looking for: - Innovative design that can maximize airflow - Energy efficient and easy to install - Robust and durable to withstand various weather conditions Experience in ventilation systems and household appliances will be a significant advantage. Proficiency in CAD software and knowledge of manufacturing processes are also required. Your proposal should include a brief discussion on how you intend to tackle the problem of poor ventilation using this product. Looking forward to creative and practical ideas.
I am searching for a skilled airflow developer who specializes in creating basic data pipelines. The specific requirements for this project are: - Creating dynamic DAG by reading data from CSV file. Say i have a CSV file:- table_name table_1 table_2 table_3 I need to create 3 dynamic DAG by using this.
...emerging technologies and best practices in data engineering. *Requirements:* - Bachelor's degree in Computer Science, Engineering, or a related field. - Proven experience as a data engineer or similar role, with a strong understanding of data modeling, ETL processes, and database technologies. - Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Java, Scala). - Hands-on experience with cloud platforms (e.g., AWS, Google Cloud, Azure) and related services (e.g., S3, GCP BigQuery, Azure Data Lake). - Familiarity with containerization and orchestration tools such as Docker and Kubernetes. - Excellent problem-solving and communication skills. - Ability to work independently and collaborate effectively in a remote team environment....
I need a Airflow expert to help redesign my data flow and put in place a solution that is scalable and can account for future needs. Our basic flow is making API calls to third-party solutions and getting data for various customers. This requires using the right API keys for each customer. The data will then be put in a Kafka queue for further processing. I am looking for someone with experience to help think through and implement solutions.
My project involves designing a rigid metal dryer vent duct of 50 feet maximum length. The design must account for efficient airflow, fire safety, and city approval requirements. Optimizing the duct to fit within the space provided is a crucial task. Ideal candidates will have experience with ductwork design, fire safety protocols, and city code requirements for HVAC systems. Expert knowledge about airflow optimization and rigid metal material is desired. There are 2 vent ducts and 2 separate dryer vent booster fans installed on 4" ducting. I need the design to have engineering approval to supply to the city inspector. I will need an engineer with a stamp or some form of verification for the state of California that the City building department will accept to sign off o...
I'm searching for a COMSOL multiphysics expert who can conduct a comprehensive analysis of various braking disks used in automotive applications. Specifically, I need someone to: - Carry out a temperature distribution analysis on the braking disk - Simulate the airflow - Perform a thermal stress analysis Materials to consider for the analysis are: - Carbon Ceramic discs - Cast Iron discs - Steel discs The ideal candidate should have extensive experience in thermal and fluid dynamics simulations as well as a deep understanding of COMSOL software. Knowledge of automotive braking systems is a plus as this would enhance the quality of analysis output. I look forward to a proficient professional who can provide valuable insights on best material composition for optimal braking ...
...there is a VSD fault, the AHU system should stop operating, and a red ‘VSD fault’ light shall illuminate at the front panel. Separate A temperature transmitters shall read the temperature inside and outside of the application and relay this information back to the HMI. Based on temperature inputs, two actuators shall open and close to control heating and cooling coil valves, thereby tempering airflow through the AHU. A simulated fire alarm switch on the front panel shall be activated or normal, and shall be indicated by a green ‘Fire Alarm Normal’ light on the front panel. The system should only be operational if fire alarm is in normal mode, and the green ‘Fire Alarm Normal’ light is illuminated. A frost stat is being simulated by a switch on ...
