My main area of expertise includes Data Engineering (Small or Big Data level), Data Mining, ETL/ELT Data Pipelines, Data Automation and Automated Bots/Crawlers. I have experience of handling up to 100s of billions of records and 100s of TBs of structured and unstructured data; migration to Big Data/SQL data stores and handling real-time data streams. I have experience of large scale data collection/scraping from web, I developed countless crawlers in past and several includes data collection up to 100s of millions of records.
Other areas of expertise includes Full Stack Data Science (Machine Learning/Deep learning). I also have a strong software engineering background and have experience in web development (MVC frameworks, Custom Development). I have skills to develop from scratch and customize exiting software/web applications.
I have solution providing skills and can adjust myself to any technical stack, programming language or framework to provide the solution. Following is a brief list of technical stacks, programming languages and tools that I have expertise or used in past:
Languages:
Python (DJango, Flask), PHP (Laravel, Yii, Zend), Javascript (Angular, Typescript, Vue.js, Node.js), Java, C/C++, Perl, ASP, Matlab
------------------
Databases/Data Streaming:
PostgreSQL, MySQL, MariaDB, SQL Server, SSIS, Oracle, NoSQL (Redis, HBase, Hive, MongoDB), Kinesis, Kafka, RabbitMQ, Solr
------------------
Big Data Ecosystem:
Hadoop, HBase, Athena/Hive, Spark, Redshift
------------------
Cloud:
AWS (EC2, ECS, RDS, S3, Redshift, Kinesis, ElastiCache/Redis, Lambda, Cloudwatch), GCP
------------------
Data Science/AI:
Machine Learning (Classification, Clustering, Regression, SVM, K-NN, Naive Bayes, Random Forest, Linear Regression, Auto Regression), Deep Learning (Autoencoders, Stacked Autoencoders, CNN etc), Scikit-Learn, Scipy, Pandas, Data Wrangling, Web Scraping, Crawlers
------------------
Customization Experience:
Opencart, Magento, Wordpress, Drupal, Zencart, other open source/custom made web applications
------------------
Other Skills:
DevOps, CI/CD, CircleCI, Ansible, Docker, Deployer, Datadog, Openwhisk, Celery, Nutch, Thrift, Selenium, Jupyter Notebook, Git, REST Apis, SSH, TDD, Robotics, Good in adopting new stacks
------------------
Management:
Agile, Scrum, Jira, Trello