Senior Data Engineer

Note:

We are looking for a Senior Data Engineer / Big Data Engineer with strong expertise in building scalable data pipelines, Big Data technologies, and ML pipeline productionization, along with hands-on experience in the Hadoop ecosystem (HDFS, Hive, Spark, Kafka), cloud platforms, DevOps/CI-CD, Docker, Kubernetes, Microservices, REST APIs, and SQL/NoSQL databases. The ideal candidate should have 10+ years of overall IT experience. This position is available in multiple locations: Tampa, FL; Plano, TX; and Jersey City, NJ. We currently have both Full-time and Contract openings for this role.

Job Description :

Given the input from the Data Scientist and Data Modeler, create the data pipeline to ingest, cleanse and standardize the data so that it can be ready for analysis. Once the data is processed, apply the machine learning algorithm as prescribed by the Data Scientist with the production performance readiness. Finally, once the result is processed, activate the models through either Micro Services, Rporting, Dashboard, or emails Experience with Big Data Technologies or Cloud Technologies building data pipelines DevOPs Experience (source control, continuous integration, deployments, etc.) Experience with containerization and related technologies (e.g. Docker, Kubernetes) Experience in other open-sources like Druid, Elastic Search, Logstash etc Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr) System level understanding - Data structures, algorithms, distributed storage & compute Exposure and / or development experience in Microservices Architectures best practices Experience around REST APIs, services, and API authentication schemes Knowledge in RDBMS and NoSQL technologies