Data Engineer

Fort Mill, SC, United States (On-site) Contract (6 months) Published 19 hours ago

REST API

development MDM Architecture DevOps & CI/CD master data management apache kafka spring boot data pipelines Spark/Pyspark We are looking for a Senior Markit EDM Developer Technical SME with deep expertise in IHS Markit EDM and strong experience in modern cloud native event driven and data engineering ecosystems This role goes beyond traditional EDM development and requires ownership of end to end data mastering architecture integrating EDM with data lakes microservices APIs streaming platforms and DevOps pipelines

Key Responsibilities:

Markit

EDM CADIS

Core Development:

Lead design and development of enterprise MDM solutions on Markit

EDM CADIS

Architect and implement Core Matcher frameworks entity resolution survivorship rules Golden Record creation and lifecycle management Develop E2E data mastering pipelines including Data Constructors Data Porters Data Flows Rules Engines Ensure scalability reusability and performance optimization

Workflow UI Data Governance:

Design and develop Workflow UI components WF Pages WF Elements userdriven stewardship flows Implement data governance workflows approval exception handling audit trails Enable data lineage transparency and stewardship across domains especially

Party Client Event Driven Real Time Architecture:

Implement real time data processing and notifications using Apache Kafka Confluent Event driven EDM triggers Design near realtime mastering and synchronization patterns Integrate EDM into enterprise event ecosystems

API Microservices Integration:

Develop and expose RESTful APIs using Data Generators Process Launchers Integrate with microservices architecture Spring Boot Java Nodejs Enable API first EDM integration strategy across systems

Cloud Modern Data Platform:

Deploy and manage EDM solutions on AWS or Azure equivalent AWS Glue ETL orchestration Lambda serverless triggers S3 Data Lake integration RDS Aurora SNSSQS eventing Integrate EDM with modern data platforms Snowflake Databricks Redshift Design hybrid architecture EDM golden source Data Lake analytical layer

Data Engineering Advanced Processing:

Work with Python PySpark Spark SQL for Data transformation Enrichment pipelines Largescale data processing Handle ingestion from APIs flat files streaming feeds databases Optimize large volume batch and streaming pipelines

DevOps CICD Infrastructure:

Build and maintain CICD pipelines for EDM GitHub GitLab Octopus Deploy Jenkins Azure DevOps Use Infrastructure as Code IaC Terraform CloudFormation Implement Automated deployments Environment promotion Version control strategies

Containerization Platform Engineering:

Work with Docker Kubernetes EKSAKS for Microservices deployment Integration layers around EDM Enable scalable resilient cloud native deployments

Observability Performance Engineering:

Implement monitoring and logging using Splunk ELK Prometheus Grafana CloudWatch Perform Performance tuning of matchers and batch jobs Root cause analysis and production support

Database Data Modeling:

Design and optimize across SQL Server Oracle PostgreSQL Snowflake cloud native warehouses Develop Complex SQL Stored procedures Indexing and performance tuning strategies

Leadership Delivery:

Lead onsite/offshore model delivery Act as technical SME and solution architect for EDM Collaborate with: Business Analysts Data Architects Scrum Teams Conduct Code reviews Design reviews Release governance

Required Qualifications:

5-10 years of experience in MDM with 5 years in Markit EDM IHS Strong expertise in: Core Matcher design Data mastering pipelines Workflow UI development Hands-on experience with: Kafka or equivalent streaming AWS cloud services

REST API

development Strong database experience across RDBMS cloud data warehouses Experience with CICD Git Terraform Proficiency in Python or Spark based data processing

Preferred Qualifications:

Experience with Markit EDM Cloud 19x Strong domain knowledge in: Party Client Instrument MDM Experience with: Databricks Snowflake ecosystems Microservices architecture Spring Boot Nodejs Kubernetes containerization Exposure to data governance frameworks Collibra Alation etc

Key Competencies:

Deep expertise in Markit EDM modern data ecosystem Strong architecture and system design mindset Ability to lead complex data platform transformations Excellent communication and stakeholder management Strong problem solving and performance optimization skills The pay range that the employer in good faith reasonably expects to pay for this position is $34.67/hour - $54.18/hour. Our benefits include medical, dental, vision and retirement benefits. Applications will be accepted on an ongoing basis. Tundra Technical Solutions is among North America's leading providers of Staffing and Consulting Services. Our success and our clients' success are built on a foundation of service excellence. We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic. Qualified applicants with arrest or conviction records will be considered for employment in accordance with applicable law, including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.

Unincorporated LA County workers:

we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: client provided property, including hardware (both of which may include data) entrusted to you from theft, loss or damage; return all portable client computer hardware in your possession (including the data contained therein) upon completion of the assignment, and; maintain the confidentiality of client proprietary, confidential, or non-public information. In addition, job duties require access to secure and protected client information technology systems and related data security obligations.