Big Data/Machine Learning Engineer Sr

Roles and Responsibilities End-to-End Pipeline Development:

Build, normalize, and enrich data pipelines that pull transaction data from various Lines of Business (LOBs) and publish them into OneLake and Snowflake.

Infrastructure Management:

Utilize AWS Glue and PySpark to develop and manage ETL processes within the AWS ecosystem.

Data Transformation & Optimization:

Decompose complex SQL logic into smaller, high-performing units to ensure pipelines are optimized for scale and speed.

Quality Assurance:

Perform thorough data quality checks and testing on all ingested data to ensure it meets the strict requirements of AML (Anti-Money Laundering) models and investigators.

SDLC Ownership:

Manage the full software development life cycle, including testing, validating data, and promoting code into production environments.

Production Support & On-Call:

Participate in an on-call rotation with the team to provide ongoing production support and maintenance for all pipelines. Top 5-6 required

Skills Python & PySpark:

Essential for building data pipelines.

AWS & AWS

Glue:

Proficiency in AWS compute/storage and Glue for ETL is required.

Snowflake & Delta Lake:

Experience with these data warehousing environments.

Strong SQL:

Critical for decomposing logic into smaller units and optimizing pipelines.

Data Pipeline Engineering:

Experience building, testing, and promoting code into production (full SDLC). Good to have/ desired skills Experience with OneLake, OneStream, and Clarity. Experience with wireless data ingestion/transformation. Dexian stands at the forefront of Talent + Technology solutions with a presence spanning more than 70 locations worldwide and a team exceeding 10,000 professionals. As one of the largest technology and professional staffing companies and one of the largest minority-owned staffing companies in the United States, Dexian combines over 30 years of industry expertise with cutting-edge technologies to deliver comprehensive global services and support. Dexian connects the right talent and the right technology with the right organizations to deliver trajectory-changing results that help everyone achieve their ambitions and goals. To learn more, please visit https://dexian.com/ . Dexian is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.