Job Description
Client:
FHL Bank (Federal Home Loan Bank of Chicago) # R2600246
Job Title:
Sr Data Engineer Duration :
6+ Months Contract to Hire Location:
Chicago, IL. Job Description:
Key Responsibilities Data Engineering & Pipeline Development:
Design, develop, and maintain end-to-end data pipelines in Databricks using Spark and Delta Lake Build and optimize ELT/ETL processes for structured and unstructured data ingestion into the Data Lakehouse Implement scalable ingestion patterns (batch and event-driven) from internal systems, third-party APIs, and cloud sources Develop data models (bronze, silver, gold layers) to support enterprise reporting, analytics, and downstream consumption Data Platform & Integration:
Integrate the Data Lakehouse with enterprise tools such as Tableau, Alteryx, and machine learning platforms Design and implement data access controls, identity management, and secure data sharing mechanisms Support API-based integrations and downstream data consumption patterns Data Quality, Governance & Controls:
Implement data quality checks, reconciliation processes, and monitoring within Databricks pipelines Ensure adherence to enterprise data governance standards, including lineage, metadata, and audit requirements Support regulatory and compliance requirements (e.g., data integrity, privacy, and security controls) Cloud & Automation:
Develop and manage workflows using orchestration tools (e.g., Airflow, Control-M) Automate data pipelines, deployments, and operational processes through CI/CD pipelines Leverage cloud-native services (AWS/Azure) for data processing, storage, and event-driven architectures Operations & SupportMonitor, troubleshoot, and optimize data pipelines and Spark workloads for performance and reliability Support production data platforms, including incident resolution and root cause analysis Ensure high availability, data integrity, and SLA adherence across enterprise data systems Collaboration:
Partner with data architects, data scientists, BI teams, and business stakeholders to deliver data solutions Participate in Agile ceremonies and contribute to iterative delivery of data products Translate business requirements into scalable technical data solutions Required Qualifications3+ years of experience in data engineering, data platforms, or related roles Hands-on experience with Databricks, Apache Spark (PySpark), and Delta Lake Strong SQL and data modeling skills (relational and dimensional) Experience building and supporting data pipelines in a cloud environment (AWS or Azure) Experience with ELT/ETL tools (e.g., Fivetran, custom ingestion frameworks) Familiarity with data orchestration tools (Airflow, Control-M) Experience working in Agile development environments Preferred Qualifications:
Experience in financial services or regulated environments (e.g., banking, risk, regulatory reporting) Knowledge of data governance frameworks and tools (e.g., Collibra) Experience with real-time or streaming data pipelines Exposure to machine learning pipelines and feature engineering in Databricks Cloud certifications (AWS, Azure, or Databricks) Technical Skills:
Databricks (Lakehouse architecture, notebooks, jobs, Unity Catalog) Spark / PySpark SQL (advanced querying and optimization)
