Skip to main content
Tallo logoTallo logo
Apply for this opportunity

This job application is on an outside website. Be sure to review the job posting there to verify it's the same.

Apply Offsite

Lead Data Engineer

Job

METANLYTICS LLC

Malvern, PA (In Person)

Full-Time

Posted 3 days ago (Updated 16 hours ago) • Actively hiring

Expires 7/10/2026

Review key factors to help you decide if the role fits your goals.
How is this calculated?
Pay Growth
?
out of 5
Not enough data
Not enough info to score pay or growth
Job Security
?
out of 5
Not enough data
Calculating job security score...
Total Score
97
out of 100
Average of individual scores

Were these scores useful?

Skill Insights

Compare your current skills to what this opportunity needs—we'll show you what you already have and what could strengthen your application.

Job Description

Lead Data Engineer Location:
Malvern, PA Experience:
8-10+ Years Must-Have Skills Python AWS (S3, Lambda, Glue) PySpark SQL DynamoDB Kafka / Kinesis GenAI & LLMs Advanced Java / Core Java Job Responsibilities Design and build scalable data engineering solutions on AWS. Develop batch and real-time data pipelines using PySpark, Kafka, and Kinesis. Work on GenAI and Large Language Model (LLM) based applications. Develop and optimize SQL queries for large-scale data processing. Design distributed data processing systems using Spark or similar technologies. Build and maintain data workflows using Airflow or Prefect. Implement CI/CD pipelines and Infrastructure as Code (Terraform, Docker, Kubernetes). Ensure data security, governance, monitoring, and reliability. Lead technical design discussions, code reviews, and mentor junior engineers. Preferred Skills Apache Spark or Ray Airflow / Prefect Docker, Kubernetes, Terraform Snowflake, Redshift, BigQuery Event-Driven Architecture Data Governance & Security Ideal Candidate Strong hands-on experience in Python and AWS. Experience building large-scale Data Engineering platforms. Good exposure to GenAI and LLM technologies. Strong knowledge of Kafka/Kinesis and real-time data processing. Ability to lead architecture decisions and mentor development teams. Top 3 Skills Python + PySpark AWS (S3, Lambda, Glue, DynamoDB) GenAI / LLMs with Kafka/Kinesis Data Engineering