Compare your current skills to what this opportunity needs—we'll show you what you already have and what could strengthen your application.
Job Description
Position:
Data Engineer /
Data Modeller/ Senior Advisor Location:
Morris Plains, NJ (hybrid 3 days week)
Location:
12+ Months Job description
Duties:
We are seeking an experienced Engineer to join a team of highly talented professionals responsible for designing, building, and testing (including automation scripts for unit, integration, and performance testing) Individual & Family Retail Health Insurance applications and their components that interface with complex enterprise systems and external vendor systems. The role requires strong expertise in distributed applications, data integration, and modern technologies, with a passion for solving complex challenges and staying current with industry trends. The ideal candidate is a motivated, well-rounded team player and self-starter who thrives in a collaborative, dynamic, agile environment. Excellent written and verbal communication skills are essential, as this position interacts with remote scrum teams, business owners, enterprise architects, security, infrastructure, and end users through various channels.
Key Qualities Analytical Skills:
Ability to understand customer needs and create simple, effective solutions. Ability to architect and design solutions ground-up.
Communication:
Clear articulation of ideas to peers, stakeholders, and management.
Creativity:
Innovative approach to problem-solving and application development.
Customer Service:
Strong client-facing skills and consultant mindset. Attention to
Detail:
Ensure all application components work seamlessly together.
Problem-Solving:
Make decisions that move projects forward effectively.
Teamwork:
Collaborate within distributed agile (SAFe) teams.
Technical Skills:
Proficiency in programming languages and design patterns.
Leadership:
Lead by example, influence technical direction, and advocate for shared vision.
Responsibilities:
Design, develop, and maintain scalable data pipelines using AWS services and Databricks for extracting, transforming, and loading data into lakes or downstream systems. Ensure data accuracy, quality, and reliability for analytics and reporting. Implement monitoring systems to proactively resolve pipeline issues. Integrate multiple internal/external systems, mobile apps, and data marts for a 360-degree customer view. Apply test-driven development with a strong focus on automation. Support and enhance existing health insurance applications for consumers and operations staff. Participate actively in all agile ceremonies.
Skills:
Required Skills & Experience:
8+ years experience architecting, designing solutions ground-up Proven expertise in AWS Cloud Services (Glue, ECS/EKS, Fargate, Redshift/Aurora/DocumentDB, ECR, S3, Lambda, SNS, IAM). Proficiency in Databricks DLT (Delta Live Tables), SQL, Spark, Python/PySpark, Java. Experience with Databricks orchestration frameworks (internal workflows or Apache Airflow). Strong understanding of data integration strategies, ELT/ETL techniques, and data warehousing principles. Hands-on experience with data transformation, quality, and observability for production-grade pipelines. Familiarity with reporting/BI tools (e.g., Tableau) and data governance tools (e.g., Alation, Collibra). Solid grasp of design patterns, performance tuning, and DevOps practices.
Desired:
Migration and setup of large-scale cloud data pipelines using Databricks. Expertise in partitioning strategies, metadata management, and file formats (Delta Lake, Iceberg, Apache Hudi). Experience with document databases, JSON/XML data, and real-time ingestion. Knowledge of Lakehouse architecture, dimensional modeling, and Data Vault. Conversion of on-prem pipelines to Databricks DLT for cloud efficiency.
Keywords:
Education:
Bachelor''s degree in Information Systems or any advanced degree (e.g., MBA) or equivalent work experience
Skills and Experience:
Required Skills:
AWS CLOUD SERVICES GLUE ECS EKS FARGATE A
Additional Skills:
REDSHIFT AURORA DOCUMENTDB ECR S3 LAMBDA SNS IAM DATABRICKS DLT DELTA LIVE TABLES SQL SPARK PYTHON PYSPARK JAVA DATABRICKS ORCHESTRATION FRAMEWORKS APACHE AIRFLOW DATA INTEGRATION STRATEGIES ELT TECHNIQUES ETL TECHNIQUES DATA WAREHOUSING PRINCIPLES DATA TRANSFORMATION DATA QUALITY DATA OBSERVABILITY TABLEAU ALATION COLLIBRA DESIGN PATTERNS PERFORMANCE TUNING DEVOPS PRACTICES PARTITIONING STRATEGIES METADATA MANAGEMENT FILE FORMATS DELTA LAKE ICEBERG APACHE HUDI
