Data Engineer

Our client is looking Data Engineer in San Diego, CA below is the detailed requirements.

Job Title :

Data Engineer Location :

San Diego, CA Job description: Bachelor's degree in Computer science or equivalent, with minimum 12+ Years of Overall IT experience. Should have Strong Spark + Spark SQL + hands-on performance tuning (not only SQL writing) Should have Python for Spark/data engineering. Design, build, and performance-tune Apache Spark workloads using Spark SQL and PySpark for complex transformations (JSON/semi-structured data, nested structures, window functions, joins, aggregations).

Profile and optimize Spark jobs:

partitioning, shuffles, join strategies, skew, memory/spill, and right-sized resource usage-especially on EMR Serverless-for large-scale and petabyte-scale data. Support Customers and Monitor Pipelines with Strict SLA for Fixs and Re Instating Issues around the clock. Implement reusable patterns for incremental loads, deduplication and CDC-style processing. Build and maintain ETL/ELT on AWS EMR Serverless (Spark), with S3 as the data lake layer: partitioning, compression, external tables, and layouts that support fast Spark and downstream SQL. Should have

AWS, EMR

Serverless, S3 (delta and data lake patterns), Redshift (SQL + tuning). Should be Familiar with access control concepts for data platforms (AWS IAM, lake/warehouse permissions, RLS / column-level security as applicable). Strong analytical and problem-solving skills, with a proven track record of identifying and resolving complex billing issues. Excellent communication and presentation skills, with the ability to explain complex technical concepts to both technical and non-technical audiences.