Data Engineer

Job Duties :

Design, architect, develop, and optimize enterprise-scale data pipelines and data warehouse solutions using Apache Spark processing framework in Scala, Spark SQL, Realtime Streaming with Kafka & Spark Streaming, Apache Hadoop, Apache Hive. Build and optimize scalable ETL processes on Google Cloud Platform using GCS, Dataproc and BigQuery. Develop enterprise data models, establish data lineage and metadata management, and implement "Data Quality Assessment" based data quality controls. Configure CI/CD pipelines, orchestrate workflows with Airflow using Python, and perform performance tuning, monitoring, and root-cause analysis. Apply strong engineering practices through test-driven development, code reviews, and technical guidance to ensure scalable, efficient, and maintainable data solutions. Collaborate with cross-functional teams to gather requirements, translate them into technical designs, and deliver reliable, secure, and scalable data solutions. Work across databases such as Teradata, MySQL, SQL Server, Azure SQL, HBase and MongoDB. Integrate data solutions with Legacy Mainframe environments using COBOL, JCL and DB2.

Other :

Full Time. Travel (60%) involved to unanticipated client locations within US.

Requirements :

Bachelor's Degree in Information Technology or Equivalent + 5 years of Experience in job offered or related. Any suitable combination of education, training or experience accepted.

Location :

Alpharetta, GA Respond to: HR-Code AP, Pyramid Consulting, Inc., 3060 Kimball Bridge Road, Suite 200, Alpharetta, GA 30022 This position is eligible for our employee referral program

Job ID :

IM - SR