4101078
Job Summary
We are seeking a Data Engineer with strong expertise in Big Data technologies, SQL, workflow orchestration, and analytics platforms. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and supporting data driven decision making.
Key Responsibilities
Design and build scalable data ingestion, transformation, and processing pipelines.
Develop ETL/ELT workflows for large scale structured and unstructured datasets.
Work with distributed data processing frameworks such as Spark and Hadoop.
Optimize SQL queries and data models for performance and scalability.
Develop and manage workflow orchestration using Airflow, Dataswarm, or similar tools.
Support reporting and analytics through dashboarding and visualization platforms.
Collaborate with cross functional teams to deliver high quality data solutions.
Ensure data quality, reliability, governance, and operational excellence.
Required Skills
Strong experience with Apache Spark, Hadoop, and Big Data ecosystems.
Proficiency in SQL, including Presto, Hive, and SparkSQL.
Experience building and maintaining ETL/ELT pipelines.
Hands on experience with Airflow, Dataswarm, or similar orchestration tools.
Experience with visualization tools such as Tableau, Unidash, Power BI, or similar platforms.
Strong understanding of data warehousing, data modeling, and performance tuning.
Proficiency in Python or other scripting languages.
Experience working with cloud based data platforms is preferred.
Preferred Skills
Knowledge of AWS, GCP, or Azure data services.
Experience with Data Lake/Lakehouse architectures.
Familiarity with CI/CD, Agile, and DevOps practices.