Data Engineer Introduction As a Data Engineer, you will be responsible for designing, building, and maintaining the infrastructure that supports the extraction, transformation, and loading of data from various sources into a data warehouse. You will work closely with data scientists, analysts, and other stakeholders to ensure that the data is accessible, accurate, and secure. Responsibilities Develop and maintain ETL processes to move data from source systems to the data warehouse Design and implement data pipelines for real-time and batch processing Optimize data pipelines for performance and scalability Collaborate with data scientists and analysts to understand their data requirements Ensure data quality and integrity through data validation and testing Identify and address performance bottlenecks in the data infrastructure Implement data security and privacy measures to protect sensitive information Document data infrastructure and processes for knowledge sharing

Requirements Required Skills:

Proficiency in SQL and one or more programming languages (e.g. Python, Java) Experience with ETL tools and data processing frameworks (e.g. Apache Spark, Talend) Knowledge of data modeling and database design principles Strong problem-solving and analytical skills Ability to work independently and collaboratively in a team environment

Preferred Skills:

Experience with cloud data services (e.g. AWS, Google Cloud Platform) Familiarity with big data technologies (e.g. Hadoop, Hive) Certifications in data engineering or related field Knowledge of machine learning and data visualization tools