Pay Growth
?
out of 5
Not enough data
Not enough info to score pay or growth
GCP Data Engineer
Job
Compunnel, Inc.
Lisle, IL (In Person)
Full-Time
Review key factors to help you decide if the role fits your goals.
Job Security
?
out of 5
Not enough data
Calculating job security score...
Total Score
80
out of 100
Average of individual scores
Skill Insights
Compare your current skills to what this opportunity needs—we'll show you what you already have and what could strengthen your application.
Job Description
JOB SUMMARY
This role involves collaborating with business, IT, Analyst, and Data Science groups to define data requirements. The position requires designing, developing, deploying, and supporting high-performance data pipelines, modeling data platforms, and ensuring data quality. It also includes developing REST APIs for data exposure and mentoring junior data engineers. Key Responsibilities- Collect business requirements by working closely with various business, IT, Analyst, and Data Science groups.
- Design, develop, deploy, and support high-performance data pipelines, both inbound and outbound.
- Model the data platform by applying business logic and building objects in the semantic layer.
- Optimize data pipelines for performance, scalability, and reliability.
- Implement CI/CD pipelines for continuous deployment and delivery of data products.
- Ensure the quality of critical data elements, prepare data quality remediation plans, and collaborate to fix quality issues.
- Document the design and support strategy of data pipelines.
- Capture, store, and socialize data lineage and operational metadata.
- Troubleshoot and resolve data engineering issues.
- Develop REST APIs to expose data to other teams.
- Mentor and guide junior data engineers. Required Qualifications
- 6+ years of experience in data engineering solutions such as data platforms, ingestion, data management, or publication/analytics.
- 2+ years of experience in Google Cloud Platform (GCP) with services like BigQuery, Composer, GCS, DataStream, or Dataflows.
- Expert knowledge of SQL and Python programming.
- Experience working with Airflow as a workflow management tool, including building operators to connect, extract, and ingest data.
- Experience in tuning queries for performance and scalability.
- Experience in real-time data ingestion using GCP Pub/Sub, Kafka, Spark, or similar technologies.
- Proven experience working in an agile environment.
- Proven experience working in incremental execution through successful launches. Preferred Qualifications
- None specified in the provided text. Certifications
- None specified in the provided text.