Job Title:
Senior Data Engineer (Databricks & Cloud Analytics)
Location:
Salt Lake City, Utah Work Model:
Hybrid - onsite and remote Responsibilities
- Design, develop, and support enterprise data platforms, cloud analytics solutions, or large-scale data engineering initiatives.
- Work hands-on with Databricks, Apache Spark (PySpark), Python, SQL, and Scala to build scalable data pipelines and analytics solutions.
- Manage and implement Databricks ecosystem components, including Workspaces, Delta Lake, Unity Catalog, Delta Live Tables, Databricks SQL, MLflow, and Databricks Jobs.
- Optimize SQL queries and improve query performance through tuning and best practices.
- Design and implement ETL/ELT processes utilizing cloud platforms such as Azure Data Factory.
- Work with event streaming technologies like Apache Kafka and integrate enterprise applications via REST APIs and web services.
- Utilize Azure Data Lake Storage (ADLS Gen2) and other cloud storage solutions.
- Collaborate within Linux environments, including shell scripting and command-line utilities.
- Use source control tools like Git for collaborative development.
- Participate in Agile teams practicing Scrum, Kanban, or SAFe methodologies.
- Provide technical mentoring and leadership within data engineering teams. Requirements
- Bachelor's degree in Computer Science, Information Systems, Engineering, or related field or equivalent professional experience.
- At least 6 years of experience designing and supporting enterprise data platforms or cloud data solutions.
- Extensive hands-on experience with Databricks, Apache Spark, Python, SQL, and Scala.
- Proven SQL development skills, including query optimization and performance tuning.
- Experience designing ETL/ELT solutions with cloud platforms such as Azure Data Factory.
- Knowledge of Kafka or similar event streaming systems.
- Familiarity with cloud data storage (Azure Data Lake Storage or equivalents).
- Proficiency using Git for version control.
- Experience working in Linux environments and scripting.
Preferred:
Infrastructure as Code (Terraform), CI/CD pipelines (Azure DevOps, GitHub Actions).
- Strong analytical and troubleshooting skills.
- Prior experience working in Agile methodologies.
- Ability to prioritize multiple tasks independently and mentor team members.
#M- #
LI- Ref:
#404-IT Pittsburgh