Databricks Data Engineer

Databricks Data Engineer Tata Consultancy Services - 3.9 Irving, TX Job Details $125,000 - $140,000 a year 21 hours ago Qualifications Databricks Spark System performance optimization Scalable systems System design for system development Scalability Distributed computing System tuning Data performance optimization Python Design (software development lifecycle)

Full Job Description Roles & Responsibilities Job Title:

Databricks Data Engineer Job Description:

As a Databricks Data Engineer, you will be responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing in Big Data environment using predominantly PySpark/Python. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across systems using PySpark.

Roles & Responsibilities:

Collaborate closely with data scientists, data engineers, and business stakeholders to gather requirements and understand the business objectives driving data pipeline development. Design, develop, and maintain robust, scalable high-performance Data Pipelines using Databricks. Leverage Databricks features such as Lakehouse and Delta Lake for efficient data storage and Spark for distributed processing Develop ETL/ELT pipeline using Databricks Monitor pipeline health, troubleshoot data issues Migrate on Prem Pyspark, SAS data pipeline and ML Models to Databricks Define and implement best practices in Databricks Evaluate new Databricks features and tools, helping the organization stay at the forefront of innovation in AI and Big Data Collaborate with cross-functional teams to identify and resolve data-related issues.

Qualifications:

Proven expertise in implementing Lakehouse and Delta Lake using Databricks. Strong PySpark and Python experience Databricks Certified Data Engineer Professional Certification Familiarity with ML Ops/LLM Ops and distributed systems. Experience with Big Data platform like Cloudera Hadoop and Could platforms like

AWS, GCP.

Solid understanding of system design patterns, scalability, observability, and performance tuning. Strong analytical and problem-solving skills. Passion for exploring and building with emerging technologies. Good to

Have Skills:

AWS EKS Experience, Dockers and Containers Salary Range:

$125,000 to $140,000 per year Location Irving, TX Job Function

TECHNOLOGY

Role Engineer Job Id 416705 Desired Skills Data Analytics Salary Range $125,000-$140,000 a year

Desired Candidate Profile Qualifications :

BACHELOR OF COMPUTER

SCIENCE