Databricks Lead

Databricks Lead Tata Consultancy Services - 3.9 Raleigh, NC Job Details $100,000 - $140,000 a year 9 hours ago Qualifications Data model design Performance tuning Power BI Databricks Data visualization software proficiency Data Integration (Data management) Azure Computer Science Data lake Continuous Delivery (CD) implementation Data modeling projects Spark Scalable systems Automating deployment processes SQL Bachelor's degree Cloud-based systems Scala Data quality monitoring Data analytics Scalability Senior level Cross-functional collaboration Bachelor's degree in computer science Communication skills Python 10 years Database software proficiency Full Job Description Must Have Technical/Functional Skills 10+ years overall experience in data engineering or related fields. 3-5 years hands-on experience with Databricks and Spark. Strong proficiency in SQL and data analysis techniques. Experience with ETL processes, data modeling, and performance tuning. Familiarity with Python or Scala for data engineering tasks. Excellent problem-solving and communication skills. Roles & Responsibilities We are seeking a hands-on Databricks Data Engineer to design, develop, and optimize data pipelines and analytics solutions. The ideal candidate will have strong experience in data engineering, ETL development, and production support, ensuring reliable, scalable, and high-performing data operations within Azure environment and can work in a fast-paced environment. Knowledge of insurance domain and Power BI is a plus but not mandatory. Development Design, develop, and deploy scalable ETL/ELT data pipelines using Apache Spark, PySpark, and Databricks. Develop and optimize SQL queries for data transformation and analysis. Collaborate with product owners, data architects and analysts to build data models, delta lake structures, and data workflows. Collaborate with data analysts and business teams to deliver actionable insights. Build job orchestration and monitoring solutions Ensure data quality, performance, and reliability across workflows. Develop and maintain CI/CD pipelines for Databricks notebooks, jobs, and workflows. Work with cloud-based data platforms (Azure preferred).

Required Skills & Experience:

10+ years overall experience in data engineering or related fields. 3-5 years hands-on experience with Databricks and Spark. Strong proficiency in SQL and data analysis techniques. Experience with ETL processes, data modeling, and performance tuning. Familiarity with Python or Scala for data engineering tasks. Excellent problem-solving and communication skills.

Nice-to-Have:

• Knowledge of insurance industry data.•

Salary Range:

$100,000 to $140,000 per year Location Raleigh, NC Job Function

TECHNOLOGY

Role Lead Job Id 410599 Desired Skills Python Salary Range $100,000-$140,000 a year

Desired Candidate Profile Qualifications :

BACHELOR OF COMPUTER

SCIENCE