Lead Data Engineer Position Available In New York, New York
Tallo's Job Summary: This job listing in New York - NY has been recently added. Tallo will add a summary here for this job shortly.
Job Description
Lead Data Engineer
ABOUT US SG
Analytics (SGA), a Global Insights and Analytics company, focusses on ESG, Data Analytics, and Investment & Market research services.
The company has presence in New York, San Francisco, Austin, Seattle, Toronto, London, Zurich, Pune, Bengaluru, and Hyderabad and growing consistently for the last few years.
SGA is a Great Place To Work (GPTW) certified company, and with its thriving work environment shaped by a growth mindset, abundant learning & collaboration opportunities, and a meritocracy-driven culture, SG Analytics has also been awarded regional best employer in 2016, 2018 & 2020.
Job Overview:
We seek a highly skilled and motivated Data Engineer with strong expertise in Python, PySpark, AWS, Databricks, and Snowflake to join our dynamic team. The ideal candidate should have hands-on experience with Spark optimization, SQL data processing, and AWS tools and technologies. Exposure to Informatica and data streaming tools is a plus.
Location:
New York
Must Have Skills _
Data Engineering, Python, Pyspark, SQL, AWS, Snowflake, Data Bricks Key Responsibilities:
Design, develop, and optimize scalable data processing pipelines using Spark and PySpark.
Implement Spark optimization techniques to enhance performance and efficiency.
Work with complex SQL queries for data manipulation and transformation.
Collaborate with data scientists, analysts, and engineering teams to meet data needs.
Work on AWS Cloud services like Redshift, AWS Glue, SQL Server, and Databricks.
Utilize Python for data processing tasks, ensuring modularization and packaging.
Basic knowledge of Informatica.
Engage in real-time data streaming and integration using tools such as NiFi, Kafka, and EventHub (optional but preferred).
Hands-on experience with Snowflake for data warehousing and analytics.
Required Skills:
Spark Expertise:
Strong experience in Spark and PySpark, including optimization techniques.
Complex SQL:
Advanced SQL knowledge and data processing skills.
Cloud Exposure:
Familiarity with AWS services and Databricks.
Python Proficiency:
Experience with Python for data processing and scripting.
Data Streaming:
Experience with NiFi, Kafka, and EventHub is desirable.
Snowflake:
Hands-on experience with Snowflake for data warehousing and analytics.
Media Domain
– Experience is Media domain is plus.
Qualifications:
Bachelor s or Master s degree in Computer Science, Information Technology, or related fields.
8-15 years of professional experience in Data Engineering, Data Analytics, or related roles.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job
Dice Id:
90962613
Position Id:
8675506
