A Data Engineer designs, builds and manages the information or big data infrastructure. Develops the architecture that helps analyze and process data in the way the organization needs it. Makes sure those systems are performing smoothly.
East Hanover, NJ (Hybrid) Job Description Leverage analysis design techniques to solve business problems using information technology to operationalize data and analytical models to drive long-term business results.
You will:
Partner with MDS in providing accessibility, retrievability, security and protection of data in an ethical manner. Parter with Product Owners, BI Developers and Business Leads to conduct business requirement gathering & continuous improvement sessions to support the business and understand requirements for analytical projects and develop necessary requirement documents and acceptance criteria to support their development in line with process guidelines Develop UAT (User Acceptance Testing) and SIT (System Integration Testing) test cases based off analytical data product requirements. Support for scoping, building, testing, and maintaining transport/data pipelines and retrieving applicable data sets for specific use cases. Support building, testing, and maintaining data models with the flexibility to change when business requirements change, while also reconciling multiple logical source models into a single, logically consistent model. Analyze the structure, format, and reliability of new and existing data sources, considering factors such as data integrity, completeness, accuracy, and drive continuous improvement. Participate in data governance processes to ensure data is properly maintained and document for reusability by the business Understand data and metadata to support consistency of information retrieval, combination, analysis, pattern recognition and interpretation. What you will bring Understanding of Data Engineering concepts within a business setting, including working with multiple systems such as SAP, internal and external data sources. Experience in developing, enhancing, testing, and maintaining data products. Experience in using diverse range of languages and tools, including scripting languages, for data extraction, transformation, storage, processing, and integration. Analyze business requirements as a guide for data modeling and apply data analysis, design, modeling, and quality assurance techniques, continuous improvement, based on a detailed understanding of business processes, to establish, modify or maintain data structures and associated components (entity descriptions, relationship descriptions, attribute definitions). Ability to simplify complex problems and effectively communicating technical concepts to a diverse audience. More about this role Cloud computing technologies (Azure, GCP)
Programming Languages:
SQL, Python, PySpark (Apache Spark), DAX, etc.
Analytics platforms:
Databricks, Google Cloud Dataproc, etc. Experience with CI/CD, DevOps, and DataOps techniques. Experience with ETL, Data pipelines and architecture Technical Documentation Experience in data modelling and wrangling techniques and tools i.e., Star Schema, Snowflake Schema, Data vault, Alteryx, Power BI, Tableau, etc. Data quality and profiling methods Education /
Certifications:
Bachelor's degree in a related field. BS Computer Science, Information Systems, etc.
Job specific requirements:
1+ years Data Engineering/Modeling 3+ years Database Design
