Job Description
Project Description :
We are seeking an experienced, detail-oriented IT Big Data Developer to expand and optimize our data and data pipeline architecture, as well as optimize data flow and collection for economic policy and research teams. The ideal candidate is a hands-on data modeler with working knowledge of database design and administration, data pipeline building, and data wrangling who thrives on improving existing data systems and building new ones from the ground up. This role will support economists and technical experts, ensuring optimal data delivery architecture is designed and developed. The right candidate will be excited by the prospect of optimizing or re-designing the division's data architecture to support the next generation of data initiatives. Qualification Requirements :
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field with at least 7 years of related experience; advanced degree preferred Advanced working knowledge of SQL and relational database platforms including PostgreSQL, Microsoft SQL Server, and MySQL Advanced working knowledge of Python, R, and other scripting languages used for data engineering and analytics Experience with large-scale data systems including distributed computing, scalable data processing, data storage architecture, and high-volume data workload optimization Experience designing, developing, and automating ETL/ELT workflows and data integration pipelines Experience with workflow orchestration and pipeline automation tools such as Apache Airflow, Prefect, Dagster, or AWS Step Functions Experience migrating workflows and data pipelines between on-premises and cloud environments Experience developing in Linux environments and using source control platforms such as GitLab and/or GitHub In-depth experience designing and implementing database, data lake, and enterprise data platform solutions Strong hands-on software engineering experience including development, testing, and deployment of data applications and services Experience working in a research environment and/or with economic or financial data Working experience with cloud technologies such as AWS, Microsoft Azure, and Snowflake Experience implementing data warehouses utilizing Change Data Capture (CDC) methodologies Experience implementing and maintaining CI/CD pipelines and DataOps platforms Skills Requirements :
SQL - advanced proficiency (PostgreSQL, Microsoft SQL Server, MySQL) Python and R for data engineering and analytics ETL/ELT workflow design, development, and automation Workflow orchestration tools: Apache Airflow, Prefect, Dagster, or AWS Step Functions Distributed computing and scalable data processing architecture Database, data lake, and enterprise data platform design and implementation Cloud platforms: AWS, Microsoft Azure, Snowflake Linux development environments and source control (GitLab, GitHub) NoSQL and graph database technologies Machine learning model development, training, deployment, and maintenance Change Data Capture (CDC) methodologies CI/CD pipelines and DataOps platforms Time series data analysis and forecasting techniques Structured and unstructured data processing and integration Additional programming languages: Java, Scala, JavaScript, or Perl Excellent oral and written communication with a strong customer service orientation Exceptional analytical, problem-solving, and troubleshooting skills Responsibilities :
Expand and optimize data and data pipeline architecture to support economic policy and research teams Design and develop optimal data delivery architecture to meet multi-team and multi-system data needs Build, optimize, and maintain scalable databases, data pipelines, and data processing frameworks Design, develop, and automate ETL/ELT workflows and data integration pipelines Process, analyze, and integrate structured and unstructured data sources Design and communicate enterprise information architecture at conceptual, logical, and physical levels Perform root cause analysis on internal and external data and business processes to identify opportunities for improvement Migrate workflows and data pipelines between on-premises and cloud environments Develop, train, deploy, and maintain machine learning models Implement and maintain CI/CD pipelines and DataOps platforms Support economists and technical experts with their data needs across multiple teams and systems Job ID :
1553
