Job Description Design, develop, and maintain scalable ETL/ELT pipelines that ingest, transform, and serve structured and unstructured data from enterprise systems, documents, communications, and regulatory sources.

Build and optimize data architectures, lakehouse solutions, and curated data models to support analytics, machine learning, and generative AI applications.

Create document ingestion, metadata extraction, indexing, and retrieval pipelines that enable enterprise search, knowledge management, and RAG-based solutions.

Partner with Data Scientists and Applied Scientists to operationalize AI and machine learning solutions through reliable, high-quality data infrastructure.

Develop data solutions supporting regulatory intelligence, compliance reporting, requirements extraction, and impact assessment workflows.

Implement data quality, governance, lineage, monitoring, and security controls to ensure trusted and auditable enterprise data assets.

Optimize data processing performance, scalability, and reliability across cloud-based environments.

Collaborate with cross-functional stakeholders to translate business requirements into scalable technical solutions.

Drive engineering best practices around testing, CI/CD, observability, documentation, and operational excellence. Skills and Requirements Bachelor's degree in Computer Science, Data Engineering, Information Systems, Software Engineering, or related field; equivalent experience considered.5+ years of experience in Data Engineering or related data platform roles.

Advanced proficiency in SQL and Python.

Experience building large-scale ETL/ELT pipelines and data integration solutions.

Hands-on experience with Azure data technologies including:

Azure DatabricksAzure Data Factory (ADF)Microsoft FabricAzure Synapse AnalyticsAzure Data Lake Storage (ADLS)Experience working with structured, semi-structured, and unstructured data sources.

Strong understanding of data modeling, data warehousing, and lakehouse architectures.

Experience implementing data governance, data quality, and lineage frameworks.

Experience supporting analytics, machine learning, or AI solutions through scalable data infrastructure.

Strong communication skills and ability to work across technical and non-technical teams.

Experience supporting Generative AI, RAG, LLM, or enterprise search solutions.

Experience building document ingestion and knowledge management platforms.

Familiarity with vector databases, embeddings, and retrieval architectures.

Knowledge of Azure AI Search, Azure OpenAI, or similar AI services.

Experience working with legal, compliance, regulatory, governance, or risk-focused organizations.

Experience in a large enterprise or Microsoft environment.

Experience with Spark, Delta Lake, and modern cloud-native data platforms. We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.