Compare your current skills to what this opportunity needs—we'll show you what you already have and what could strengthen your application.
Job Description
Azure Data Engineer with AI Fabric at Wall Street Consulting Services LLC Azure Data Engineer with AI Fabric at Wall Street Consulting Services LLC in Lyons, New Jersey Posted in 2 days ago.
Type:
full-time
Job Description:
Job Title:
Azure Data Engineer with
AI Fabric Location:
Warren, NJ (Onsite)
Domain:
Insurance Experience:
10 + Microsoft Fabric & lakehouse engineering Hands-on production experience with Microsoft Fabric One Lake, Lakehouse, Data Factory pipelines, Spark notebooks, and Direct Lake mode for Power BI. Must have built and operated something real on Fabric Fluent in
PySpark and Delta Lake:
MERGE, schema evolution, time travel, OPTIMIZE, partitioning. Has built incremental ingestion from ADLS Gen2 or S3 into curated Delta tables. Strong SQL (window functions, CTEs, query optimization) and a working knowledge of Parquet internals, partitioning, predicate pushdown, and compaction. Event-driven ingestion on Azure Has built production ingestion using Azure Event Hubs Python engineering & Azure platform fundamentals Modular Python code with type hints, unit tests, and packaging. Git-based workflow with pull requests and CI. Working knowledge of ADLS Gen2, Azure Active Directory, Key Vault, and Managed Identity permissions and the engineer need to handle them confidently from day one. Source-to-target data modeling with a canonical layer Has built or contributed to a canonical model EDP, data vault, or dimensional that decouples source systems from downstream consumers. Understands medallion architecture (Bronze / Silver / Gold) and can explain why each layer exists and what belongs in it. Production LLM & embedding experience using an LLM API (Azure OpenAI strongly preferred for our stack) with structured output via JSON mode or function calling, paired with an embedding model for semantic search or matching.