Big data Pyspark Developer

Job

Quantum World Technologies Inc.

Grand Prairie, TX (In Person)

Full-Time

Posted 2 days ago (Updated 6 hours ago) • Actively hiring

Expires 6/26/2026

Apply for this opportunity

This job application is on an outside website. Be sure to review the job posting there to verify it's the same.

See Job Scorecard

Review key factors to help you decide if the role fits your goals.

How is this calculated?

Pay Growth

out of 5

Not enough data

Not enough info to score pay or growth

Job Security

out of 5

Not enough data

Calculating job security score...

Total Score

out of 100

Average of individual scores

Were these scores useful?

Skill Insights

Compare your current skills to what this opportunity needs—we'll show you what you already have and what could strengthen your application.

Job Description

Big data Pyspark Developer at Quantum World Technologies Inc. Big data Pyspark Developer at Quantum World Technologies Inc. in Grand Prairie, Texas Posted in about 21 hours ago.

Type:

full-time

Job Description:

Job Title:

Big data

Pyspark Developer Work Location :

Irving, Texas/ Tampa, FL Job Summary Exp 6 Years Must have good technical experience and should be able to provide technical solutions for multiple modules in parallel on need basis and bring the task to closure on time Unix SQL and Shell Scripting experience is a must have Expertise in Designing and developing scalable Apache spark ETL based Data processing pipelines Strong commandline knowledge in UnixLinux with Shell scripting using Bash Kornshell or Perl and File processing using awk scripts Expertise in SQL querying and complex joins Implementing comprehensive Spark based Data validation frameworks transforming large volumes of Financial data within the Project lifecycle Expertise with complex Data workflows with Apache AirFlow managing task dependencies SLAs etc to ensure timely data delivery and corresponding automated validation controls Strong Analytical skills and expertise on SparkSQL for Data analysis and validation ensuring the delivery of clean queryready datasets for business consumption Expertise in Data quality checks and monitoring Quality Engineering team where 70percent of effort will be for developing automation frameworks for testing Remaining 30percent effort will be on manual testing until its fully automated Handson with Automation Framework Design for ETL and API SME in Data Analysis Database testing Messaging queues Experience with coding standards code reviews source management build processes CICD pipeline

Skills Mandatory Skills :

Apache Spark, Big Data Hadoop Ecosystem, Python, Python for DATA, SparkSQL