Pay Growth
?
out of 5
Not enough data
Not enough info to score pay or growth
Senior Data Engineer
Job
Pinnacle Government Solutions, LLC
Arlington, VA (In Person)
Full-Time
Review key factors to help you decide if the role fits your goals.
Job Security
?
out of 5
Not enough data
Calculating job security score...
Total Score
79
out of 100
Average of individual scores
Skill Insights
Compare your current skills to what this opportunity needs—we'll show you what you already have and what could strengthen your application.
Job Description
Senior Data Engineer Pinnacle Government Solutions, LLC Arlington, VA Job Details Full-time 2 hours ago Qualifications Software engineering Data Integration (Data management) Tooling Spark Public Cloud SQL ETL process automation Spark implementation Agile software development Python Database software proficiency Full Job Description Job Description Pinnacle Government Solutions is a minority-owned small business that provides Software, AI/ML, and Cloud Engineering Solutions for our Department of Defense, Intelligence Community, and Federal Civilian agencies. We bring SME-level expertise and over five decades of combined industry knowledge and Talent Acquisition experience. We believe in getting it right for our customers and our country. We are seeking a Data Engineer that will develop new tools, code, and services to execute data engineering activities involving data of varying types and in varying conditions.
Activities include the following tasks:
Movement of structure and unstructured data using approved methods. Execute data ingestion activities for storing data in a local or enterprise level location. Develop code to format data that supports exploration. Analyze source data formats and work with Data Scientists and partners to determine the formats and transforms that best meet mission objectives. Develop code and tools to provide one-time and on-going data extraction from various repositories, formatting and transformations into enterprise or standalone data models. Develop new ETL and perform O&M and enhancements on existing ETL code using best practices/standards. Develop and deliver documentation for each project including ETL mappings, code use guide, code location and access instructions.Responsibilities:
- Design and optimize Data Pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR cloud services, NiFi and Kubernetes containers
- Ensure the pedigree and provenance of the data is maintained such that the access to data is protected
- Clean and preprocess data to enable access for advanced analytics
- Collaborate with enterprise working groups to advance the state of data standards
- Collaborate with the engineering team, data stewards, and mission partners to aid in getting actionable value out of the data holdings
- Collaborate with software engineers to update, configure, and maintain data services based on the requirements
- Ensure data quality by working with the testing and data quality team to enhance standardization of data conditioning pipelines
- Experience adapting to various types and formats of data, and working with development teams to integrate new data processing platforms, Required Skills 10+ years' experience with:
- Data lifecycle engineering
- Development and maintenance of extract, transform and load (ETL) tools and services
- Cloud and on-prem data storage and processing solutions
- Python, SQL, Spark and other data engineering programming
- COTS and open-source data engineering tools such as ElasticSearch and NiFi
- Processing data within the Agile Lifecycle, Desired Skills Experience using AI/LLMs to accelerate the processing and transformation of data, About Pinnacle Government Solutions, LLC Pinnacle Government Solutions is a minority-owned small business that provides Software and Cloud Engineering Solutions for our Department of Defense and Federal Civilian agencies.