Senior Data Engineer (Data Quality)

Overview

Responsible to evolve Data Quality capabilities and improve quality of NFCU data assets in Azure Cloud, this includes Enterprise Data Lake, Data Warehouse, and Data Hubs. This role will partner with Enterprise Data Governance stakeholders to generate product roadmap that aligns to strategic goals, assess current state capabilities to understand gaps, define architecture and design, and implement. The candidate will lead and hands-on coding to mature Databricks based bespoke framework that controls and monitors Data Quality. Additionally, the candidate will have experience in working with industry leading COTS solution on Data Quality to mature NFCU Data Quality coverage and effectiveness in Azure Cloud.

Responsibilities

• Establish and lead Data Quality Center of Excellence to govern and mature technology and process
• Collaborate with key stakeholders to generate Enterprise Data Quality product roadmap
• Assess and identify Data Quality gaps and challenges to improve in Azure Cloud, prioritize the efforts, and lead engineering team to implement
• Collaborate with stakeholders and engineers to assess COTS solution on data quality tools, evaluate options and define future state
• Define architecture and design that matures data quality solution for batch and data streaming services
• Evolve inline Data Quality framework for batch data processing in Azure cloud
• Enable Data Quality monitoring and alerts for Data Streaming services
• Apply Data Quality dimension principles to improve and measure quality of data
• Deliver key Data Quality metrics to measure the success of the efforts for business stakeholders
• Evolve and enforce data engineering standards for Data Quality
• Generate Monthly Status Report for Data Quality efforts and review with leadership
• Provide technical direction and leadership routinely to other engineers; serve as a coach and mentor
• Perform other duties as assigned

Qualifications

• Bachelor's degree in information systems, Computer Science, Engineering, or related field, or the equivalent combination of education, training, and experience
• Expertise in delivering and improving data quality in a Big Data environment in cloud for Enterprise Data Lake (Azure or similar), Data Warehouse (Azure Synapse or similar), and Data Hubs (Azure Cosmos DB or similar)
• Expertise in data processing (Databricks ETL, Azure Event Hub, Azure Functions, or similar), data quality solution (Acceldata, Ataccama, Anomalo, or similar), data governance solution (Unity Catalog, Alation, Azure Purview or similar)
• Expertise in enabling error handling, data validation, data reconciliation, data quality monitoring and alert
• Experience with processing various file types in Data Lake. (Delta, JSON, Parquet or similar)
• Strong understanding of data quality principles and best practices
• Proficiency in programing languages such as SQL, Python & Spark
• Experience with DevOps toolset (Azure DevOps or similar) to collaborate and deploy code
• Excellent verbal and written communication skills, with the ability to effectively communicate complex concepts to stakeholders at all levels

Hours: Monday - Friday, 8:00AM - 4:30PM

Location: 820 Follin Lane, Vienna, VA 22180

About Us