Summary of Duties and Responsibilities
We are seeking a self-motivated Data QA Engineer to lead the testing framework design, development, automation for our Big data infrastructure leveraging the latest technologies from Microsoft Azure, both on-premise and in the cloud. This position requires API testing and automation of data pipelines, data services, cloud data warehouses, business intelligence, and machine learning platforms, especially around unified transactional data. The ideal candidate is a passionate and highly skilled individual who can utilize programming languages and analytics tools/technologies to validate products, machine learning models, data pipelines, and data deliverables.
- Build and maintain automated testing frameworks for big data environments that are highly secure, scalable, flexible, predictive analytics, and performant using C#, Python, and SQL technologies.
- Develop test strategies, plans, test cases, and engineering best practices related to software test engineering, manual and automated testing, with a specific focus on data quality.
- Lead data governance and data profiling efforts to ensure data quality and proper meta data documentation for data lineage.
- Creating quality metrics to evaluate data pipelines, products, and customer deliverables.
- Participate in and contribute to design and code reviews.
- Knowledge of statistical methods, models, and processes and develop automated testing solutions to validate the predicted outcome.
- Ability to work within an iterative software development lifecycle, under Agile development processes
- Work with other developers to design and implement data science features in support of established security and acceptance criteria in collaboration with product owners.
- Proficiency with common software engineering best practices, such as pairing, test-driven development (TDD), writing unit and integration tests, and participating in code reviews.
- Work with engineers, designers, and analysts to deliver innovative AI/ML product feature enhancements.
- Assist the QA team with feature and regression testing for feature acceptance.
- Provide excellent written and verbal communication.
- 7+ years of experience using any of the following IDE or BBD tools (SpecFlow, Cucumber, Visual Studio, Visual Studio Code, etc.)
- 7+ years of hands-on working experience of testing REST, Web APIs or SOAP APIs (ASP.Net, MSTest, NUnit, Postman, and SoapUI)
- 5+ years of experience in data integration (ETL/ELT) / data engineering in a near real-time data warehouse environment
- 5+ years of experience in data modeling and distributed data architecture
- Experience with visualization tools such as Power BI, Tableau, SpotFire, TIBCO, QlikView, MicroStrategy, Information Builders, and other reporting and analytical tools
- Familiarity with data sets in myriad of formats and storage technologies/databases such as HDFS, Avro, CSV, JSON
- Hands-on testing experience in working Microsoft Big data architecture and technologies
- Awareness of tools used in the manipulation of large, complex datasets for use in analytic use cases.
- Bachelor’s or Master’s degree in Computer Science or related field such as Mathematics and Statistics, preferably with a focus on data analytics.