data_validation_scripts.md

Multiple Platform Data Validation Scripts

What is Data Validation Testing?

"Data Validation Testing" ensures that the data you work with is accurate and complete, that any necesary transformations occur without loss, that your processes can handle incorrect data, and the final output is correct.

How Can You Use Data Validation Testing?

• Test/Stage Regression: You could setup a more extensive set of data validation tests and schedule them to run daily. If it take 45 minutes to run 1,200 tests to thoroughly exercise the business logic of an entire schema, running it daily is a good trade-off. You'll be surprised at the number of times where out-of-the-blue you catch some odd application bug based on data rules.
• Manual QA: When it makes sense, you can build a data validation test script that you manually run during the QA process at the appropriate gates/milestones.
• Dev BVTs: You could setup a lean/fast validation script that runs as a Build Verification Test every time developers check-in code...a baseline set of rules that should never be violoated.
• Production Checkouts: You can use data validation testing for after prod deplosy using carefully crafted read-only SELECTs so as to not impact performance.
• Baked Into Applications: Where appropriate, you can build data validation tests directly into your ETL or data pipeline code.

Note

Because each database platform has its own unique flavor of SQL, this page is just a central landing page to redirect you to the appropriate database platform's specific pages.

	MS SQL Server Rule Set 01 - Row Counts Rule Set 02 - Keys Rule Set 03 - Heuristic Thresholds Rule Set 04 - Numeric Values Rule Set 05 - Date Values Rule Set 06 - Text Values Rule Set 07 - Reg Expr's Rule Set 08 - Diff Checks Rule Set 09 - Defect Regression Best Practices Basic Validation Script Advanced Validation Script
	MySQL Rule Set 01 - Row Counts Rule Set 02 - Keys Rule Set 03 - Heuristic Thresholds Rule Set 04 - Numeric Values Rule Set 05 - Date Values Rule Set 06 - Text Values Rule Set 07 - Reg Expr's Rule Set 08 - Diff Checks Rule Set 09 - Defect Regression Best Practices Basic Validation Script Advanced Validation Script
	Oracle Rule Set 01 - Row Counts Rule Set 02 - Keys Rule Set 03 - Heuristic Thresholds Rule Set 04 - Numeric Values Rule Set 05 - Date Values Rule Set 06 - Text Values Rule Set 07 - Regular Expressions Rule Set 08 - Diff Checks Rule Set 09 - Defect Regression Best Practices Basic Validation Script Advanced Validation Script
	PostgreSQL Rule Set 01 - Row Counts Rule Set 02 - Keys Rule Set 03 - Heuristic Thresholds Rule Set 04 - Numeric Values Rule Set 05 - Date Values Rule Set 06 - Text Values Rule Set 07 - Reg Expr's Rule Set 08 - Diff Checks Rule Set 09 - Defect Regression Best Practices Basic Validation Script Advanced Validation Script
	Greenplum Use the Postgresql scripts above

If you like these scripts, be sure to click the "Star" button above in GitHub.

Also, be sure to visit or subscribe to our YouTube channel www.DataResearchLabs.com!