ETL Testing – Automation

Advertisements

Previous Page
Next Page  

ETL testing is mostly done using SQL scripts and gathering the data in spreadsheets. This approach to perform ETL testing is very slow and time-consuming, error-prone, and is performed on sample data.

Technical Challenge in Manual ETL Testing

Your ETL test team writes SQL queries to test data in a warehouse system and they need to execute them manually using a SQL editor and then put the data into an Excel spreadsheet and compare them manually. This process is time-consuming, resourceintensive, and inefficient.

There are various tools available in the market to automate this process. The most common ETL Testing tools are QuerySurge and Informatica Data Validation.

QuerySurge

QuerySurge is a data testing solution designed for testing Big Data, Data Warehouses, and the ETL process. It can automate the entire process for you and fit nicely into your DevOps strategy.

The key features of QuerySurge are as follows −

  • It has Query Wizards to create test QueryPairs fast and easily without the user having to write any SQL.

  • It has a Design Library with reusable Query Snippets. You can create custom QueryPairs as well.

  • It can compare data from source files and data stores to the target Data Warehouse or Big Data store.

  • It can compare millions of rows and columns of data in minutes.

  • It allows the user to schedule tests to run (1) immediately, (2) any date/time, or (3) automatically after an event ends.

  • It can produce informative reports, view updates, and auto-email results to your team.

To automate the entire process, your ETL tool should start QuerySurge through command line API after the ETL software completes its load process.

QuerySurge will run automatically and unattended, executing all tests and then emailing everyone on the team with results.

Just like QuerySurge, Informatica Data Validation provides an ETL testing tool that helps you to accelerate and automate the ETL testing process in the development and production environment. It allows you to deliver complete, repeatable, and auditable test coverage in less time. It requires no programming skills!

Advertisements