Let's Organize Hexawise-Generated Test Sets into Categories

When you compare the total possible number of tests in a system to the number of Hexawise-generated tests, you will notice some very interesting results.

  1. Some plans will result in relatively unexciting differences between those two numbers.
  2. Many plans will have impressive differences between those two numbers.
  3. Some plans will have incredible differences between those two numbers.

In order to design tests with Hexawise thoughtfully,  test designers need to understand why such hugely different results occur. And be able to practically apply that important knowledge to modify sets of tests to best suit the needs of their testing projects.

This lesson is designed to help you understand why such different outcomes will result.

This is a "Normal-Sized" Hexawise set of test inputs:

The "Normal"-shaped Hexawise test inputs results in an impressive difference between the number of total possible tests and the number of 2-way tests

This is the first test from this "normal-shaped" plan.  How does Hexawise "look" at it?

Every Hexawise-generated test is carefully-structured to test "as much stuff as possible." What does that mean exactly? Well, one of the ways that Hexawise "sees" or "understands" what is being covered by a given test is to analyze the precise number of pairs of values that get tested (for the first time) by the test. More pairs get covered in a test than you might realize.

It is worth pointing out that the first test case in this "normal-shaped" example is testing for precisely 36 pairs of values.

I.  What happens when you create a set of 2-way tests in Hexawise with the following test inputs?

Instructions: generate a set of 2-way tests in Hexawise with the following inputs:

 

Size: Very Small, Small, Medium, Large, Very Large

Location: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

Customer Type: Individual, Family

 

 

A) How many total possible tests are there?  ____________________________.

B) How many 2-way tests are there?  ____________________________.

C) How is the number of total possible tests calculated in this example? What 3 numbers are multiplied together to result in that total? Total possible number of tests = _______ X _______ X _______.

D) As compared to our "normal-shaped" test inputs that we began with in the above example, why is there a much lower number of total possible permutations?  In other words, what did the "normal-shaped" set of test inputs have that this set of test inputs does not have?  ____________________________.

E)  As compared to the results from the "normal-shaped" test inputs plan, how would you describe the results achieved in this example? E.g., is the test designer who generated this test set getting relatively more value or relatively less value by using Hexawise?  ____________________________.  Why? ____________________________.

F)  Go to the Matrix Chart for this plan. Move the slider all the way back to test number 1. How many pairs of values are tested in test number 1?  ____________________________.  How many additional pairs of values are tested in test number 2 that were not already tested in test number 1?  ____________________________.  Now slide the Matrix Chart slider all the way to the final test. How many pairs of values get tested for the first time in the final test?  ____________________________.  

Open up the Sample Plan called "Google Maps - Create Map Directions" example.

Instructions: Go to Sample Plans --> Find the Google Maps - Create Map Directions plan --> Click on plan name -->  Click on the green "Create an editable version" button.

 

A) How many total possible tests are there in this example?  ____________________________.

B) How many 2-way tests are there?  ____________________________.

C) How would you describe the difference in the size of these two numbers? _______________________.

D) How many numbers need to be multiplied together in this case to calculate the total possible number of tests?  ____________________________.

E)  As compared to the results from the "normal-shaped" test inputs plan, how would you describe the results achieved in this example? E.g., is the test designer who generated this  set of 2-way tests getting relatively more value or relatively less value by using Hexawise?  ____________________________.  Why? ____________________________.

When you have documented answers to the above questions, please email your answers to instructor@hexawise.com.

Did this answer your question?