2015 Testing Year in Review: 12 months of macro- and micro-yes experiments

Tweet Earlier this month, I brought you the first in a series of 2015 Testing Year in Review blog posts. For this next post, we’ll be looking at tests using both KPI macro-yes(s) (conversions) and micro-yes(s) (clickthrough). Here are results of the 2015 tests we ran based on the primary test metric and level of confidence achieved. Anything under 95% level of confidence (LOC) is considered inconclusive, meaning the test did not have statistically significant findings. We want all the tests to be above 95% LOC because then we can say (with 95% confidence) that the changes we made are having the impact we want. The size of the boxes represents the range of LOCs, and the black lines are the midpoint, where half the data is above, and half below the line. The box represents the middle 50% of the data.   The big blue box shows the wide range of LOC’s for macro-yes tests. The base of the box is at 50%, and the black line (the median) is at 88%, indicating that less than half of these tests reach 95% level of confidence. The small green box on the right is micro-yes tests. The range is high and tight, and the median is 97%! This is what we want. Half of these test are above 95% LOC, and the other half stay above 88%. Micro-yes tests are producing statistically significant, actionable insights about our audience. Of course it’s much easier to increase clickthrough on a page than…
Source: Marketing Experiments
Read the Original Article: 2015 Testing Year in Review: 12 months of macro- and micro-yes experiments