Use Split Testing to Improve a WordPress Theme

Technology Blog

What’s the best colour for your headlines? If you make your contact form bigger will more people use it? Should a call to action be left- or right-aligned? If you are trying to make decisions like these about the design of your site, it can be hard to come to definite conclusions. Two designs may look equally attractive, but which will have more impact on visitors?

In this post I discuss a new plugin that will help you to carry out split testing on a self-hosted WordPress site, so you can arrive at such decisions using actual data rather than relying on guesswork.

What is Split Testing?

Split testing is a technique that’s been around for quite a while. It’s commonly used by online marketers to determine which version of a single page is most effective in persuading visitors to perform a desired action such as ordering a product. Split testing tools enable you to create different versions of pages and show them to visitors at random, then record the results.

Suppose you have a page with a headline in red (R) and you’d like to test it againtst another version of the same page with the headline in green (G). Using a Split Testing tool you can set up what is called an A/B Test of one against the other.

Once the test is running, when a visitor first comes to the page they will have a 50/50 chance of seeing either version. The Split Testing software makes a record of which version they have been shown and also tracks their activity. After a large enough number of visitors have come to the page you can use the result to compare  conversion rates between the versions. So, if 200 out of 5,000 visitors shown version R click on the buy button but only 100 of 5,000 do the same from version G, you can conclude that version R is roughly twice as effective as G.

WordPress Theme testing

For testing individual pages, the free tool Google Website Optimizer is probably the best for most purposes. But you may well want to test changes that affect many or all pages of your site, in which case a different approach  is required.

With that in mind, I wrote a plugin called “Little Split Test” which is intended to make it easier to test changes to WordPress themes. It integrates with Google Analytics so you can quickly determine which of your design variations works best for visitors.

Installation and Configuration

This is just a quick summary. Click on links in this list to skip to the detailed instructions which follow.

  1. Download the plugin and unzip it in your site plugins directory as normal.
  2. Activate the plugin
  3. Add custom variables to your Google Analytics code so your test data gets sent.
  4. Configure your first test through the plugin settings page.
  5. Edit theme code as required.
  6. Configure Google Analytics to measure the success of your tests .
  7. That’s it!

Sending your data to Google Analytics

To get Google Analytics (GA) to record your test data for you, you will need to add extra code to store values in what are called custom variables. There’s more than one way to do this depending on how the GA code for your site is generated, so I’ve provided instructions for some common cases.

Case 1: GA code pasted into your theme footer

If the GA code is hard-coded into your theme, you need to add the following lines above the one that says pageTracker._trackPageview(); in your template:

pageTracker._setCustomVar(1, 'experiment',
     <?php print get_option('LST_experiment') ?>, 3);
pageTracker._setCustomVar(2, 'test',
     <?php print get_option('LST_test_id') ?>, 3);

Case 2: Using the Google Analyticator plugin

The Little Split Test plugin has been designed to work painlessly with the Google Analyticator plugin. The custom variables needed will be inserted automatically, with no effort on your part.

Case 3: Using the Google Analytics for WordPress plugin by Joost de Valk

Currently this plugin is not compatible without a small change to its code. I have contacted the author with a view to getting this change adopted. In the meantime, here’s what you need to do. Locate the following code in the file googleanalytics.php:

if ( $options["extrase"] ) {
 /**
 * We need to load another script, so we need to close the try / catch, load the script, and open it again.
 */
 echo "\t} catch(err) {}\n";
 echo "</script>\n";
 echo '<script src="'.gapp_plugin_path().'custom_se.js" type="text/javascript"></script>'."\n";
 echo '<script type="text/javascript">'."\n";
 echo "\ttry {\n";
 }

After that piece of code, insert these new lines:

$custom_vars = apply_filters('ga_custom_vars', array());
 $index = 1;
 if(is_array($custom_vars)) foreach ($custom_vars as $name=>$value) {
 echo "\t\t".'pageTracker._setCustomVar('.$index.',"'.addslashes($name).'", "'.addslashes($value). '",3);'."\n";
 $index++;
 }

Case 4: Other plugins

Other plugins will need to be patched in a similar manner as the Google Analytics for WordPress plugin. Contact me for specific instructions (though I can’t promise to supply these for free!)

Configuring a test

The settings page for the plugin is very simple, and only contains two options:

  • The name for the experiment. I suggest you use something memorable, such as “logo_test_1”.
  • The number of different variants you are testing, which must be at least 2. These will be shown to visitors at random with an equal probability for each one. (If a visitor returns to the site they will be shown the same variant again each time, to ensure they are not confused.)

(Note that if need be you can tweak the probabilities for a variant by treating some variants as identical in your theme code. For example if you set up three variants A,B and C, of which B and C are generated exactly alike, this is equivalent to having two variants, one of which is shown to 1/3 of visitors and the other to the remaining 2/3.)

Editing your theme templates for testing

The exact edits you’ll need to make to your theme templates will depend on the nature of the design variants you want to test. Here though are some examples that you can adapt to your own requirements:

Example 1: Pure CSS

To be able to compare different CSS rules, first add a class to your page body element (usually found in a header.php template):

<body class="split-test-<?php print get_option('LST_test', 'A');?>">

This will set a class on the body of value “split-test-A”, “split-test-B”, etc. depending on which variant is chosen when the plugin is executed.

Now you can use this class in selectors in your stylesheet. Suppose you want to test changing the colour of your H1 elements from red to green. In the stylesheet you’ll already have code something like this:

h1 {color:red;}

It’s a good idea to stick to a convention in which the existing design is variant A, so all you need to do is to add overriding code for the other variants, such as this for B:

body.split-test-B h1 {color:green;}

Visitors who are served variant A will see a red header as before, while those who get variant B will see the green one.

Example 2: HTML

To test the effect of different HTML variants, use code like this in your template files:

<?php $test_id = get_option('LST_test', 'A');
 if('A' == $test_id): ?>
... existing HTML ....
<?php elseif('B' == $test_id) : ?>
... variant B HTML ....
<?php else : ?>
... variant C HTML ....
<?php endif ?>

Always code the get_option call exactly like this:

get_option('LST_test', 'A')

and you’ll ensure that even if the Little Split Test plugin is deactivated visitors will still see the A variant.

Example 3: Widget code

If you want to display different widget variants in your sidebar, install the Widget Logic plugin then apply conditions similar to the following to select which one(s) are shown:

get_option('LST_test','A') == 'B'

This is the condition for displaying a widget only for test variant B.

Configuring Google Analytics to measure your tests

  1. Login to Google Analytics and click on view report for the site in question.
  2. In the left hand column, click on the menu entry for Advanced segments.
  3. Click on Create new custom segment.
  4. Locate the Dimensions section in the left column, then under it click on Visitors.
  5. Scroll down to the entry for Custom Variable (Value 1) and drag it to the area marked dimension or metric.
  6. Choose condition “matches exactly” and enter the name of your experiment.
  7. Click on Add “and” statement
  8. Now drag Custom Variable (Value 2) from the left column to dimension or metric in the new statement area.
  9. Choose condition “matches exactly” and enter ‘A’.
  10. Go to the Name Segment field further down the page and enter a memorable name for this segment (eg. “My test 1, headline red”) and press the Save Segment button.
  11. Repeat for each variant of this experiment, using ‘B’, ‘C’ … etc in step 9.

Example of a segment configuration panel

Viewing results on Google Analytics

Once your tests have been running for a day or so, you’ll be able to use the segments you have configured to view the results from your site. At the top right of each report screen, you’ll see a control labelled Advanced segments. By default, this will be set to All Visits. Click on the control to show all the segments that are defined, and select your tests from the list.

You’ll now see that your GA reports show values for each of your test segments. To measure which is more successful you may want to define goals and then look at the respective conversion rates for each segment, but that process is outside the scope of this article: see the Google Analytics documentation for more details.

Summary

Split Testing can aid you in coming to informed conclusions about design choices for your site. The Little Split Testing plugin enables you to set up some kinds of test more easily though it is still at an early stage of development so we’d welcome your suggestions for additional features.

Figure W are experienced in all kinds of WordPress work. Call us today on 07783 386951 or use our enquiry form to get help with your WordPress site.

6 Responses to “Use Split Testing to Improve a WordPress Theme”

  1. Scott

    We just released a WordPress plugin that is a simpler alternative to Google Optimizer that allows you to run split tests inside of WordPress using shortcodes. Would love additional feedback on it and any suggestions on more advanced features.

    WP Cash+ Automatic Split Testing WordPress Plugin
    http://www.wpcashplus.com

    Thanks!
    Scott

Leave a Reply

  • (will not be published)

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>