GWO provides for an A/B style of testing “out of the box”. However, sometimes you may find it does not quite suit your needs, or, you may need more control over the URL to which visitors get redirected. This article describes how to perform an A/B test where you have much more control over how the redirection to the alternative pages takes place.

The following is a technique for performing an A/B test such that you have an opportunity to dynamically participate in the construction of the alternative URL’s.

First, instead of creating an A/B experiment, create a Multi-Variate experiment. Place the control script at the top of your test (A) page. Place the test page tracking script at the bottom of your test page and alternate pages (B, C, etc). Place the goal tracking script at the bottom of your goal page.

Now, instead of introducing the multi-variate sections scripts, place the following somewhere in your test page (I recommend it go near the control script):

This HTML comment is very much like a section script in that it declares a section named “page-url”, but does not modify the page or the user’s experience at all.

Now, you can validate the test and goal pages and move on to specifying the alternate URLs which you want to test. Enter these into the GWO UI as the content of variations for the page-url section. Here I’ve specified a simple relative URL, but you can specify a complete URL if you want:

Before launching or previewing your experiment, place the following script immediately after the control script on your test (A) page:

<script>
function filter(v) {
var b = utmx('variation_content', 'page-url');
var u = v[0].contents;
if (b && u.substr(0,7) == 'https://' && b.substr(0, 7) != 'https://') {
u = u.substr(7);
}
return u;
}
utmx('url', 'page-url', 0, filter);
</script>

At this point, you can preview and launch the experiment. It will behave just like any other A/B experiment.

You can see an example of this in action here as follows.

Test (A) Page (inactive)

Note that that URL will take you to the A page without enrolling you in the experiment. In order to experience a possible redirect, remove the content after the # in the link’s URL:

Test (A) Page (active)

The Filter Function

This script above will perform the redirection to alternative pages, should GWO decide that a given visitor is not to see the A page. Let’s look at a bit more closely.

<script>
function filter(v) {
...
}
utmx('url', 'page-url', 0, filter);
</script>

The utmx function is defined by the control script. It is the main entry point for a variety of GWO functionality available in test pages. In this case, the first argument, ‘url’ tells the function that it should treat this experiment as an A/B experiment and perform a redirect if necessary. The second argument, ‘page-url’, is the name of the section which defines the alternative URL’s. The third argument is a positional indicator and should be set to zero in this case. Otherwise, I will not describe it here.

The fourth argument is a filter function which you define and is called just before redirection takes place. It takes, as an argument, an object containing the redirection URL computed by the utmx function and returns the actual URL to which the user will be redirected. It is your opportunity to get involved in the form of the URL the visitor is redirected.

Before calling the filter function, the utmx function does a number of things to the target URL. First, it merges all query parameters of the current URL (document.location) with the query parameters of the target URL. This allows an alternate page to have the same information the A page has. For example, you might encode product ID’s as a query param:

https://www.mystore.com?product=slinky

You might enter:

https://www.mystore.com/b-page.htm

As the alternative B-page URL for your experiment. Because you are testing all your product pages, you can only specify the B-page URL, sans the product. GWO will redirect to:

https://www.mystore.com/b-page.htm?product=slinky

Which allows your B-page to know which product is being queried and present that product in the context of the B-page.

The utmx(‘url’, …) function also looks at the URL and adds https:// to it if it does not already have it. Many times, this is fine, if you don’t specify the protocol. For example:

www.mystore.com/b-page.htm

But it can sometimes get in the way. For example, you might want to specify (as I do in my example above), a simple, relative URL for the alternate pages:

b-page.htm

The the code in the custom script above will strip this away as needed:

var b = utmx('variation_content', 'page-url');
var u = v[0].contents;
if (b && u.substr(0,7) == 'https://' && b.substr(0, 7) != 'https://') {
u = u.substr(7);
}
return u;

If the “raw” version of the alternative in the variable ‘b’ does not begin with https:// but utmx’s version does, then it will be stripped away. Finally, the URL in the variable u is returned where GWO will perform a redirection to it.

URL Customization

The filter function allows you to inspect and modify the redirection URL at will. To demonstrate this further, consider my example above where the product ID is a query parameter:

https://www.mystore.com?product=slinky

But, what if my site encodes the product in the path of the URL? Like so:

https://www.mystore.com/slinky/a-page.html

And, I want to test an alternative page, like so?

https://www.mystore.com/slinky/b-page.html

You’d might enter the alternative URL in GWO as:

https://www.mystore.com/slinky/b-page.html

But, because your test page will be called for more than one product, like:

https://www.mystore.com/tofu/a-page.html

You can’t enter that URL, otherwise all users will see only the tofu product, regardless of which product they may have clicked on. Or, you might enter:

https://www.mystore.com/b-page.html

But, no product is specified here, and your web server might produce an error page.

What you need to do in cases like this is write some custom JavaScript which builds the correct URL. So, building off the last example, consider the following example. Let’s say the the following URLS are two of among many products:

https://www.mystore.com/slinky/a-page.htm
https://www.mystore.com/tofu/a-page.htm

And you specify an alternative URL like so:

https://www.mystore.com/PRODUCT/b-page.htm

The idea is while computing the URL to which a redirection will take place, inspect the current URL (document.location.href) for the name of the product, and replace the word PRODUCT with the name of the current product in the a-page. Like so:

<script>
var b = utmx('variation_content', 'page-url');
function filter(v) {
  var u = v[0].contents;
  if (b && u.substr(0,7) == 'https://' && b.substr(0, 7) != 'https://') {
    u = u.substr(7);
  }

  var l = document.location.href;
  var prefix = 'mystore.com/';
  var i = l.indexOf(prefix);
  var j = l.indexOf('/', i + prefix.length);
  u = u.replace('PRODUCT', l.substring(i + prefix.length, j));
  
  return u;
}
utmx('url', 'page-url', 0, filter);
</script>

This is very much like the first example above, but instead of simply returning the URL, we get the current product name and use it to replace the place-holder token, “PRODUCT” which is present in all alternative URL’s. This allows us to redirect to the proper alternative URL, while preserving the current product the visitor is interested in.

You can see this in action here:

https://www.gwotricks.com/abadvanced/slinky/b-page

Happy redirecting!

Comments:

This is awesome stuff Eric thanks for this, it will help alot with some of those advanced problems.

# posted by Gavin Doolan : March 20, 2009 3:03 AM

This is great thanks Eric. One question, do you have to alter the conversion script in any way? Our test is tracking visitors no problem, however conversions seem to be another issue.

# posted by Began : April 2, 2009 11:02 AM

Great stuff Eric, thanks. One question, do you have to alter the conversion script as well? We are tracking visitors with no problems, yet conversions is another story.

# posted by Began : April 2, 2009 11:03 AM

Hi Began,

No, the conversion scripts need not be chnaged using this technique. All this does is change the way that the redirection takes place.

– Eric

# posted by Eric Vasilik : April 20, 2009 9:48 PM

Hi Eric,

Great article! I had a question regarding the way GWO handles the URL defined in an AB test.

I have a test running with 1 variation. The scripts are all in place and I’m seeing conversions but the visitor count is not counting on the variation.

I have the URL for the variation defined correctly in the tool but once the user hits that page, it passes them to different pages depending on their login status, and then to a third page (I have the tracking script defined on that page).

It is important that the user hits the main page that handles the redirect. But it doesn’t seem to register as visits.

Is that because the tool has to match the URL to the page triggering the script. And if so why am I seeing conversions? Is it registering the trackPageview but not counting as a visit.
Is there any way around this?

Any help would be appreciated

Thanks,
Hans
[email protected]

# posted by Hans : August 12, 2009 2:08 PM

Really wonderful stuff here Eric. In one experiment I’m now testing thousands of templated pages versus a new version of the template. It’s working perfectly

# posted by Anonymous : September 16, 2009 10:04 AM

Thanks for your great trick, Eric. It’s working flawlessly in FF and Chrome. Though I seem to have poblems with IE – it simply won’t redirect to alternative urls. Have cleaned my cookies and Ctrl+F5-ed for about 30 times – still nothing. This seems to be happening on all machines using using IE8 | Vista. I’ve tried it with your examples, as well (https://www.gwotricks.com/abadvanced/slinky/a-page and https://www.gwotricks.com/abadvanced/a-page) and they don’t redirect either.
The redirects work fine for my colleagues who use IE8 on XP and IE7 on Vista.
Any ideas? Could it potentially mess up the GWO results?
Thanks in advance.

# posted by Cristina : September 21, 2009 8:03 AM

Hi Cristina,

I have the same experience you do with IE8 under Vista. However, if you clear cookies *and* close IE, and then go back to IE and visit your experiment, I find that it can redirect to an alternate page. IE seems to remember stuff in a browser session.

– Eric

# posted by Eric Vasilik : September 21, 2009 10:11 AM

Eric, thanx a lot! I want to test product pages so I guess this will help me a lot!

I have one question. What to do if even a Goal page has URL like www.example.com/buy-it/PRODUCT?

Thank you very much in advance!
Best Regards,
Michal

# posted by Michal : November 5, 2009 9:54 AM

Michal,

Just put the goal tracking script on all your goal pages. Only those who entered the experiment and get to a goal will trigger a conversion.

– Eric

# posted by Eric Vasilik : December 27, 2009 8:05 PM

I frequently see test pages in which the control script is not placed in a good location. In this article, I want to talk about the things to consider when placing the control script into your test pages.

Latency

The presence of the control script in your page will introduce latency into the total load time of the page. When the control script executes, it generates a request for a Google resource called siteopt.js. The latency is attributed to the time it takes for siteopt.js to load. To demonstrate this, with Firefox, you can load siteopt.js in the presence of the Firebug add-on that can measure the amount of time that it takes the page to fetch various resources. For me, inside the Google corporate network, it takes on average about 20 milliseconds to load siteopt.js:

Redirection

If you are running an A/B experiment, the control script may cause a redirection to happen if Google decides that this particular visitor should see a page other than the A page. This means that all the processing that the browser is doing when the redirect takes place will be aborted when the new page is loaded.

Other Resources

Given these aspects of the control script, it is very important that the control script appear before any references to external resources. These include CSS, script, image, objects, and the like. The reason for this is that if the control script decides to perform a redirect, all the time and work involved in loading these resources will be wasted and, most likely, performed again in the target of the redirect. This leads to increasing the total latency that the visitor experiences.

Displayable Content

Because the control script loads alternative content used in the display of the page, it needs to appear before the points in the page that potentially use this alternative content. Additionally, it is very important that the control script appear before any content in the page that is displayed to the user.

The reason for this is, again, latency. If the control script were to appear after, say, the first paragraph of the page, the user would see that paragraph, experience a very brief latency, and then the rest of the page would display. However, if the control script were to appear before this paragraph, then the window remains blank during the small latency, and then the page would render as a whole. This is a better experience for the user.

Also, a browser may spend less time laying out the page because there is no interruption of the display of the page.

Document Type Declaration

Many pages have a document type declaration. It may look something like this:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "https://www.w3.org/TR/html4/strict.dtd">

Browsers will change the way they parse an HTML file based on this declaration. In order to determine the type of a page, browsers will “sniff” for this declaration at the very beginning of the page. If they find a well formed declaration, then the parser for that document type will be instantiated.

It is very important that the control script appear after any document type declaration. The reason for this is that browsers will only look so far into an HTML document when sniffing for these declarations. The presence of the control script before the declaration may cause the browser to not find the declaration and to choose the wrong parser. This can have devastating effects on a page, potentially rendering it unusable.

Conclusion

So, in a well formed HTML document, the control script should be:

• After any document type declaration
• Before any other resources (CSS, scripts, etc)
• Before any displayable content (text, tables, etc)

In a well formed document, these restrictions are usually accommodated by placing the control script as the very first element of the head element, just after the beginning <HEAD> tag.

Comments:

Nice, totally agree Eric, I always recommend placing the tag just inside the head tag. However as you mention IE can trigger quirks mode if anything appears above the doctype so you need to place it after doctype and ideally before anything in the head section.

# posted by Gavin Doolan : June 16, 2009 5:46 AM

Thanks for explaining why the control script should as close to the top of the page as possible. Putting the control script in the head of our HTML would be somewhat difficult for me, because the web framework I’m using uses “layouts”, which make it difficult to customize the head of a page.

The other thing is that we try to push the loading of external scripts to the bottom of the page, which prevents blocking while external scripts are loaded. One way to achieve this might be to have your page sections consist of JavaScript at the bottom of your page that replaces the original content in the middle of the page with alternate content. This adds a bit of indirection to your page, but I think it would work.

# posted by allyourcode : February 4, 2010 3:04 AM

A while back, ROI Revolution (a Google Website Optimizer Authorized Consultant) came up with a technique for importing GWO information into GA (Google Analytics). This way, you can see how different variations of a web page/site will affect the various numbers that GA measures.

The original technique relied on munging through the __utmx cookie that the control script sets. I improved upon this by describing a technique which uses some new functionality I incorporated into siteopt.js. Their most recent description of this technique is here.

I’ve been giving some thought to this technique and wanted to 1) Improve the technique a bit, and 2) Suggest an alternative which has some significant advantages.

First, an look at my script for the ROI technique. The script I suggested is (note, all these scripts assume that ga.js and the control script have already been included on the page):

<script type="text/javascript">
if (utmx('combination') != undefined) {
var l = document.location, s = l.search;
s = s + (s.length ? '&' : '?') + 'combination=' + utmx('combination');
var pageTracker = _gat._getTracker("UA-XXXXXX-Y");
pageTracker._trackPageview(l.pathname + s);
}
</script>

This suffers from two things. First, global variables are introduced into the namespace of the page and, second, there is no specification of which experiment is annotating the hits. So, consider the following, modified, script:

<script type="text/javascript">
(function(){try {
if (utmx('combination') != undefined) {
var l = document.location, s = l.search;
s = s + (s.length ? '&' : '?') + 'combination=' + utmx('combination');
s += '&experiment=MyExperiment';
var pageTracker = _gat._getTracker("UA-XXXXXX-Y");
pageTracker._trackPageview(l.pathname + s);
}
}catch(err){}})();
</script>

Here I’ve enclosed the script in an anonymous function and a block to catch exceptions (should ga.js fail to load), and also added a query param describing the experiment being tested to disambiguate one experiment from another.

With this technique, you can go into your content report in GA and see how many visitors assigned to the various experiment variations actually end up visiting pages on your site.

This technique requires you to modify the GA tracking script on each page of your site which you want to track against an experiment. If you’ve already instrumented your whole site with GA tags, it might be cumbersome to edit them all, especially when you’ll need to modify them again when you end and experiment, or start a new one.

Experimenting on the entire Site

The answer I’ve come up with to deal with these issues is to integrate GWO experiment information a different way. Instead of modifying each page URL with the experiment information, you can set the GA user variable with experiment information (assuming you are not already using the variable for other purposes). Consider the following script:

<script type="text/javascript">
try {
var pageTracker = _gat._getTracker("UA-XXXXXX-Y");
pageTracker._setVar("MyExperiment-" + utmx('combination'));
} catch(err) {}
</script>

First note that this script does not call _trackPageView. Its job is not to track anything, but to set up the user variable to contain information about an experiment. This variable is stored in the __utmv cookie, and every subsequent call to _trackPageView in a tracking script will communicate the value of this cookie to Google servers and annotate the page view with the value in the cookie.

This script needs to placed in a particular place in your pages. First, it must only be located in GWO test pages which have already executed the control script. The script uses the function utmx, which is defined by siteopt.js, which is obtained through the control script. Next, this script needs to be executed before any calls to _trackPageView in any other GA tracking scripts. It also needs to be after the inclusion of ga.js.

I recommend that you place this script after the ga.js inclusion script and before the GA tracking script in the following manner:

<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "https://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script><script type=”text/javascript”>
try {
var pageTracker = _gat._getTracker(“UA-XXXXXX-Y”);
pageTracker._setVar(“MyExperiment-” + utmx(‘combination’));
} catch(err) {}</script>
<script type=”text/javascript”>
try {
var pageTracker = _gat._getTracker(“UA-XXXXXX-Y”);
pageTracker._trackPageview();
} catch(err) {}</script>

This way, the variable gets set before the tracking script which will use the variable annotate this page visit. Furthermore any visits that this visitor makes in the future (for up to an absence of two years) will also be annotated.

You can see this in action here:

Test Page

Special Consideration for A/B Experiments

The technique described above works for multi-variate experiments. That is, experiments which do not use redirection to present different pages to visitors. If you were to use this technique on GWO A/B experiment, the code which sets the user defined variable would sometimes not get executed because the original page would redirect to an alternate page, skip the setting on the original page for one which does not have the Control Script, and can’t use the “utmx” function.

Now, there are is a way to have the Control Script on A/B alternate pages. I’ll save that technique for another article. However, there is a way to get the above technique to work A/B (redirection) experiments. The idea is to set the user defined variable before the redirection on the original page takes place. To do this, consider the following A/B Control Script:

<script>
function utmx_section(){}function utmx(){}
(function(){var k='3923492669',d=document,l=d.location,c=d.cookie;function f(n){
if(c){var i=c.indexOf(n+'=');if(i>-1){var j=c.indexOf(';',i);return c.substring(i+n.
length+1,j<0?c.length:j)}}}var x=f('__utmx'),xx=f('__utmxx'),h=l.hash;
d.write('<sc'+'ript src="'+
'http'+(l.protocol=='https:'?'s://ssl':'://www')+'.google-analytics.com'
+'/siteopt.js?v=1&utmxkey='+k+'∓utmx='+(x?x:'')+'&utmxx='+(xx?xx:'')+'&utmxtime='
+new Date().valueOf()+(h?'&utmxhash='+escape(h.substr(1)):'')+
'" type="text/javascript" charset="utf-8"></sc'+'ript>')})();
</script>
<script>utmx("url",'A/B');</script>

Notice that this is really two scripts. The first script is that which, among other things, defines the “utmx” function. The second, using the “utmx” function, is the one responsible for performing the redirect to an alternative page, should it be required.

It is between these two scripts where you will need to include ga.js and set the user defined variable in order for this technique to work for A/B experiments. So in the A/B case, move that code from just before the tracking script to be between these scripts.

Experiment without GWO reports

When you normally set up a GWO experiment, you are asked to place test and goal page tracking scripts on your test page and goal page. This allows GWO to track when visitors hit your test and goal pages, and render a GWO report.

If you only plan to look at the GA reports for this experiment, you can forgo placing the GWO tracking scripts. Instead, you need only place the control script and user defined variable script in any pages which have experiment variations. In fact, you can place these scripts on pages which do not have any experiment variations. The effect of doing this is that visitors to these non-variation pages will have their experiment variations chosen for them before they encounter any test pages. This means that you will be able to track their activity before they encounter experiment variations.

This is useful in that, if you do this at every entry point to your site, all tracking events will be tagged with experiment information.

Report Segmentation

So, once you get all this set up, how do get experiment segmented reports? On each report, there is a Dimension dropdown:

This will show a report broken down by the values of the user defined variable. You can use this to get an idea of how individual variations of your site influence this GA stat.

Note that this is not like a GWO report. These are just raw numbers and are not analyzed for confidence with respect to which variation is statistically significant.

Comments:

Hi Erik,

Thank you for the excellent post. I’ve been using ROI Revolutions getCombo script, but your edits and using User Defined field would result in an overall cleaner output (not duplicating pageviews for example)

I’ve tried to implement your method using external files but with no luck as of yet (please forgive my ignorance).

I’ve 1) Activated the var _utcp 2) Run the MVT Head Script 3) Started and Ended the section swap 4) Run the standard GA 5) GWO Track Variation Pageview.

However, the custom variable does not seem to pass: https://www.geocities.com/nik7k7/set_var/test1.html

# posted by Nik K. : March 3, 2009 12:25 PM

Hey Eric,

What do you think about using Event Tracking once it is available for everyone?

Also nice updates to the ROIRevolution code, but I’m woundering if it should be altered by removing the query params which represent the combination and experiment (and be replaced with some made up virtual path name), because if the “Exclude URL Query Parameters:” GA feature is used there will be multiple pageviews recorded for a single pageview of the test I believe.

# posted by Erik Vold : March 26, 2009 12:05 AM

Very interesting post!

I’m curious, how would this code need to be modified to accomodate the Website Optimizer tracking script being executed via an onclick event, such as from clicking a link? In that case, the ga.js tracking would already have occurred, right?

Am I correct in presuming this method would not work with onclick?

If not, what method would allow you to track onclick to Website Optimizer and both onlick and page views to Analytics?

# posted by David : April 20, 2009 6:04 PM

This method is not intended to work with the GWO tracking scripts. It is meant to tag GA tracking events with the variation chosen for a visitor to an experiment. If you want to track links in GA, you can do that with event tracking or call trackPageView with a synthetic URI. The GWO visitor information will travel along with that GA event and you will able to track the performance of those events per variation in GA.

# posted by Eric Vasilik : April 20, 2009 9:42 PM

Using this experiment with the old Urchin GWO allows you to track both the GWO Conversions and Google Analytics User Generated Fields.

Also, I found that it was possible to track conversions for GWO and GA by using GA onclick for example, and placing the GWO conversion code in a div wrapper around the GA Code.

# posted by Vengeful : June 8, 2009 11:23 AM

As I understand this trick will not track multiple experiments that might be running in parallel, since only one combination id is recorded.

I’ve been thinking of using the __utmv cookie value instead, so that’s set with setVar. This value records all experiment ids and combinations. A small catch though is that is doesn’t include the combination id, but the permutation string, e.g. 1-3-2 for a test with 3 sections and where each number represent the chosen section variant.

What do you think about this suggestion?

# posted by devteam : June 10, 2009 6:42 AM

This is a fine idea. You can get the combination ID by calling utmx(“combination”).

# posted by Eric Vasilik : June 12, 2009 1:14 PM

Hello Eric,

have you ever tried to get equivalent reports from IndexTools (Yahoo Web Analytics) instead of GA?

kind regards,

Toepper

# posted by Anonymous : July 10, 2009 5:36 AM

I have not tried other analytics products, but as long as you can inject segmentation information, you should be able to do something similar.

# posted by Eric Vasilik : July 10, 2009 3:14 PM

Hi Eric,

I would suggest moving the pageTracker function outside of the utmx function check. That way if they have GA on the page but GWO failed, they still get a GA tracking hit.

Here is the code:

(function(){try {
var l = document.location, s = l.search;
if (utmx(‘combination’) != undefined) {
s = s + (s.length ? ‘&’ : ‘?’) + ‘combo=’ + utmx(‘combination_string’);
s += ‘&test=TestName’;
}
var pageTracker = _gat._getTracker(“UA-XXXXXX-Y”);
pageTracker._trackPageview(l.pathname + s);
}catch(err){}})();

Thoughts?

# posted by Ophir Prusak : September 23, 2009 3:11 PM

Nice input Erik

I have a couple of comments:

using _setVar
As you know this is a persistent cookie, so when the visitor is labelled on their first visit as e.g. “MyExpertiment-0-1-3”, on their next visit they will already have this label – even if they receive a new combination. So returning visits must be discarded with this method.

Experiment without GWO reports
“This means that you will be able to track their activity before they encounter experiment variations.”

With this method, have you given any thought to catering for visits who do NOT go on to see a test page?

# posted by Brian Clifton : October 4, 2009 7:50 AM

As a response to those above who are talking about the duplicate pageviews that result from adding the combination as a query parameter (via the original ROI Revolution code that I wrote):

First, our code did not inflate pageviews (the very first version two years ago may have), as it simply adds a query parameter to the end of the GA code that’s already on the page. Second, we have always recommended excluding this information from your main profiles and only looking at the data in a separate profile used only for GWO – preferably using the Site Search report.

We’ve certainly used the User Defined variable as well, although we tend to use that variable for other things.

Thanks for the code updates Eric! It’s nice to see this go from a hack to a cleaner and more official implentation.

# posted by Shawn P. : October 13, 2009 3:48 PM

In response to Brian’s post on Oct. 4: GWO does not choose a different variation for returning visitors. So setting the var again does not hurt. And, if a visitor does not go on to a test page after executing the control script, the visitor does not see a variation, so I see no need to track them.

# posted by Eric Vasilik : October 16, 2009 1:28 PM

In response to Ophir on Sept. 23:

That seems like a fine idea!

# posted by Eric Vasilik : October 16, 2009 1:56 PM

Is this happening only to me? The last few days integration does not work as it used to work.

# posted by netstudio.gr : November 1, 2009 1:25 PM