ICTA Nomination: RSA’s Technology to Measure, Test and React

2015 ICTA Nomination.  

The Adobe Analytics tool was purchased and implemented by the RSA eBusiness web team to measure and drive the effectiveness of its 20+ websites. The tool facilitates A | B & Multivariate testing based on its own analytics and search keywords, in order to test different hypotheses in a live environment.

The technology is used to measure the effectiveness of different tactics on the attainment of the business objective on RSA’s various transactional websites. Tactics that are measured include everything from the structure, the design (creative) and the content. An example would be testing a different colour of the main call to action button, ie., “Get a quote” or testing two different images based on the end user’s search keywords, to provide a more personalized approach with the hope of driving higher conversion rates or engagements. This essentially allows us to test the effectiveness of our websites in real-time. We can observe and compare real/potential customer’s behavior and aim to make improvements based on this.

The tool can work on any website on RSA’s eBusiness web platform. Currently, the platform has about 30 websites, and continues to grow. The tool is generally used on transactional websites, as these are revenue generating, and the slightest change can have a monetary impact. For the purpose of this award, we have considered only the petsplusus.com and Johnson.ca websites as they had transactional websites active in all of 2014.

Specific Business Uses


  • Wanted to test that if capturing an email address before the auto quote is generated would have an impact on quote completion rates
    Traffic was split 50/50 to the quote flow with or without the email capture
  • Tested different types of imagery on Quote Start and Completion rates
    Traffic was split 3 ways, Control, QuoteA and QuoteB
  • Wanted to test if the quote start page discouraged users from starting the online quote
    Traffic was split 50/50 with the quote page vs. without


  • Wanted to test if a Static vs. Rotating carousel banner had an impact on click through rates
    Traffic was split 3 ways, Control, V1 and V2
  • Wanted to test the difference between 3 Landing Page layouts on quote conversions
  • Wanted to test the impact of Personalization on quote to purchase rates
    Traffic was split 3 ways, depending on if the customer searched for “dog insurance” or had previous searches suggesting they were a dog owner, the quote landing page image would serve up an image of a dog and the same would happen for suspected cat owners. These would be compared to the user going to directly to the quote and a control version.

Implementation Period

The tool was purchased, implemented and deployed in 2013 on the eBusiness platform, but not used to its full potential until late 2013.

Essential Features

  • Direct & real-time measurement – This is by far the best feature of the tool, it allows us to measure real customer behaviours when they are shopping online for insurance
  • Built-in analytics – Uses the website’s and browser data to create personalized experiences for the end user


  • Ability to compare the placement of on-page elements and components such as images, content, colours etc. and the impact they have on conversions/driving the business objective.
  • From an RSA eBusiness perspective, the insights from one experiment can benefit other areas of the business, as online behavior will not be dependent on the type of insurance.

In each of the examples above, there were measurable results which helped determine the action we then took. These tools and techniques help RSA get better results from the investments made in approaching the market.

About RSA Canada

RSA Canada is Canada’s third largest property & casualty insurance company, with the broadest national personal and commercial lines proposition. For more information, please visit www.rsagroup.ca.