A/B Testing: Measuring the Cost of Words

go back
Smartphone displaying two adverts adjacent to each other


The value of a word: how A/B testing empowers marketing decisions

When it comes to marketing, every action must be measurable – we need to understand how our customers are reacting to our campaigns, and the return on investment on each decision made. One of the best ways to measure impact is to conduct A/B testing.

How A/B testing works

A/B testing involves the marketing of two or more versions of content simultaneously to an identified audience, or marketing the same message to two or more defined target audiences. These variants could be design-orientated, for example, the position of a call to action (CTA) button on a page, or text-based, for example, the language used in a CTA.
Of course, A/B testing isn’t used exclusively for CTAs – for example, manufacturers selling to wholesalers may trial versions of their content with and without pricing.

However, CTAs are a great way of measuring your marketing efforts.

Measuring the “cost of words”

In a recent A/B test for one of our social media campaigns, Hydra Creative ran an A/B test on the wording in a CTA – testing the verb “see” against the word “choose”.



We discovered that “choose” was 3x more expensive, a factor which would have been affected by consumer search and click-through rates.

However, the success of A/B testing does not all come down to the cost of the advert. The most important factor is the value of engagement that each advert is receiving.

For example, the selection of 'see', received significantly higher levels of engagement than its rival: 5x the number of reach and 13x more impressions.

This powered the marketing decision to focus on the word “see” to achieve the best ROI.

Why calls to action are so important

There have been countless studies into the impact of CTAs – not only through their language but the psychology involved in their positioning, colour, and effects on different audiences.

For example, a study by Hubspot revealed that creating an illusion of scarcity, e.g. “buy today’s specials” encouraged click-throughs.

Organic versus paid search

CTAs may also impact SEO efforts – for example, search engines may index sitelinks that invite users to engage. Rather than simply a brand name appearing in position 1, a sitelink with phrasing such as “Download our product brochure” immediately outlines the benefits of that click.

CTAs should follow the famous Steve Krug UX maxim: “don’t make me think” – they need to illustrate what users will get out of their interaction.

Marketeers can implement this manually with paid search, too – using features such as sitelinks or rich snippets in ads, subtle changes to language can have a huge impact on click-through rates. We can measure these simply by trialling different ads in the same ad group: one with a particular verb phrase, for example, and one with another.

The psychology of marketing

As search engine algorithms continually update to suit user preferences, the importance of having an awareness of the psychology behind a user action increases daily for marketeers.

For example, a study into user downloads showed a 90 per cent increase in conversions when changing “your” to “my”.

There is even evidence to suggest that colour can have an effect on user engagement, provoking different emotional reactions.

While every target audience is different, we are all affected by tiny changes, from wording to UX design. A/B testing helps us to measure the impact of these changes, giving us a better understanding of our audiences and empowering marketing decisions for the future.


Find out more about Hydra Creative’s SEO services.


post by Alice
Client Portal.