Accessing Radio Memories

A study into methods for measuring effectiveness of radio advertising campaigns

Click here to e-mail the RAB if you have a question about this publication

Radio recall levels are prone to being understated. There appears to be a category problem which stops consumers from recalling radio advertising as accurately as we might wish. The barrier is based on the fact that TV is seen as the first place to think of when searching for advertising memories. Radio is much lower on the list, and local radio – despite its extensive use by national brands – is unthinkingly assumed to be carrying local advertising.

Different prompting methods elicit different apparent levels of recall, and the differences are very significant – 6% versus 63%. The different prompting methods represent measures of rather different things.

Spontaneous radio-attributed recall averages only 6% and is not a useful figure, as the comparable figure amongst non-listeners was nearly as high at 4%.

Prompting listeners with the brand name raises awareness levels to an average 13%; this is as far as many advertising tracking studies will go.

Prompting which also specifically names the radio medium sees average awareness levels rise to only 19%. Again this is where many tracking survey methods will stop, and is clearly demonstrated here to be a huge understatement of the number of people who have actually heard the campaign.

Verbal descriptions of radio campaigns appear to be an unreliable method; scores on this method are a measure of describability more than anything else.

Commercial recognition seems the method most able to access radio memories. It offers the additional benefit that, in delivering a larger and more usable sub-sample, the group of listeners who have heard the campaign can also be analysed for their brand attitudes etc.

Recallability or recognition? Researchers need to be clear what they are trying to measure. The fact that we cannot recall being exposed to something may be irrelevant to whether we understood - and would recognise - the message.

The origins of this study lie in a challenge which was laid down to media planners and researchers by Douglas McArthur of the RAB, who had seen Jonathan Durden of New PHD present to the Media Research Conference about research milestones – without mentioning a radio research milestone. The challenge to media people was to suggest candidate ideas which might develop into milestone research.

From about 40 high quality entries, the proposal selected to win was from Charlie Makin, partner at Booth Lockett Makin. Charlie’s proposal concerned the notion of Implicit Memory, an effect which had been clinically demonstrated to exist with exposure to print messages.

In essence, Implicit Memory is demonstrated by the fact that. although respondents do not recall having been exposed to something before, they show a demonstrably more favourable attitude to it in subsequent exposures. Charlie Makin’s theory was that this effect was probably in play when it comes to radio, perhaps even more strongly because radio is consumed passively – and there was initial supporting evidence in the fact that measuring radio effectiveness is known to be tricky.

In fact, from an analysis of the literature conduted by psychologist Chris Wilson, it became clear that measuring the ffect of Implicit Memory was likely to be extremely difficult and to be at best marginal in its contribution to advertising effect.

But this consideration threw up a wider area of analysis. There seemed to be a need to look at why radio effectiveness was so hard to measure, and what could be done about this in terms of accountability for advertisers.

The second two questions are summarised in detail in the following sections, while the first two questions are written up in more detail elsewhere by Chris Wilson.

There are peculiarities in the nature of radio listening identified by Chris Wilsdon which are widely documented elsewhere(e.g. Wireless Wisdom 1994).

Radio listeners tend to be on their own (physically and/or mentally) and to be doing something else while listening – so most radio listening is essentially solitary and passive. In addition, radio messages are invisible and leave no tangible trace after they are heard (unless tape-recorded, which is highly unusual).

Also, radio is not perceived by consumers as the "home of advertising". Commercial Radio may carry advertising but, when consumers are asked about advertising, their first response typically is to think about television (historically the most conspicuous and recognised advertising medium, although become more passive and fragmented these days).

There is also a "category problem" in terms of recall. When asked who advertises on local radio, listeners instinctively tend to look for the answer amongst local businesses like retailers. This makes it difficult for them to recall which national brands have been on the radio – even though they may be demonstrably aware of those advertisers’ messages.

Measuring the role of radio in advertising effectiveness has historically been relatively difficult for national brands which use radio as maybe 5-15% of their advertising investment. Put simply, the noise from the other 80-95% of the advertising (TV, press, posters, wherever) tends to confuse the measurement. This difficulty is of course true of any medium playing this type of secondary role in the mix.

So a key challenge for the field-test was to suggest the optimum way to identify the effect created by radio as opposed to other, larger campaigns in other media.

Most advertising effectiveness tracking tends to use awareness as the currency. There are debates about the validity of this measure (as it leaves out issues such as emotional response or sales results), but as it is the most common "intermediate parameter" it is the one we have tested here.

Our intention was not to re-invent the way radio effectiveness is measured but to analyse current methodologies in order to recommend best practice.

Six brands with relatively high levels of radio spend were chosen for the test (identified from the station’s advance October bookings data). The brands were:

Using these higher-spending brands meant that the test brands would, if anything, be more easily recalled as having been advertising on radio (according to MMS data, over the whole previous year their advertising spend was a combined £13.7m, of which £3.7m was spent on radio). Test station for the exercise was 95.8 Capital FM in London.

Each of the six brands was researched separately (in order to replicate the way tracking studies normally look at brands within sectors), and in each case the sample was split into about 100 listeners and 35 non-listeners to the station. This gave a combined total of

The sample was biased in favour of females (59% female) because the sub-sample for Feminax was female-only.

This was an essential component in order to offset the effect of any misattribution in the minds of the listeners. In other words the difference in awareness scores between listeners and non-listeners is likely to have been generated by exposure to radio advertising. The non-listeners in this sample did not listen to ANY Commercial Radio station, in order to offset any possible advertising exposure from other stations.

NB In describing the stages below we have used the example in the graphs of the Prudential – i.e. sector prompt was pension companies – but it could equally well have been any of the other five advertised brands.

This stage of the questionnaire took respondents through three linked stages – spontaneous brand awareness, spontaneous advertising awareness and anyattribution of that advertising awareness to radio.

Chart One shows that overall spontaneous brand awareness for the test brands averaged at around 50% (in other words, half the sample mentioned the test brand when naming brands), and this did not vary significantly between listeners and non-listeners.

In Chart Two we see that spontaneous advertising awareness is significantly lower for both listeners and non-listeners, though listeners are somewhat more likely to mention advertising for the test brands.

This fits with existing advertising research, which shows that, in typical markets, remembering brands is significantly easier than remembering advertising.

Finally at this spontaneous stage, respondents were asked "You mentioned advertising for [name of company mentioned] Can you tell me where you have seen or heard that advertising?"

Clearly, awareness of the test brands advertising on radio is very low. This was fairly consistent across brands: while the average is 6%, individual brand scores ranged between 2% and 9%.

Advertising attribution to radio is similar across listeners and non-listeners. This seems to represent a level of "background noise" in terms of attribution of advertising, and therefore suggests that this method does not deliver an advertising of any significance.

This section saw the interviewers show respondents a list of half a dozen relevant brands in the sector, and at first simply ask "Which of these companies have you heard of?"

Average awareness for the test brands in the sample was extremely high – most are household names – and there was little difference between listeners and non-listeners.

When asked if they could recall any recent advertising for any of the named brands, again scores fell to below half the level for brand awareness.

As Chart Five shows, listeners were significantly more advertising aware than non-listeners at this stage, although of course it hard to tell exactly what advertising is being remembered here. Some brands in the sample, e.g. TV Licensing and the Prudential, have an extensive advertising heritage which consumers can sometimes carry with them for years.

The final stage of 2a was to prompt with the name of the specific test brand, asking "You mentioned advertising for [test brand]. Can you tell me where you have seen or heard that advertising?"

This was only asked of those who had mentioned the test brand in the previous question. Radio was not prompted at this point.

As Chart Six shows, there is again a significant difference in attribution levels between listeners and non-listeners, but still only 13% of listeners can recall that radio is a medium where the advertising was running.

This basic form of prompted awareness is a conventional measure in ad tracking. If one assumes that the 4% attribution to radio amongst non-listeners is statistical or survey "noise", then the true measure here is about 9%, which is a very small sub-sample to work with. For example, it would be tricky to look at brand attitudes amongst those who were "radio advertising aware" by this measure – a larger survey sample would be needed.

Given the "category problem" descibed at the start, it makes sense that, if radio is difficult for consumers to bring to mind when thinking of advertising, then the questionnaire can directly prompt them by naming the medium.

So at 2b, the interviewer used the brandname prompt card again, asking "Which, if any, have you seen advertising on posters recently?", "Which, if any, have you heard advertising on radio recently?" etc.

Other media were deliberately included (on a rotated order basis) at this stage to stop respondents from beginning to focus too much on radio.

As Chart Seven shows, this deliberate prompting by medium still only generates a 19% radio awareness amongst listeners.

Again, this is the level at which many advertising awareness studies leave the search for radio awareness. At 19% however, we are beginning to have a sample size which is both statistically robust and large enough to be divisible.

At this point the interviewers read descriptions of the commercial for the test brands (there was a description of each individual advertisement, and the interviewers used these on a rotated basis).

So for example, for the Prudential pensions campaign, the description of one of the executions was:

Respondents were then asked if they had heard any this, or similar, advertising on the radio recently.

Scores on this method varied very widely, and some of the test brands had a much higher score for "Might have heard it" than "Yes have heard".

In effect, some of the campaigns were much more describable than others. The results, varying widely across the test brands, are really a measure of this describability.

It makes sense that a very effective radio commercial could be difficult for a respondent to recognise from someone reading out a written description in this manner. It is also a fact with radio that what a listener experiences or "sees" when the advertising is broadcast may be entirely different from the way the ad is structured on paper.

At this stage respondents were given headphones and asked to listen to a selection of the radio commercials for the relevant brand. They were then asked if they had heard any of this advertising, or advertising like it, recently.

As can be seen in Chart Nine, something like two-thirds of the listeners were able to say they had heard the advertising in question. This rises even higher – to 73% - when one includes the people who answered "Maybe".

Again, about 5% of the non-listeners are saying Yes at this stage of the questionnaire, indicating that this level of "noise in the numbers" is pretty consistent all the way through, rather than a quirk of any part of the questionnaire.

In fact, the 63%of listeners saying Yes in this stage of the interview are doing something rather different from the other stages. They are being offered the most direct route to recognition, rather than being tested on whether the memory of the advertising is salient in their minds.

The four-stage test clearly shows that the different prompting methods elicit entirely different levels of apparent "radio advertising awareness", with a maximum difference of nearly 60% between Stage 1 and Stage 4.

Splitting the samples into listeners and non-listeners is a reasonable guide to isolating the effect of radio. The samples need to be balanced.

The contrast scores amongst non-listeners at each stage indicate a "statistical noise" level around 5%; true findings have to be significant against this background.

Listeners’ spontaneous ability to recall and identify radio campaigns seems to be very low level on average – lower than for advertising in general.

Prompting with brand name aids recall to some extent, but even when the medium is also prompted, average attribution to radio to radio rises to only 19%. Given that over 60% positively confirmed in the end that they had heard the test campaigns, this demonstrates that "awareness measures" generated in surveys may be limited by the prompting method.

Describing the commercial verbally to the respondent seems to be an unreliable method. Some commercials are more describable than others, and the figure generated by this method seems to be a measure of this describability.

Commercial recognition appears to be the most reliable form of identifying whether people have heard the campaign or not, and certainly accesses the memories of a far larger part of the sample.

The fact that this accesses memories in a larger number of respondents means that it can potentially deliver a much larger sub-sample to analyse subsequently in terms of their attitudes to the brand or the sector.

With thanks to: Clark Chapman Research; Capital Advertising; Chris Wilson; Charlie Makin. This study was first presented to the Media Research Group conference in November 1998, and published in Marketing Week in December 1998.

SAMPLE QUESTIONNAIRE

The following questionnaire was used for researching one of the test brands - Prudential (pensions). The same questionnaire format was used for all the other test brands in this research.
____________________________________________________________________

INTRODUCTION: Good morning/afternoon I work for Clark Chapman Research, an independent market research company. We are carrying out a survey in this area. Can I just check …....
Q1 Do you or any of your family, work in any of these occupations ?
Advertising, Marketing, Market Research, Journalism, Broadcasting - TV or Radio

IF ANY OF THESE, CLOSE
______________________________________________________________________________

STAGE 1

Q1 Which pension companies can you think of?

DO NOT PROMPT. PROBE Any others? MULTICODE POSSIBLE

	(18)
Allied Dunbar	1
Equitable Life	2
Prudential	3
Scottish Widows	4
Standard Life	5
Other (write in)	6

Q2 And which pension companies have you heard or seen any advertising for recently? DO NOT PROMPT. PROBE Any others? MULTICODE POSSIBLE

	(19)
Allied Dunbar	1
Equitable Life	2
Prudential	3
Scottish Widows	4
Standard Life	5
Other (write in)	6

ASK FOR ALL MENTIONED AT Q2
Q3 You mentioned advertising for -- pension company -- . Can you tell me where you have seen or heard that advertising? DO NOT PROMPT. PROBE Anywhere else? MULTICODE POSSIBLE

	Radio	Poster	N’paper/ Mags	TV	Other
	(20)	(21)	(22)	(23)	(24)
Allied Dunbar	1	1	1	1	1
Equitable Life	2	2	2	2	2
Prudential	3	3	3	3	3
Scottish Widows	4	4	4	4	4
Standard Life	5	5	5	5	5
Other (write in)	6	6	6	6	6

SHOW CARD A (Pension company list)
Q4 And which of these pension companies have you heard of?
MULTICODE POSSIBLE

	(25)
Allied Dunbar	1
Equitable Life	2
Prudential	3
Scottish Widows	4
Standard Life	5

SHOW CARD A (Pension company list)
Q5 And which of these have you heard or seen any advertising for recently?
MULTICODE POSSIBLE

	(26)
Allied Dunbar	1
Equitable Life	2
Prudential	3
Scottish Widows	4
Standard Life	5

ASK FOR ALL PENSION COMPANIES MENTIONED AT Q5
Q6 You mentioned advertising for -- pension company -- . Can you tell me where you have seen or heard that advertising?
DO NOT PROMPT. MULTICODE POSSIBLE.

	Radio	Poster	N’paper/ Mags	TV	Other
	(27)	(28)	(29)	(30)	(31)
Allied Dunbar	1	1	1	1	1
Equitable Life	2	2	2	2	2
Prudential	3	3	3	3	3
Scottish Widows	4	4	4	4	4
Standard Life	5	5	5	5	5

SHOW CARD A (Pension company list)
Q7 Which, if any, of these have you heard advertising on the radio recently?
And which, if any, have you seen advertising on posters recently?
And which, if any, have you seen advertising in newspapers/magazines recently?
And which, if any, have you seen advertising on TV recently?
MULTICODE POSSIBLE

	Radio	Poster	N’paper/ Mags	TV
	(32)	(33)	(34)	(35)
Allied Dunbar	1	1	1	1
Equitable Life	2	2	2	2
Prudential	3	3	3	3
Scottish Widows	4	4	4	4
Standard Life	5	5	5	5

ASK FOR ALL PENSION COMPANIES AWARE OF ADVERTISING ON RADIO AT Q’S 3, 6 & 7
Q8 You say you have heard advertising for -- pension company -- on radio recently.
- Could you describe the advertising you heard for -- pension company -- on the radio?
- And what was the main message of the advertising (apart from trying to get you to buy theirs)
PROBE FULLY WRITE IN FULLY
WRITE IN PENSION COMPANY, THEN THE COMMENTS ABOUT THAT COMPANY’S ADVERTISING

SHOW CARD B (Written description of radio commercials)
Q9 I’m going to read out descriptions of 3 radio commercials.

1. In this advert a newsagent talks about the way he was encouraged by his mother many years ago to start a pension early, and how glad he is that he did.
2. In this advert a woman who runs a busy restaurant talks about the way her personal pension company allowed her to take a break from her contributions while times were hard.
3. In this advert a man who is writing a book describes how his pension scheme has allowed him to do this because he has retired early.

Can you tell me if you have heard any of this, or similar, radio advertising on Capital recently?

------------------------------------------------------------------------------------------------------

ALL HEARD ANY OF THE RADIO ADVERTISING (any code 1 or 2 at Q9)

Q10 And thinking of the individual adverts, which ones have you heard?

	No 1	No 2	No 3
	Newsagent	Restaurant	Writer	Skip (44)
	(41)	(42)	(43)
Yes	1	1	1
Might have	2	2	2
No	3	3	3
Don’t know	4	4	4

ALL HEARD ANY OF THE RADIO ADVERTISING (any code 1 or 2 at Q9)
Q11 Which pension company was being advertised?

Q12 I’d like you to listen to these radio adverts.
PLAY TAPE OF ALL 3 ADVERTS. ENSURE ALL 3 ADVERTS ARE HEARD (ONCE ONLY).
Can you tell me if you have heard any of this advertising or advertising like it, recently?

-----------------------------------------------------------------------------------------------------

ALL HEARD ANY OF THE RADIO ADVERTISING (any code 1 or 2 at Q12)
Q13 And thinking of the individual adverts, which ones have you heard?
THEY MAY NOT BE IN THE SAME ORDER ON THE TAPE AS ON THE QUESTIONNAIRE SO PLEASE BE CAREFUL ABOUT GETTING THEM RIGHT WHEN CODING RESPONSES.

	No 1	No 2	No 3
	Newsagent	Restaurant	Writer	Skip (50)
	(47)	(48)	(49)
Yes	1	1	1
Might have	2	2	2
No	3	3	3
Don’t know	4	4	4

Male	42%
Female	58%
18-34	50%
35-54	50%
ABC1	52%
C2DE	48%
Listen every day	44%
Listen most days	56%