The value of user research

Redeveloping the Diabetes UK online risk score – a test for Type 2 risk

In my post about becoming vegan I mentioned how it had helped me finally reach a fitness goal – of reducing my waist size. If you’ve met me you might not think I have a particularly large waist and you might wonder why it was was a goal for me to reduce it – but working for Diabetes UK I’m conscious of really how relatively small your waist size needs to be before your risk of developing Type 2 diabetes is increased. Especially when, as in my case, this is combined with age, family history of the condition, and my particular ethnicity.

There has also been coverage in the press today (31 July) about Public Health England’s report on the relationship between Type 2 and obesity – for example, on the Diabetes UK websitethe Guardian and BBC News online – with a clear emphasis on waist size and belly fat as a measure for Type 2 risk.

Online risk score test, version 1 – mid 2010 to mid 2013

Diabetes UK launched its online risk score – to test your level of risk of developing Type 2 – on 1 July 2010. Developed with the University of Leicester and the University Hospital of Leicester NHS Trust, a paper version of the questionnaire is used in face-to-face tests at Diabetes UK events. Answering seven questions gives a score from 1 to 47, which puts you into one of four risk levels: Low, Increased, Moderate and High. If you score 16 or above – Moderate or High – we recommend you see your GP for a blood test, with the option for users to be sent an email with a GP referral letter attached.

We in the Digital team at Diabetes UK developed an interactive online version of the tool, with the agency Nonsense, on a small budget, in a quick turnaround time, working closely with the charity’s Clinical team of diabetes healthcare professionals. The first iteration centred around photos and testimonies of real people, each with one of the four risk levels. After answering each question, the user saw a pop-up with a comment from the relevant case study, and their final result was also accompanied by a case study.

As a charity with a limited budget we were unable to make significant improvements to the risk score until 2013, when we received funding as part of charity corporate partnership. In the meantime, I did develop a handful of user stories to inform possible development should funding become available.

Last month the online risk score passed the milestone of 500,000 uses since launch, and on 1 July it reached its fourth birthday, so, with a lot of recent media coverage about the issue of type 2 risk, I thought now would be a good time to blog about it, and what has happened in the year since I embarked on the redevelopment.

Aims of the revamp, mid 2013 onwards

The charity planned a major outdoor and online advertising campaign to help identify some of the seven million adults at most risk of developing Type 2 diabetes. The advertising in autumn 2013 encouraged people to see their GP or take the test online so, we wanted to ensure the test would be effective.

I knew from our initial user stories that we had to update the technology (with the advent of touchscreens, and responsive mobile design) but also wanted to fully optimise the test by understanding how people used the tool and any problems they faced. So we commissioned ethnographic 1-1 user research with current users of the risk score, focusing on the most at-risk C2DE socio-economic groups – taking place in users’ homes, with their own devices.

The testing told us we had to improve the risk score before making it a key part of the campaign. Some changes were because technology had moved on, but most were because of insights into how users in the target groups approached and used the tool. Personally I found the testing results very eye-opening. There were low levels of digital literacy, even if people were using more up-to-date devices such as smartphones.

Our redesign addressed the following issues.

Wrong measurements:

  • Users did not actually measure their weight or waist size – eg giving their jeans waist size, which can be smaller – so the importance of accurate measurements needed to be stressed up front and with each relevant question
  • We saw that this would be exacerbated if using on a mobile device, having just seen an ad when out and about, without a tape measure or scales to hand – so we realised that a reminder for mobile users to re-take the test at home would be useful.
  • Usability issues, especially on touchscreens – eg unable to operate ‘slider’ controls, or preferring to type in values.
  • Some users didn’t realise that the sliders required actual measurements to be input – instead they just chose a position based on a more general ‘low to high’ or ‘underweight to obese’ scale.

The critical issue here is that this under-reporting means that many users would be scoring too low and not being placed in the right result for the risk band (Low/Increased/Moderate/High).

We found it would help if we show users their actual, numerical risk score too – to show how close they are to the next level up or down.

‘Test mode’ vs ‘learning mode’:

  • Users were in ‘test mode’ while taking the test, expecting to be able to work their way through the questions quite quickly without distraction.
  • As such, they weren’t in ‘learning mode’ and tended to miss or ignore the pop-ups, or felt they got in the way.
  • Preferred a simpler, cleaner design for the questions.

Conversely, users expected more information at the end:

  • Disappointed by the promised ‘full report’ just briefly reiterating the score/probability they’d seen on screen.
  • Expected fuller, personalised information at this point (after the test) about the questions and their answers, and what impact this had had on their final result.
  • Also hoping for more information about what to do next – more compelling/urgent, and more practical guidance (what to do, who to speak to).

Sharing or recommending by social media or email:

  • All users thought that this function meant their personal, individual test results would be shared – rather than just a link to take the test.
  • Needs to be clear to the user and the people they share with why it’s important to take the test, how common Type 2 is, how many people are undiagnosed without knowing it, etc – to reinforce the need to test, but to find a balance between highlighting key reasons to take the test (eg ‘you might have one of the risk factors, it’s a common condition, with many people undiagnosed’) but depersonalising it (eg ‘I’m not saying you are overweight’).
  • No-one wanted to share the test before they got their results.They couldn’t see why they would recommend the test – would their friends/family find it offensive – would they be suggesting they were unhealthy/overweight?
  • No-one wanted to share the test before they got their results.


  • Some users also found the numbering of the screens at the bottom confusing – positioning and design not distinct enough from the main input screens.

The revamp

The main elements of the 2013 revamp were:

  • Responsive redesign, so that the test scales and reformats to suit multiple screen sizes.
  • Introduction of an alert to save to home screen if it senses a user is using a mobile, and offer of a reminder email – so users can more easily take the test when they’re at home, with scales and a tape measure.
  • Cleaner/clearer design for each question screen, stripping away pop-ups of case studies.
  • Improved language on all questions – eg making it clear what counts as a close relative, and giving a wider range of options for ethnicity.
  • Clear emphasis on the importance of taking actual, current measurements of weight and waist size, and not relying on memory, estimation or jeans waist size.
  • Much improved results screen – including:
    • giving a full breakdown of points per question, what the scoring means, and which risk factors can be addressed (eg weight, waist size), which are simply genetic (ethnicity, family history of diabetes) and which will change over time (age)
    • showing your full score in points, as well as which of the four risk bands you are in, making it clearer how close you are to the next risk level up or down
    • reserving the case study content until after seeing your results, instead of throughout the test
    • giving clear links to information on the Diabetes UK website about what to do next to reduce your risk.
    • allowing simple social media sharing but making it clear that only the test, not your results, would be shared.

Results since the revamp

We provide risk assessments in person at our programme of roadshows across the UK, as well as in pharmacies of our corporate partners. This helps us to assess more than 25,000 people each year. And in a typical year, before the redesign, we had between 75,000 and 100,000 uses of our online risk score.

Using digital media for an online version of the risk score, promoted through our email marketing, social media, grant-funded Google Adwords account and through users sharing the tool, helps us to reach at least three times as many people as the face-to-face risk assessments that require more staff and financial resource, and rely on footfall.

In fact, with the success of the redesign we have now reached 300,000 users online in the past year (July 2013 to June 2014) – 12 times as many as we can reach through face-to-face assessments.

We saw immediate improvement in usage, referrals and results following the redesign.

  • Year 1: 100,000 uses
  • Year 2: 75,000
  • Year 3: 75,000
  • Year 4: 300,000 (25,000 pre-relaunch, 275,000 post-relaunch)

The relaunched test saw more uses (30,000) in the fortnight of the ad campaign, than it saw in the previous three months (25,000). We have seen more uses (245,000) in the nine months after the end of the campaign than we saw in the whole first two years (175,000).

Since relaunch, from 275,000 tests, the test has referred 75,000 people to their GPs.

This compares with 28,833 people referred after 175,000 tests in the previous 27 months, and shows the benefits of user-centred redesign, supported by focused audience research, and a boost from advertising.

Significantly, the proportion of people being referred increased from 16.5% to 27.3%, suggesting that the redesign and improved usability reduced the under-reporting of weight and waist size.

Passing the 500,000 mark:

On 10 June the online risk score passed the landmark of 500,000 uses since its launch nearly four years before. This was partly helped by a journal article predicting a large increase in Type 2, covered by the national press, and including a link to the risk score from a BBC online news piece. In fact, on the day, I watched the counter jumping up and up with every browser refresh – so much so, I thought it would be ideal for a quick Vine.

But even without this boost, we were already seeing the residual effect of the campaign and the effect of the redesign, and increased public awareness supported by regular media coverage of the issues.

And by its fourth birthday, with less than a year since the redesign, the online risk score had passed 550,000 uses – doubling the number in less than a year since the redesign, compared with the number of uses in the previous three years+.

User survey results:

We send follow-up emails to users – an average of 2,500 a month begin an email journey to support them in making lifestyle changes.

We surveyed 8,900 people who used the online test in the month following the relaunch:

  • 97% found the test easy to complete
  • 90% found the online experience helpful; just 2% disagreed and 8% were neutral
  • 83% said they would visit their GP; 38% already had – leading to 49% of these being diagnosed with Type 2 diabetes
  • 85% said they would now eat more healthily
  • 81% said they would exercise more.

Next steps

We’re not stopping there – we are now looking at the existing user stories that we haven’t had development funding for before, such as an offline electronic version for use at roadshows and other events, to help address the current issue of timelag between the data passing from point of capture and going into our CRM system for email follow-ups.

And we plan to do further assessment of the follow-up emails after this year’s programme of roadshows is complete.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s