The value of user research

Redeveloping the Diabetes UK online risk score – a test for Type 2 risk

In my post about becoming vegan I mentioned how it had helped me finally reach a fitness goal – of reducing my waist size. If you’ve met me you might not think I have a particularly large waist and you might wonder why it was was a goal for me to reduce it – but working for Diabetes UK I’m conscious of really how relatively small your waist size needs to be before your risk of developing Type 2 diabetes is increased. Especially when, as in my case, this is combined with age, family history of the condition, and my particular ethnicity.

There has also been coverage in the press today (31 July) about Public Health England’s report on the relationship between Type 2 and obesity – for example, on the Diabetes UK websitethe Guardian and BBC News online – with a clear emphasis on waist size and belly fat as a measure for Type 2 risk.

Online risk score test, version 1 – mid 2010 to mid 2013

Diabetes UK launched its online risk score www.diabetes.org.uk/riskscore – to test your level of risk of developing Type 2 – on 1 July 2010. Developed with the University of Leicester and the University Hospital of Leicester NHS Trust, a paper version of the questionnaire is used in face-to-face tests at Diabetes UK events. Answering seven questions gives a score from 1 to 47, which puts you into one of four risk levels: Low, Increased, Moderate and High. If you score 16 or above – Moderate or High – we recommend you see your GP for a blood test, with the option for users to be sent an email with a GP referral letter attached.

We in the Digital team at Diabetes UK developed an interactive online version of the tool, with the agency Nonsense, on a small budget, in a quick turnaround time, working closely with the charity’s Clinical team of diabetes healthcare professionals. The first iteration centred around photos and testimonies of real people, each with one of the four risk levels. After answering each question, the user saw a pop-up with a comment from the relevant case study, and their final result was also accompanied by a case study.

As a charity with a limited budget we were unable to make significant improvements to the risk score until 2013, when we received funding as part of charity corporate partnership. In the meantime, I did develop a handful of user stories to inform possible development should funding become available.

Last month the online risk score passed the milestone of 500,000 uses since launch, and on 1 July it reached its fourth birthday, so, with a lot of recent media coverage about the issue of type 2 risk, I thought now would be a good time to blog about it, and what has happened in the year since I embarked on the redevelopment.

Aims of the revamp, mid 2013 onwards

The charity planned a major outdoor and online advertising campaign to help identify some of the seven million adults at most risk of developing Type 2 diabetes. The advertising in autumn 2013 encouraged people to see their GP or take the test online so, we wanted to ensure the test would be effective.

I knew from our initial user stories that we had to update the technology (with the advent of touchscreens, and responsive mobile design) but also wanted to fully optimise the test by understanding how people used the tool and any problems they faced. So we commissioned ethnographic 1-1 user research with current users of the risk score, focusing on the most at-risk C2DE socio-economic groups – taking place in users’ homes, with their own devices.

The testing told us we had to improve the risk score before making it a key part of the campaign. Some changes were because technology had moved on, but most were because of insights into how users in the target groups approached and used the tool. Personally I found the testing results very eye-opening. There were low levels of digital literacy, even if people were using more up-to-date devices such as smartphones.

Our redesign addressed the following issues.

Wrong measurements:

  • Users did not actually measure their weight or waist size – eg giving their jeans waist size, which can be smaller – so the importance of accurate measurements needed to be stressed up front and with each relevant question
  • We saw that this would be exacerbated if using on a mobile device, having just seen an ad when out and about, without a tape measure or scales to hand – so we realised that a reminder for mobile users to re-take the test at home would be useful.
  • Usability issues, especially on touchscreens – eg unable to operate ‘slider’ controls, or preferring to type in values.
  • Some users didn’t realise that the sliders required actual measurements to be input – instead they just chose a position based on a more general ‘low to high’ or ‘underweight to obese’ scale.

The critical issue here is that this under-reporting means that many users would be scoring too low and not being placed in the right result for the risk band (Low/Increased/Moderate/High).

We found it would help if we show users their actual, numerical risk score too – to show how close they are to the next level up or down.

‘Test mode’ vs ‘learning mode’:

  • Users were in ‘test mode’ while taking the test, expecting to be able to work their way through the questions quite quickly without distraction.
  • As such, they weren’t in ‘learning mode’ and tended to miss or ignore the pop-ups, or felt they got in the way.
  • Preferred a simpler, cleaner design for the questions.

Conversely, users expected more information at the end:

  • Disappointed by the promised ‘full report’ just briefly reiterating the score/probability they’d seen on screen.
  • Expected fuller, personalised information at this point (after the test) about the questions and their answers, and what impact this had had on their final result.
  • Also hoping for more information about what to do next – more compelling/urgent, and more practical guidance (what to do, who to speak to).

Sharing or recommending by social media or email:

  • All users thought that this function meant their personal, individual test results would be shared – rather than just a link to take the test.
  • Needs to be clear to the user and the people they share with why it’s important to take the test, how common Type 2 is, how many people are undiagnosed without knowing it, etc – to reinforce the need to test, but to find a balance between highlighting key reasons to take the test (eg ‘you might have one of the risk factors, it’s a common condition, with many people undiagnosed’) but depersonalising it (eg ‘I’m not saying you are overweight’).
  • No-one wanted to share the test before they got their results.They couldn’t see why they would recommend the test – would their friends/family find it offensive – would they be suggesting they were unhealthy/overweight?
  • No-one wanted to share the test before they got their results.

Design:

  • Some users also found the numbering of the screens at the bottom confusing – positioning and design not distinct enough from the main input screens.

The revamp

The main elements of the 2013 revamp were:

  • Responsive redesign, so that the test scales and reformats to suit multiple screen sizes.
  • Introduction of an alert to save to home screen if it senses a user is using a mobile, and offer of a reminder email – so users can more easily take the test when they’re at home, with scales and a tape measure.
  • Cleaner/clearer design for each question screen, stripping away pop-ups of case studies.
  • Improved language on all questions – eg making it clear what counts as a close relative, and giving a wider range of options for ethnicity.
  • Clear emphasis on the importance of taking actual, current measurements of weight and waist size, and not relying on memory, estimation or jeans waist size.
  • Much improved results screen – including:
    • giving a full breakdown of points per question, what the scoring means, and which risk factors can be addressed (eg weight, waist size), which are simply genetic (ethnicity, family history of diabetes) and which will change over time (age)
    • showing your full score in points, as well as which of the four risk bands you are in, making it clearer how close you are to the next risk level up or down
    • reserving the case study content until after seeing your results, instead of throughout the test
    • giving clear links to information on the Diabetes UK website about what to do next to reduce your risk.
    • allowing simple social media sharing but making it clear that only the test, not your results, would be shared.

Results since the revamp

We provide risk assessments in person at our programme of roadshows across the UK, as well as in pharmacies of our corporate partners. This helps us to assess more than 25,000 people each year. And in a typical year, before the redesign, we had between 75,000 and 100,000 uses of our online risk score.

Using digital media for an online version of the risk score, promoted through our email marketing, social media, grant-funded Google Adwords account and through users sharing the tool, helps us to reach at least three times as many people as the face-to-face risk assessments that require more staff and financial resource, and rely on footfall.

In fact, with the success of the redesign we have now reached 300,000 users online in the past year (July 2013 to June 2014) – 12 times as many as we can reach through face-to-face assessments.

We saw immediate improvement in usage, referrals and results following the redesign.

  • Year 1: 100,000 uses
  • Year 2: 75,000
  • Year 3: 75,000
  • Year 4: 300,000 (25,000 pre-relaunch, 275,000 post-relaunch)

The relaunched test saw more uses (30,000) in the fortnight of the ad campaign, than it saw in the previous three months (25,000). We have seen more uses (245,000) in the nine months after the end of the campaign than we saw in the whole first two years (175,000).

Since relaunch, from 275,000 tests, the test has referred 75,000 people to their GPs.

This compares with 28,833 people referred after 175,000 tests in the previous 27 months, and shows the benefits of user-centred redesign, supported by focused audience research, and a boost from advertising.

Significantly, the proportion of people being referred increased from 16.5% to 27.3%, suggesting that the redesign and improved usability reduced the under-reporting of weight and waist size.

Passing the 500,000 mark:

On 10 June the online risk score passed the landmark of 500,000 uses since its launch nearly four years before. This was partly helped by a journal article predicting a large increase in Type 2, covered by the national press, and including a link to the risk score from a BBC online news piece. In fact, on the day, I watched the counter jumping up and up with every browser refresh – so much so, I thought it would be ideal for a quick Vine.

But even without this boost, we were already seeing the residual effect of the campaign and the effect of the redesign, and increased public awareness supported by regular media coverage of the issues.

And by its fourth birthday, with less than a year since the redesign, the online risk score had passed 550,000 uses – doubling the number in less than a year since the redesign, compared with the number of uses in the previous three years+.

User survey results:

We send follow-up emails to users – an average of 2,500 a month begin an email journey to support them in making lifestyle changes.

We surveyed 8,900 people who used the online test in the month following the relaunch:

  • 97% found the test easy to complete
  • 90% found the online experience helpful; just 2% disagreed and 8% were neutral
  • 83% said they would visit their GP; 38% already had – leading to 49% of these being diagnosed with Type 2 diabetes
  • 85% said they would now eat more healthily
  • 81% said they would exercise more.

Next steps

We’re not stopping there – we are now looking at the existing user stories that we haven’t had development funding for before, such as an offline electronic version for use at roadshows and other events, to help address the current issue of timelag between the data passing from point of capture and going into our CRM system for email follow-ups.

And we plan to do further assessment of the follow-up emails after this year’s programme of roadshows is complete.

Advertisements

The British and Modern Architecture – part 2

Triumphs, Carbuncles and Hopeful Monsters – the British vs Modern Architecture I really enjoyed this RIBA lecture. It was billed as more of a discussion panel but in fact was a series of very different individual presentations from a great range of speakers: Catherine Croft, Director of the Twentieth Century Society; Owen Hatherley, Writer; Hugh Pearman, Editor of RIBA Journal; Kester Rattenbury, architectural journalist/ critic/ author/ teacher; and Daisy Froud, AOC. It was followed by a discussion but quite a brief one, perhaps because the speakers overran – including a funny moment when Owen Hatherley resisted the two-minutes-to-wrap-up signal from the chair (Charlie Luxton off the telly), before just acknowledging he was going to take a bit longer to make his point, and it was a good point!

When I go to events and lectures these days it’s usually work-related and most of the audience are live tweeting or at least following the live tweets (although perhaps not quite as much now as the frenzy of a few years ago). So it was interesting and sort of refreshing that no-one seemed to be live tweeting from this event. With my usual reserve, and concern about what other people think, I didn’t want to look disinterested or like I was just checking my messages, so, leaving my phone put away and lacking another way of taking notes, I just sat back and enjoyed the event. This means my recollection might be a bit patchy but it was a good exercise in trying to really listen, absorb and remember what was said. Here are my highlights.

The theme – the British vs Modern Architecture

This theme, and the speakers, is what drew me to the lecture but it’s a reflection of how interesting the speakers were anyway that it didn’t really matter that they sometimes strayed off-topic. The event marked the 30th anniversary of Prince Charles’s ‘Monstrous carbuncle’ speech and, among other things, asked whether he was abusing his privileged position by attacking specific public architectural projects or in fact speaking up on behalf of the average person who supposedly shared his views but just lacked the platform to make them heard or make a difference. Just for clarity, I think it was the former, but some of the speakers made some excellent points about how effectively the latter perception took hold, fuelled by the media.

The speakers

None of panel were practising architects, though some had trained as architects – I think this point was highlighted by one of the speakers – so in some ways were just as representative of public as any one of us. A key point made was that there’s no single ‘man in the street’, no dominant viewpoint, and that individual reactions to architectural style and meaning are generated by a complex range of experiences and influences.

Catherine Croft

Croft started by sharing some of her experiences of Modernism when studying Architecture. And she made the first mention of the night of the book Britain by Alan Powers, from the Modern Architectures in History series, which I will now be reading over the summer. She went on to mention some of the Modernist buildings in Britain threatened with demolition and which the Twentieth Century Society has fought to try to protect.

I was interested in the story of the 1930s house Greenside in Surrey by Connell, Ward and Lucas. Locals – and apparently even its owners – wanted to see it demolished in favour of something in a more traditional style. It was built overlooking the 17th hole of Wentworth golf course, and was blamed by locals for putting people off their shots through its perceived ugliness or somehow through sheer disbelief that it was Grade II listed. The Twentieth Century Society fought for it to be preserved but this initially funny story of Middle England’s knee-jerk distaste at Modernist design has a sad ending – the owner demolished it without consent in 2003, and one of relatively few examples of Modernist homes in Britain was lost.

Croft finished by highlighting a newer building at risk of demolition – the Edward Cullinan-designed Ready Mix Concrete HQ in Surrey. Despite having won awards for the sensitivity with which it is designed, inserted among historical buildings, it is under threat and the Twentieth Century Society is campaigning for it to be saved:

“The offices are now under threat from demolition. Cemex (owners of RMC since 2005) are preparing to leave the site, and a scheme to redevelop the offices for housing has been submitted for planning. A decision on the planning application is expected in summer 2014, but all eyes will be on the forthcoming listing recommendation from English Heritage, due very shortly.”

Croft ended her talk by imploring the audience to support the campaign to preserve it – and in fact last week it was effectively saved, when it was given Grade II listing status.

Owen Hatherley

I’m a fan of Owen Hatherley’s writing, including his book Common, about Pulp, but mainly his writing on architecture and other subjects in the Guardian. (I also enjoyed this interview with him on the Quietus website.)

He started his presentation with ‘Your Britain: Fight for it now’ by Abram Games, a health poster from the 1930s showing Finsbury Health Centre. The centre was groundbreaking, offering free healthcare to local residents using modern methods, more than a decade before the NHS was established. Hatherley said that Churchill vetoed the poster at the time, but not because it was not resonant – in fact, seemingly the opposite – maybe because it was more of an ad for Labour’s effectiveness.

Finsbury Health Centre was designed by Lubetkin’s Tecton practice, and Hatherley quoted Lubetkin: “Nothing is too good for ordinary people!”

Hatherly next talked about Park Hill housing estate in Sheffield, designed to retain communities and social networks – “not throwing the baby out with bathwater”. He said that Park Hill was more genuinely mixed use than some of today’s modern developments  – it had three pubs, a variety of shops, a butcher and more. At Park Hill, “streets in sky endured” – there might have been muggings but no more than in conventional streets!

I liked his point that Park Hill was the first time Modernism recognised the British weather conditions – it was built in subtly-textured brick to absorb rain and soot and not look stained. (I think the architects of the Barbican later had a similar idea, giving the concrete a rough texture to disguise the pollution and rain the surface would inevitably become stained with.) It was really interesting to note that there was no bare concrete inside the flats – until the more recent refurbishment, by which time tastes had changed and people seem to want that, to see the more rough and basic materials as more ‘authentic’ somehow.

Hatherley said that the lauded regeneration of the estate has only actually refurbished one fifth of the estate. And two similar estates were completely or partially demolished. He said that Park Hill’s fame spread an it was even featured in a French/Russian language book about modern housing in Sheffield.

There have been a lot of articles recently about city skylines, the literal rise of office towers, especially in London – and I agree with his feeling that towers would be more justified if they were thoughtful designs to address the chronic shortage of homes – ” the housing issue is more important than skyline issue”.

In the brief Q&A session at the end of the talks there was a good question from someone in audience: “Have we reached peak square?” – referring to the trend for urban regeneration to feature an almost obligatory square in every design, every CAD representation. Hatherley noted that Elephant & Castle in London fails what might be called the Jane Jacobs/street syntax test but still contains more genuine shops, a mix of people and uses, and very few chain stores compared to today’s developments. But is nevertheless being demolished for regeneration.

Hugh Pearman

Hugh Pearman set the scene for Prince Charles’s speech and the media attention that was to greet it, by giving a brief history of Modern architecture in the UK after the immediate rebuilding following WWII. After the Ronan Point disaster, where people died in an explosion in a poorly built block of flats, architects were soul-searching about current approaches to Modernism. RIBA even published a book in 1974 about it.

The debate continued for about 10 years and architects were just starting to feel more positive … when they invited Prince Charles to Charles Correa’s RIBA Gold Medal ceremony in 1984, inadvertently giving him the platform to essentially insult his hosts and most of their profession.

Pearman’s was a very funny talk, with lots of well-chosen photos of not-so-good buildings in the City of London – showing the horrors of commercial Postmodernism – but through the prism of the evolutionary idea of ‘Hopeful Monsters’, that not all developments can happen gradually. Gradual evolution could account for small-scale developments within species but for whole new species to develop there must also have been some real evolutionary leaps – producing ‘hopeful monsters’, freakishly different but hopeful that at least a tiny percentage would take hold and continue a new line.

Pearman mentioned the plan from the 1960s to replace Mansion House in London with a Mies van der Rohe tower (which the Prince alluded to in his speech when he mentioned ‘glass stumps’) – which got planning permission, permission that elapsed in the 20 years it took the developer, Peter Palumbo, to buy up all the buildings on site. The public had liked the design for the building – but the tide was turning by 1984, and the plan was up for planning review around the time of the speech… and subsequently didn’t make it through.

The site ended up with James Stirling’s No.1 Poultry building instead, an early example of ‘proper’ Postmodernism, a design likened to a hen on a nest. This style became popular with the public and led to lots of buildings in and around City “designed to look like what people thought Prince Charles wanted them to look like”. Essentially varying degrees of hideousness, mostly pastiche Postmodernism (almost a tautology?) featuring lots of pink and grey stone.

Pearman also noted that squares used to be public spaces but have gradually and quietly become private – and were always able to be shut down easily in case of public unrest. For example, Paternoster Square near St Paul’s Cathedral, which meant the Occupy London protesters had to set up camp not in the square but alongside the cathedral …

Kester Rattenbury

… But Kester Rattenbury gave the Barbican as an example of a Modernist public space that was designed without a square – saying that Modernism tends not to favour traditional spaces such as squares.

Rattenbury said that in his speech the Prince was alluding to old, poorly-produced 60s buildings, rather than the more recent contemporary crop of more thoughtful modern designs – for example housing schemes such as those by Ted Cullinan and Ralph Erskine; and Oxford Polytechnic accommodation?

I’d never thought of it quite this way before but she made the excellent point that, oddly, and paradoxically for the heir to the throne, Charles effectively became the voice of the ‘man in the street’ or represented the ‘common man’ – and in this regard his lack of architectural knowledge worked in his favour.

Interestingly, Rattenbury chose for her PhD to study media representation of architecture, mainly because of the prince’s speech. As part of the study she monitored six national newspapers, including tabloids, over a year, and found architecture and building stories tended to focus on homes and housing, rather than big public schemes – perhaps the beginnings, she noted, of the TV obsession with homes, makeover shows – which, in a funny way, all paved the way for Kevin McCloud and Grand Designs…!

She said that essentially the media always reduces arguments to two sides, puts them in conflict, with no space for detail or subtlety of debate – and it was no different for the public debate on architectural design.

Daisy Froud

I hadn’t heard of AOC before but their work looks really interesting, and Froud has been involved in quite a lot of community action around building development. She raised the really good point that Modernism has in recent decades been more of a style debate rather than an ethos or an approach to design. I’ve definitely been guilty of this myself.

In trying to answer the question of why members of the British public tended not to favour Modernist architecture she said it was often not designed to meet people’s needs and so people mistrust it – maybe also because Modernism was adopted by big, rich companies for their buildings and so became representative of them to some extent? (I think it’s interesting that so many star architects start with domestic-scale housing developments or single homes but ‘progress’ to big office buildings.)

“Worth it in the long run”

Modernism shouldn’t just be thought of in purely aesthetic terms but so often it, and any objections to it, are boiled down to a simplistic, stylistic subjectivity. Viewed more widely than aesthetics, Modernism deserves to be seen as an approach to planning and design based on the belief that everyone has a right to benefit from buildings that are thoughtfully designed to provide space and light and that make the best use of contemporary building materials and methods. And an appreciation, or otherwise, of modern/Modern/Modernist architecture shouldn’t be divorced from the prevailing politics and public opinion of the times in which it is designed and built.

Design is essentially an approach to solving a problem or meeting a need, but I think this is often overlooked in the rush to subjective condemnation of appearance or style. Any failings of Modernist architecture seem so often linked in the views of the public (fuelled and reflected especially by a conflict-seeking media) with failings within society – specifically of Modern architecture being almost synonymous with deprived, run-down council housing estates and tower blocks – but this connection is too simplistic and ignores both the varying quality of design and construction as well as will (or lack of will) of governments to support and maintain it. It reminds me of this counter-argument, from a post in 2012 on Hampstead’s Branch Hill Estate by one of my favourite bloggers, Douglas Murphy on his Entschwindet und vergeht blog:

“When you look at the mass-housing buildings from the heroic post-war period, one rule of thumb seems to be glaringly obvious; the buildings that have survived to the present day were frequently the ones with bespoke designs, while those most swiftly removed are frequently the system-built, pre-fab panel construction buildings. Obviously there are exceptions to this (in Glasgow, Basil Spence’s Hutchesontown flats are long demolished but the Red Road Flats are only just going down), but a general rule seems to be that if you build it properly, if you spend the money on it in the first place, then it’s worth keeping. Indeed; this is the same for every generation of building; we love our Georgian terraces because the slums were pulled down, we love our Glasgow tenements because the single-ends were torn down. Now; we love our Balfron Tower at least partly because the Freemasons Estate was torn down.

“Does this mean that a lesson to be drawn is that architects should attempt to spend as much as they possibly can, that budgets are irrelevant to the task at hand? Of course not. Architects need no extra help in rendering themselves redundant to the modern construction industry. But perhaps the lesson here is that there might just be a secondary economic effect involved in successful estates, that it is not quite enough to have wealthy residents, but that if it’s designed thoughtfully, and built properly, then it will be worth it in the long run.”