The Digital Inclusion Strategy action plan makes a commitment to conduct research to understand where and how to target our digital inclusion efforts, with Action 10 stating that government will ‘use data to measure performance and improve what we do’. To this end, we’ve convened a Research Working Group, which brings together Digital Inclusion Charter Signatories from government, business, academia and the voluntary sector, spanning research, policy and practice.
Together, we’re pooling our knowledge to enhance, standardise and spread best practice in measurement, and identify common outcomes to track our progress and evaluate diverse activities. We’re also working to develop a segmented understanding of our audience and of ‘what works’ in the context of diverse users needs and settings. This is critical to realising the benefits of going digital for all, and achieving maximum impact for minimum resources.
This blog is the first in a series that captures some of the things we’re putting our minds to. It focusses on the critical question of ‘what?’ i.e. what to measure, which is closely linked with the question of what we define as digital inclusion. Later posts will look at ‘who’, ‘where’ and ‘why’ and - critically - ‘how’ to respond.
We’ve started by categorising existing measures - each of which offers a viewpoint into things that matter for digital inclusion. These include:
- Access/use: Do people access and use the internet?
- Abilities: How competent and confident are people online?
- Activities: What do people do online?
- Attitudes: What attitudes do people hold towards the internet and technology?
- Outcomes: What benefits do people get from being online?
Measuring digital inclusion - access and beyond
More and more people have access to the internet, and are making wider use of all things digital, in their homes and while on the move. In 2014, 22 million households in the UK (84%) had Internet access, up from 57% in 2006. Among UK adults, 87% had used the internet in the last 3 months (up from 85% in 2013, and 82% in 2012) and 76% had used the internet daily. Access to the internet using a mobile phone more than doubled between 2010 and 2014, from 24% to 58%. (ONS 2014)
But, digital inclusion is not narrowly about the number of people who simply log-on once; and the way we measure and track our progress, and evaluate our efforts, needs to reflect this. Whether a person is accessing and using the internet is now really only one small part of the story - although it’s critical that we measure and track this as a minimum requirement. As the ubiquity and diversity of use increases we need to widen our focus beyond access and use, to look at how and why people use the internet (or don’t), their intentions, motivations and attitudes towards the internet and technology more broadly and, ultimately, the utility or value people gain from being online.
Attitudes - towards the internet and technology
While top line figures on access and use are promising, the challenge of getting the ‘final fifth’ of the population online is growing year on year. Only 29% of ex-users and 9% of non-users are planning on getting Internet access in the next year, a proportion that has been declining steadily since 2005 (OxIS 2013). And, increasingly, it’s not the practical barriers of cost of devices or access that are the crucial obstacle.
Rather, attitudes and motivations are becoming critical to (non) take up of digital technologies. ONS 2014 findings show that over half (53%) of those without internet access say they don’t have a connection because they 'do not need it’, up from 34% in 2006. And, among OxIS 2013 respondents, 82% of non-users and 44% of ex-users gave lack of interest as their most important reason for not using the internet. Over two thirds of retired non-users also gave “not for people my age” as a reason - which is very similar to lack of interest. Non-users are also more likely to express fears about the Internet or technology: 59% of non-users (vs 14% of users) fear they “might break” new technologies, while only 44% of non-users (vs 79% of internet users) agree that “technology makes things better”.
Attitudes might also be an indicator of the extent to which people who are online ‘feel digitally included’. According to the latest OxIS survey of British internet use and attitudes, more than half of those who go online “do it without enthusiasm”. Nearly one in six (14%) users felt the internet was taking over their lives and invading their privacy. Only 17% said it made them more efficient;12% said they were happy going online; and 19% had mixed views, feeling efficient and happier but also frustrated. So, while people are increasingly online, more needs to be done to ensure this adds meaning and value to their lives.
Abilities - how capable and confident people are online
Digital skills and capabilities are another area that we need to focus our attention on, as these are key to empowering non-users to get online, sustaining engagement over time, and giving users the confidence and competence to get the most out of being online.
Of households with no Internet access, 32% said that this was due to a lack of skills (ONS 2014), and over 60% of non-users cite not knowing how to use the internet as an important reason for non-use (OxIS 2013). Ex-users increasingly say they have stopped using the internet because they find it difficult to use (39% in 2013, compared with only 15% in 2005) possibly reflecting growing complexity of access with the rise of more devices and modes of access (OxIS 2013).
The latest BBC Media Literacy study shows that a persistent core of the population - around one in five people, or 10.5 million people - lack basic digital skills, defined as being able to send and receive emails, use a search engine, browse the internet, and fill out an online application form. Of this group, around three quarters are offline, leaving just over a quarter online (2.8 million) - indicating that our digital inclusion efforts should not be targeted narrowly at the offline population.
And, being competent online is about more than just the technical, task-based skills listed above. Even basic learners need to develop their critical cognitive skills in order to navigate the internet in a safe and savvy way. For example, online users need to know how to apply a degree of healthy scepticism to search results. Ofcom research indicates that many people lack this - finding that 22% of search engine users assume that sites must be trustworthy simply because they form part of a list of results.
Activities - what people are doing online
It’s been noted that ”the ability of the Internet to change or shape people’s lives is directly tied to their patterns of (non)use” (OxIS 2013, p. 25). People are increasingly using the internet to carry out and enhance key aspects of their lives - whether it’s maintaining or building new relationships through communicating and social networking, or entertainment and leisure activities, looking for information or learning, or carrying out commercial or government transactions. The types of activity, frequency, and range of things that people do online are all implicated in how much people are able to benefit from using the internet.
While headline stats tell us that more people are doing more things on line, more frequently, a significant minority still use the internet in a restricted way. Ofcom research gives us a view into this, by looking at ‘narrow’ users of the internet (those who only ever carry out between one and six types of 18 categories of internet use) who represent around 1 in 6, or 17% of the population. Narrow users are less likely to say they are confident users (59% vs. 86% for all users), and are more than twice as likely as all internet users not to trust any site to be secure (15% vs. 6% for all users). Narrow users are less likely to be aware of, and therefore to have installed security measures on their devices.
Outcomes of being online - digital as a means to an end
Ultimately, of course, the goal of our digital inclusion efforts is not to get people online as an end in itself. Rather, the real value of getting people online lies in its being a means to achieving wider social and economic outcomes like improved health or employment or reduced re-offending and loneliness. These outcomes are trickier to define, measure and track progress towards, but critical if we really want to measure the impact of our efforts and identify ‘what works’ and where to invest.
At a national level, general population surveys are trying to tap into and track how people benefit - or lose out - by being online. For example, OxIS captures things like whether users have saved money or time, found information to improve their health, or found a job online. At a project level, Citizens Online and BT have been tracking outcomes longitudinally with participants from their Get IT together programme. And, this data has fed into further analysis that demonstrates the social value of digital inclusion activities in the UK - the first time that SROI methodology has been applied to digital inclusion activities. This estimates the value of digital inclusion for a new user as £1,064 per annum. This comes from having more confidence, making financial savings online, less boredom, opportunities to pursue hobbies, new job-seeking skills, and a reduction in social isolation.
Towards shared frameworks for measuring and evaluating digital inclusion
Identifying and prioritising against wider outcomes, agreeing common measures, evaluating and testing what works, as well as iterating and making things better, is critical to realising the benefits of going digital and achieving maximum impact for minimum resources.
The first priority for our Research Working Group is to develop shared frameworks for measurement and evaluation. We’re devising a set of standardised, robust measures for benchmarking and tracking digital inclusion, and developing an overarching framework for evaluating the impact of digital inclusion activities, to enable us to identify, invest in, and share what works. We’re building on existing measures and approaches, and drawing in expertise and input from across the fields of research, policy and practice, to ensure that these are meaningful, easy to adopt, and fit for a range of purposes and audiences. We’ll then work together to promote their uptake across the digital inclusion sector.