Jamie Bartlett from cross-party thinktank Demos gives us the upsides and downsides of Big Data and how to use dashboards effectively.
Data dashboards are interfaces which display complex data to a user, often displayed in real time, and drawn from many sources. A few short years ago, these were the reserve of the specialist, the data people, the IT staff. But data dashboards are an important way in which any large organisation makes decisions.
The dramatic increase in the volume and nature of data being produced – the so-called big data revolution – has created new possibilities to understand trends, spot patterns, and collect intelligence, fuelling a burgeoning industry around ‘data analytics’[1]. Dashboards have become a window on the world of big data.
They bring bring varying types and quantities of data into a single user-friendly interface for visual representation, and they are the visible face of this revolution. They have become increasingly important as a way to help decision makers navigate an increasingly complicated data environment [2].
According to their advocates (which includes sales people at companies that sell dashboards) this new technology promise a smooth, data-driven decision making environment: cutting costs, providing new insight, and allowing for smarter policy and operational choices.
Dashboards have become a window on the world of big data.
And in many ways it does of course: according to the Gartner Survey, 73% of organisations have already invested or plan to invest in big data by 2016, spotting the potential [3]. In 2013, ‘big data’ was identified by the government as one of the UK’s ‘eight great technologies’.
At the time of writing, two providers of data visualisation software packages, Tableau and Qlik, claim to collectively provide 70,000 customers with the tools to build their own digital dashboards.
Given the low and falling cost of creating, capturing and storing data, and the ongoing challenge of communicating data to non-experts, the use of dashboards and dashboard-like visual formats are likely to continue to rise. Organisations around the world now routinely use dashboards to improve the way they make decisions.
But, because of their ubiquity, no-one seems to have noticed that data dashboards also introduce a significant shift in how organisations operate. They introduce new skills, dynamics, pressures and challenges into decision making. They can be used very well, or very badly.
One example is the fact that dashboards, almost by definition, condense data for easy digestion. Handy of course, but that can obscure a user’s knowledge of how trustworthy or accurate that data is. By presenting often very complex, messy and varied data in simplified forms for consumption via a dashboard, sometimes subtle changes take place in how we understand that data.
A danger is that the focus on design principles contributes to the obscuring of certain types of bias. People can get blinded by the pretty graphs and amazing looking trends. More importantly, dashboards introduce a new emphasis on metrics, indicators and measures.
That can create a greater focus on operational issues rather than longer-term strategic ones. As with any new discipline, new types of expertise become valued, and new sources of authority become established.
These problems are not easily overcome, and they apply to all organisations that use dashboards. But, two main principles can help organisations make the most of the new opportunities dashboards present. Given the obvious organisational change dashboards create, for learning and development professionals, this is particularly important.
First, new staff and new skills are required. The skills needed to create and manage dashboards are valuable and sought after by both the private and public sectors. These skills are often composite.
A whole new generation of analysts will be required who can combine a new combination of skill sets; ranging from data analytics, design, social science, and public policy. What’s exciting about data dashboards is that people from all across an organisation can pick this up as part of professional development.
Those who are able to decipher algorithms or critically engage with big data sources will inevitably be in high demand. Of course there is the risk that these highly skilled individuals will be difficult to find in large quantities, and/or that they will be poached from public services by private firms. So, organisations will also need to develop new approaches to how to recruit, train, and retain these individuals.
Second, we will need new techniques to use dashboard data properly. Dashboards are a broad, generic approach to collecting, analysing and acting on large data sets. In and of themselves, they are not necessarily the best way of understanding all categories of problems, or research questions.
Dashboards must be designed to match real organisational needs, their design and purpose assessed in relation to the hoped-for purpose. Further, there are several factors that may lead to poor, bias, insufficient, or irrelevant data being used in dashboards.
The limitations of dashboards need to be acknowledged and, where possible, amended. On a practical level, this data may not measure what it seeks to, instead being cut and scraped until it ‘looks right’. Users can be blinded by large numbers, or have insufficient understanding of the strengths and weaknesses of the data they are using.
Dashboards have the potential to mislead as well as inform.
[1] Dashboard Development and data visualization tools for Effective BI
[2] Maximise sales with deeper and faster trade promotion planning
[3] Marr, B. (2015) Big Data: 20 Mind-Boggling Facts Everyone Must Read
About the author
Jamie Bartlett is Director of the Centre for the Analysis of Social Media at the thinktank Demos and author of the recent report Governance by Dashboard.
