New ways to run engagement surveys

Andrew Marritt

He is the founder of OrganizationView, a people analytics practice. He is an expert in Big Data and Machine Learning techniques to redefine how businesses continually learn from their employees.

Latest posts by Andrew Marritt (see all)

Andrew Marritt describes the growing use of innovative technology to measure employee engagement.

“A new market has emerged: Employee feedback apps for the corporate marketplace. These tools are powerful and disruptive, and they have the potential to redefine how we manage our organisations.”

Josh Bersin, Forbes, August 26 2015

The measurement of employee engagement is changing. Businesses have been measuring engagement for several decades and the market is currently estimated to be worth $1bn per annum yet most reports suggest that engagement is trending flat if not actually decreasing. Something is obviously not working.

There are many reasons businesses are growing frustrated with current methods. Slow, expensive and resource-intensive are some of the more commonly heard. In many businesses the only internal things that are now measured on an annual cycle are engagement and performance management – both run by HR. In both cases, business leaders are demanding better results and more real-time insight.

Today many firms are capturing an “alwayson” stream of customer data from a wide variety of channels from short surveys to social media. Here, the emphasis has switched to continual listening, rapid resolution and bringing deep insights of customer needs into everything from product development to service provision.

Many of the trends observable in engagement measurement can in fact be viewed as an application of a “voice of the customer philosophy” by employees. At the same time there is a shift in the demand side to run surveys differently. We are also seeing a shift in supply caused by technology. These technology changes can be classified into four categories:

Automation
Technology-led automation is something that is happening across society and it should be no surprise that it is also surfacing in engagement measurement.

Firms are automating the parts of the engagement measurement and analysis process that before was typically done by analysts. It started with relatively simple report automation: the production of thousands of template-based pdf reports moments after a survey closed. While this might, and should, have been utilised by firms for some time to increase their margins, now most of the new entrants are using automation to reduce prices and complexity radically. The new business models are disruptive.

Real-time reporting via dashboards is becoming the norm. We are seeing a shift from the multipage result presentation to one-page infographicstyle reports. Ultimately there is a shift from seeing the provision of a large numbers of individual reports as complexity to seeing it as a commodity solution.

Mobile and user interface changes
The second technology-led shift has to do with the way that employees are able to take surveys.

Mobile has become the dominant channel for employees to take surveys. We see respondents taking surveys just before the work day, during lunchtime and even in the evening. They’re interacting on the edges of their days and grabbing a mobile device to do it.

Consumer web technologies have changed the way we expect to interact with our devices and engagement surveys cannot escape this trend.

Many of the question types used were the same as we used on paper but digitalised. Digital-only surveys are not bound by these constraints. In a world where people expect to touch, slide and scroll through long-form sites, survey interfaces need to adapt to become more user-friendly.

Big data technology
The majority of the new entrants are focusing on the previous two technologies. While this is right for medium-sized businesses, larger companies and organisations typically have a set of needs that extend these simple cases.

Whereas employee survey data has historically been treated as an island – analysed in the context of the perception of data or a predefined limited set of demographic information – survey data can now be used to give critical insight into the reasons why.

To do this type of analysis requires that the survey data can be linked on an individual basis to both an extended set of demographic data, and to behaviour data, either from HR or business systems.

In addition to the ability to handle large data sets, analysts are increasingly using non-table data structures to better answer questions. One alternative that offers great potential is so-called “graph databases”, where data is stored in a network. Such data structures allow us to ask very different questions.

With network data we can more easily answer questions not only about the individual employees but also the relationships between the employees. We see early promise in a network perspective where we look at contagion of engagement –how changes in employee engagement can spread across an organisation.

Machine learning
The final technology trend that is starting to disrupt the engagement survey world is the application of machine learning – the use of algorithms to search for and learn patterns in large quantities of data. Machine learning is also the basis of much so-called “predictive analytics”.

With employee survey data we are seeing great success with three applications of machine learning: using text analytics to make sense of vast amounts of open text answers; using patternspotting techniques to make probabilistic assessments of which populations are most likely to raise certain topics; and to use survey data to answer business questions.

Historically it has often been acknowledged that open text is the most valuable part of a survey, however it’s been very difficult and resourceconsuming to deal with it at scale. Text analytics can solve this problem and therefore provide new opportunities to capture this richer form of information.

Experience shows that with these techniques it is possible to analyse open text responses in almost any language, to categorise a comment against a continually evolving set of categories and score against things such as sentiment and Special supplement | An engaging place to work | New ways to run engagement surveys to do so in near real time. With this capability it is possible to radically rethink how and what kind of data is captured.

The second use of machine learning is to identify groups most likely to be discussing certain topics. For example, you might discover that those talking about a shortage of career development opportunities are much more likely to be women and/or Generation Y-ers who are in the upperperformance grades.

Finally, survey data is increasingly important to answer strategically important business questions that involve the workforce.

For example, you might link the survey data to sales data from a CRM system to try and optimise sales performance. In some cases it is possible to use existing survey data. In others surveys need to be used to collect new data.

The quote at the head of this article cites tools that are powerful and disruptive, that change the way we manage and lead our organisations. Used well, this kind of technology can be an organisational learning tool, deployed in a wide variety of applications far beyond the traditional engagement survey.

So with all these opportunities, where to get started? It is advisable to do four things:

  1. be open with your employees about how their data is being used and how the new approaches do not need to mean lower levels of confidentiality
  2. pilot some approaches with new use cases or in discrete populations
  3. consider these pilots as initially supplementing existing work until you are ready to use a more innovative approach
  4. consider how to prepare your organisation to respond quickly to ongoing, real-time feedback.

The overall goal of “employee listening is the same as the “voice of the customer”: provide insight that drives revenue opportunities while enriching the employee experience.

read podcast

Leave a Comment