We need to talk about Norm!

So many times clients ask us about which is the right external norm or benchmark to use to compare themselves with in their employee survey. It is a good question. Norms are useful. Norms help put results in context. Norms tell you what is ‘normal’. For example, it is normal for items on ethics to score high (who wants to think they work for an unethical company) and items on pay to score low (who thinks they are ever paid enough?). So what norms should you use and what do you need to be aware of?

The first things to say is that external norms are composite. They are made up by averaging scores from a number of employees in other organisations. Every single organisation in that average is different, with a unique history, culture and leadership style all of which impact on the employee experience. The point here is that there is no such thing as the ‘perfect’ norm for your organisation, so they should only ever be used as a guide.

The next point is that you need to be aware of how the external norm was created. There are various ways of doing this. One way for providers to do this is to use their own clients’ data to create a norm database. To avoid big shocks of clients joining and leaving the database they are often smoothed by using a three year rolling average. This has several advantages: it is cheap to collect the data, as the projects are being run for clients anyway; it allows some segmentation by industry and country; and it can allow for specific comparisons (“how do we compare with these companies?”). It does however have some big drawbacks though: -

1. The size of the norm database is directly related to the number of clients a provider has
2. Industries or countries where the provider has little data will mean these norms are not robust
3. A few very large organisations will have a huge weighting in the results, skewing them, and a large impact if and when they are no longer a client
4. The smoothing effect of a rolling average will hide the impact of large changes in the population e.g. the negative effect of the financial crisis in 2008 was masked for 3 to 4 years in employee survey results, making individual organisations results seem much worse versus the norm

An alternative method is to use a panel interview where individuals from a number of organisations are interviewed on a regular basis on a number of pre-set topics. This allows for segmentation to be controlled ensuring that there is a representative sample of industries, countries and also demographics such as management level, age and tenure, giving the database a much more granularity. It also avoids the skewing of results towards one or two large organisations, and highlights significant changes in the employee landscape after each fieldwork.

Thirdly, carefully consider what norm grouping you are using. Most organisations will use industry or country norms or both. It is worth mentioning here that country norms tend to be much more robust than industry norms - by robust we mean there is less variation in the pattern of answers. This means that on average a technology company in the UK is going to have scores closer to a utilities company in the UK, then another technology company in either India or Japan. This is because there are differences in the way that we view work and view the questions, which are driven by strong and pervasive country characteristics.

Finally, organisations often focus so intently on external norms that they forget that they have a very robust, easily attainable database available at no extra cost. I’m referring to internal norms. These may either be in the form of previous years’ data (history or trend data), or using the overall score as a benchmark for sub-divisions within the organisation. It may well be that the organisation is a lot different from when you previously surveyed, but the chances are, given the unique history, culture and leadership, it will still be a lot closer than an average of a lot of other companies!

So, in conclusion, when it comes to norms always start with your own organisation first – what can you use as an internal benchmark? When looking at external benchmarks find out how the database has been constructed. Have a discussion with your provider about what norms are most relevant, and remember that industry norms are likely to have more variation than country ones. Whatever you choose, make sure your supplier has enough responses for it to be representative. And always consider having multiple norms (internal/country/industry) to allow you to get a better read on how positive (or not) your results are.