The limitations of HR analytics in nonlinear systems by David
Verhagen
One of the many benefits of a summer vacation is the chance to
catch up on your reading list. This summer I read the excellent book ‘fooled by
randomness’ by the famous author Nassim Nicholas Taleb, who also wrote the book
‘The black swan’.
The book is mainly about the impact of chance and luck on
financial markets. He writes in a humorous way about human flaws in judging
cause and effect and especially the common mistake of confusing success caused
by personal skill with success caused by luck.
As a professional with an interest in HR analytics there were two
short chapters that particularly struck me as relevant for data driven HR. In
HR analytics we use the logic of math, data and statistics to make better
decisions about people and predict the behavior of people. But maybe the
reality of work, organizations and behavior is not always driven by logic.
Maybe chance plays a bigger role then we would like to think. If that is
(partly) true then we are trying to predict a non-logic reality through
logically constructed algorithms. It is like trying to understand Chinese with
only a French dictionary. Chances are your algorithms will be about as accurate
as flipping a coin.
Nonlinear systems
First the author talks about nonlinear systems. A nonlinear system
is a system where a change in output is not proportional to a change in input.
For instance one drop of water can make a bucket of water overflow while the
millions of drops before did not have the same impact. We have a tendency to
think that life progresses in a linear way. We think that if you study at
university for 4 years you will have learned twice as much as when you studied
for two years. In reality, your actual intellectual progress can stagnate for
months when studying a tough new subject, while on the other hand making a
giant leap during an inspiring class of just two hours.
Organizations often perform like nonlinear systems. For example, a
manager with ten years of experience may perform worse than a manager with just
one year of experience. You can have meetings that last for hours and not
accomplish a thing but also make a decision in five minutes that will
completely change the performance of an entire company for years to come. A
startup with little resources can disrupt a billion-dollar industry. Small
events can have a big impact.
Nonlinear systems often behave in a chaotic way and are hard to
predict with math. Seemingly small events can lead to major change and vice
versa. It is hard to predict which of all those small events will have impact
as these systems are often dynamic and can change faster than you can do
research.
Most startups will never be able to disrupt an industry. But you
just cannot ignore the small chance that one will because if it does your
company might be out of business.
Path dependent outcome
A second interesting topic is that of ‘path dependent outcome’.
This means that economic outcomes of events depend on earlier economic outputs.
In other words: history matters. A famous example is that of the ‘QWERTY’
keyboard. The QWERTY keyboard was designed for old school typewriters. The
format is not optimal for our current computers, it is slow and not logical.
Attempts to introduce more logical keyboards failed because people got used to
the old way of typing and did not want to change. Anybody in a large
organization will recognize these patterns. Large corporations often resist
change. Things might not be organized in an optimal way but it is hard to
change things when people are used to a certain way of working.
Organizations carry a huge amount of legacy with them. Decisions
made in the past influence the current situation. Using data analysis to
understand and improve processes that are influenced by these path dependent
outcomes can lead to misleading conclusions if insufficient historical data was
included to truly determine the influence of past decisions.
Add chance and luck to this mix and you get a system that is hard
to predict using statistics. Someone might get selected for a promotion because
he is friends with the right people, and not because of his skill. A sales rep
can land a million-dollar deal because she happens to know someone from high
school. We use data analysis to understand these differences in performance
while in reality there is a lot of chance involved.
HR analytics
So how can we deal with this in HR analytics?
First off, finding predictable patterns in nonlinear systems is
very complex but it is not impossible. There is a field in math called ‘chaos
theory’ that deals with this. Chaos theory is complex and it will take a lot of
time and resources to get the right results.
Although the effects explained by chaos theory play a role in
organizations they do not influence each and every HR process. It is possible
to find replicable patterns in a lot of common HR data and these will
definitely help in improving decision making. But chaos theory provides us with
a glimpse of the limitations of HR analytics. We might never be able to fully
understand all the aspects of work floor behavior.
Besides that, we should always be hesitant in accepting large
impact conclusions. Just like in science where a theory is only accepted if the
same conclusion is reached through multiple experiments. Another best practice
in science is running randomized controlled trials and experiments. This is
hard in HR analytics but it is not impossible. Sometimes you can offer a
similar training or other intervention at different times to different teams.
You can compare the team that received ‘treatment’ with the one that did not
yet receive training. If you carefully collect data you might get information
on actual causality.
In the end HR analytics is not science. We are trying to run a
business and make a profit. If your only goal is to make the smartest algorithm
in existence you are better off pursuing a career in science. In business there
is no time to conduct large scale research when faced with tough decisions
under a deadline. But it is good to be aware of the way emotions, chance and
chaos influence our judgment. Chance and history have a big impact on the
current state of your organization. But in the end a poorly informed decision
is probably still better than a decision based on no information at all.
Regards!
Librarian
Rizvi Institute of Management 
No comments:
Post a Comment