Making better decisions in organizations
calls for decision making as a competence in the assessment center
What we forget to measure in assessment centers (1)
In this blog an overview on decision making and how it is measured in assessment centers. what makes a good assessment center? In this blog I state that today’s assessment center is incomplete. Information in the assessment center is clear and too readily available. This is while in many work situations it is very different. Assessments of an applicant’s analytical skills, decision making ability and critical reasoning are therefore premature.
From the olden days
Ultimately, a large part of people’s work consists of making decisions. Weighing up the options, making a decision and taking action. In classic assessment centers, therefore, intelligence is often measured and something like “courage” is measured. Intelligence and courage are two abstract psychological characteristics or traits of an applicant. If the applicant displays those psychological traits, it is assumed that he or she will “solve problems” or is “a decision maker“.
For this reason, the intelligence test and the personality questionnaire are regular tools in many assessment centers. These are used to identify traits that are assumed to be necessary for a person to do a good job. It is an assumption and remains in the nature of abstract reasoning.
It is an underlying assumption that cannot always be sustained. In a meta-analysis Judge et al. conclude that the relationship between intelligence and leadership is low [1]. Namely, .21. It is also an assumption that is difficult to explain to the employer and the applicant. They must accept that a “5 on the intelligence test” and “a 6 on courage” are the right job requirements.
Modern assessment tools
are work samples/simulations
The last twenty years this has improved. In fact, there are more and more assessment exercises on the market that measure whether someone is doing the right things at work, in real life. Take SJTs as an example. In an SJT, the applicant is presented with X amount of situations on which he or she will make a decision. SJTs are often seen as measuring “a practical skill.” And that is making decisions and doing the right things in the real world (=job relevant behaviours). With in-basket exercises, the same thing is accomplished. The applicant is presented with a complex work situation in a mailbox. Then it is measured what decisions are made, and what the applicant achieves by doing so. In serious games, the situation is similar: the applicant is presented with a work environment that is dynamic and complex, and then it is measured what the applicant accomplishes.
The good thing about these modern real-life exercises is that they are much easier to explain to the applicant. The applicant sees the parallel between the exercise and the job for which is applied (=relevance). The exercises actually measure what someone achieves at work. It is about what a person does and what the outcome is. For the assessment consultant or psychologist, these exercises seem to be somewhat inconvenient. They make it less clear what “traits” the applicant has. You know that a job applicant is executing the right action. But in doing so, you don’t know if that is based on a trait such as intelligence, professional knowledge, or experience. The correlations between scores on SJTs or mailboxes with scores on intelligence are also low.
Forgotten dimensions of decision making
If you look closely at the assessment center, you will notice that the applicant is consistently presented with information that is very explicit. Every situation in tests and exercises in which the applicant works is clear and framed. Everything is well explained. All you have to do is read carefully and give answers. It is often actually worded that way: “Read the explanations/instructions/questions carefully first.“
In the workplace, this is very different. The information you receive and use in the real world is often not explicit. In that same reality, two things come into play.
Missing and contradictory information
In the work situation, information is incomplete and not everything is known. You don’t get all the information handed to you in a ready-made manner. Think of a police officer meeting someone on the street or a manager deciding on a promotion. It just depends on what information is already there and what other information there may be. Then it just really comes down to the person and what they are looking for.
Sometimes the opposite is true. There is an abundance of information. The information that comes at you in a work situation would make it almost impossible to do anything else. Think of the project manager or the bus driver. They have the ability to select and filter information in that overabundance [2]. If they do not apply filters in that overkill they become “trapped” in the information and the bus does not run and the project does not progress. And it’s about more than filters. People who master a trade often know what information they need.
Information is often conflicting and can contradict itself. Take the two stories of colleagues. They can tell very different stories about a situation. It is sometimes difficult to reconcile information from different sources. Similarly, think of the district nurse who hears a different story from Mrs. Johnson than from Mrs. Johnson’s daughter.bus driver
About the difference between data and information
data is not information
Ackoff’s (1988) DIKW model makes it clear that there are processes by which available data are transformed into information. Within DIKW, questioning is a way of processing and structuring data into information.
Gathering information and making decisions as a social process
People who make decisions have the choice of acting alone or involving others. The latter, consulting other people by asking questions, has all kinds of effects.
Asking a question has an emotional effect. Take the manager who asks questions of his own team members. Asking the question has an emotional effect. Take the manager who asks questions of his own team members. Because the manager asks questions of team members, those same team members get a different, a more positive image, of the manager. People who ask questions are perceived to be nicer [3]. In many of the situations, team members perceive it as interest and concern rather than weakness [4].
Consulting people often leads to more widespread support. But does it also lead to better decisions? We say, ” two people know more than one”. But is that true? The reports on this are mixed. Post et al. show in a research study that it mainly leads to better decisions in new situations and not so much in familiar ones [ 5]. In a recent article, Hamada et al. describe the nuances [6]. Having a group make the decisions collectively does not necessarily have the best outcome. But it is worthwhile to know many different opinions and to weigh those opinions. This is the so-called wisdom of crowds.
And then there’s another effect of consulting and engaging people: you even run the chance that people will start to reflect on situations or that they and even consult with each other more often. For example, in management. And you have the possibility that people will start to think about situations and even consult each other more often. When that happens, there is more than a sense of inclusion, dialogue within a group improves.
Collecting information
to make intelligent decisions
When you decide something at work or are going to do something, you have certain information. The question is, what additional information do you collect? And very importantly: Which information do you gather? Not all information is significant.
Gathering information is a skill. It certainly requires someone to ask questions. There is a lot of literature and research on questioning. Much of it is about students and scientists or about child development [7]. And in that research, asking questions is seen as a skill. That is, you can teach children, scholars, students and scientists to ask good questions [8]. In many vocational programs, students are taught to ask, “the right questions” and to listen.
In the past, the leader was the person who came up with the right answers;
in the future, the leader will be the person who comes up with the right questions.
Peter Drucker
For 50 years it has been argued that leadership is first and foremost about asking good questions [9] and then the answers will follow. When it comes to leadership, there is increasing empirical evidence in support of this. What we know is that the leader who asks questions has an effect on employees and teams. And it extends further. Some leaders exert influence and steer a team or group by asking the right questions. [10]. Furthermore, it is certain that leadership styles influence the extent to which a team uses knowledge, learns, and develops. [11]
Asking questions as a skill
in assessment centers
Information is not something that is there. Information is created by people [12], and that is seen as a skill. When it comes to training leaders and managers, gathering and organizing information should be number one: Asking questions, listening, weighing information. Great for people and outcomes. This speaks even more in favor of also determining in development centers how someone asks questions and gathers information.
The above is an opinion. I hope you, the reader, have an opinion as well. Would you like to respond with comments, additions, improvements or questions? Fine! You email me at info@cbpsy.com .
For this blog I have been looking for and reading a wealth of articles. Below you will find the most interesting.
[1] Judge, T. A., Colbert, A. E., & Ilies, R. (2004, June). Intelligence and leadership: A quantitative review and test of theoretical propositions. Journal of Applied Psychology. https://doi.org/10.1037/0021-9010.89.3.542
[2] Gigerenzer, G., & Brighton, H. (2009). Homo Heuristicus: Why Biased Minds Make Better Inferences. Topics in Cognitive Science, 1(1), 107–143.
https://doi.org/10.1111/j.1756-8765.2008.01006.x
[3] Huang, K., Yeomans, M., Brooks, A. W., Minson, J., & Gino, F. (2017). It doesn’t hurt to ask: Question-asking increases liking. Journal of Personality and Social Psychology, 113(3), 430–452. https://doi.org/10.1037/pspi0000097
[4] Cojuharenco, I., & Karelaia, N. (2020). When leaders ask questions: Can humility premiums buffer the effects of competence penalties? Organizational Behavior and Human Decision Processes, 156, 113–134. https://doi.org/10.1016/j.obhdp.2019.12.001
[5] Post, C., De Smet, H., Uitdewilligen, S., Schreurs, B., & Leysen, J. (2022). Participative or Directive Leadership Behaviors for Decision-Making in Crisis Management Teams? Small Group Research. https://doi.org/10.1177/10464964221087952
[6] Hamada, D., Nakayama, M., & Saiki, J. (2020). Wisdom of crowds and collective decision-making in a survival situation with complex information integration. Cognitive Research: Principles and Implications, 5(1). https://doi.org/10.1186/s41235-020-00248-z
[7] Ronfard, S., Zambrana, I. M., Hermansen, T. K., & Kelemen, D. (2018, September 1). Question-asking in childhood: A review of the literature and a framework for understanding its development. Developmental Review. Mosby Inc. https://doi.org/10.1016/j.dr.2018.05.002
[8] Farmer, L. S. J. (2007). What Is the Question? IFLA Journal, 33(1), 41–49. https://doi.org/10.1177/0340035207076408
[9] Baker, E. L., & Gilkey, R. (2020). Asking better questions-a core leadership skill. Journal of Public Health Management and Practice, 26(6), 632–633. https://doi.org/10.1097/PHH.0000000000001177
[10] Aritz, J., Walker, R., Cardon, P., & Li, Z. (2017). Discourse of Leadership: The Power of Questions in Organizational Decision Making. International Journal of Business Communication, 54(2), 161–181. https://doi.org/10.1177%2F2329488416687054
[11] Koeslag-Kreunen, M., Van den Bossche, P., Hoven, M., Van der Klink, M., & Gijselaers, W. (2018). When Leadership Powers Team Learning: A Meta-Analysis. Small Group Research, 49(4), 475–513. https://doi.org/10.1177/1046496418764824
[12] Glynn, P. D., Chiavacci, S. J., Rhodes, C. R., Helgeson, J. F., Shapiro, C. D., & Straub, C. L. (2022). Value of Information: Exploring Behavioral and Social Factors. Frontiers in Environmental Science, 10. https://doi.org/10.3389/fenvs.2022.805245
[13] Ackoff, R. L. (1989). From data to wisdom. Journal of Applied Systems Analysis, 16(1), 3–9. Retrieved from https://doi.
org/citeulike-article-id:6930744.
data is not information
Ackoff’s (1988) DIKW model makes it clear that there are processes by which available data are transformed into information. Within DIKW, questioning is a way of processing and structuring data into information.