Psychometrics or Pseudo-metrics?
Recent surveys indicate that 70% of employers now use psychometric testing as a recruitment aid. Whatever the actual figure, psychology has increasingly infiltrated industry over the last century. Personality, intelligence, individual and group behaviour, team structures, cognition and conflict resolution are all areas that have provided psychologists with a lucrative income. But is our money well spent or are psychometrics merely the height of speculation posing as deep learning?
Few commercial users of psychometric tests have any real grasp of the underlying theory. It is difficult to think of any other area in which buyers will part with cash so readily with so little knowledge of the product. The prevailing attitude throughout industry seems to be to trust psychologists to provide tools that can be used without having to engage with the theory. This naïve attitude practically invites exploitation. Unless buyers have basic knowledge of how the products work, how will they know if they are not working? How can they tell which products are based on fact and which on fantasy? Psychologists selling the tests are not necessarily the most reliable source of information. Buyers need independent guidance.
In reality, psychology is not as complex as many psychologists would have us believe. In fact, any psychologist claiming that psychology beyond the comprehension of the layman is trying to imply that they are cleverer than you – a dishonest and egocentric little manoeuvre psychologists themselves call ‘invalidation’. Provided we look at the basics systematically, psychology is no dark art.
Psychologists and indeed most scientists begin by taking their own specialist field as the area of study. Chemists specialise in chemistry, biologists in biology, psychologists in psychology and occupational psychologists in occupational psychology. Once these mental divisions are created, few consider the relevance of ideas from outside their own field. Professionals of every discipline become territorial, arrogant, blinkered and defensive.
To describe this lack of interchange between disciplines George Kelly uses the term “accumulated fragmentalism”. Accumulated fragmentalism is a fundamental error of the modern ‘scientific’ model. In order to represent external reality accurately, any mental model must take account of how all the fragments fit together. The system used here examines a chain of ideas without losing sight of how the links of the chain are connected. These are the links: -
• The External Environment – time, space and energy/matter.
• Sensation – the ways in which our five senses experience objects and events in the external environment.
• Memory – the ways in which we create, store and retrieve information about previous experiences.
• Perception – derived from comparison between sensation and memory or between one memory and another.
• Thinking – comparing, combining and dividing ideas.
• Intention – how we use information about past experiences to formulate and plan current or future activities and behaviour.
• Behaviour – our interactions with the external environment and with one another.
• Personality – regularly used behavioural patterns that determine our identity and how others perceive us.
• Feedback – We observe the effects of our actions and behaviour on the external environment and on other people. We respond to external stimuli and the behaviour of others. We have experiences and we experiment. The results of our observations and experiments are fed back via the senses and stored in memory so we can modify and improve our actions next time. We call it ‘learning’ or ‘education’.
We do not always make observations or conduct experiments in the real world. We can imagine events and anticipate outcomes adding this self-generated knowledge to memory. The difficulty we then face is differentiating between actual events and self-generated ones until, in the words of Immanuel Kant, “with the long practice of attention, we become skilled at separating them.”
The bulleted list above forms an endless loop like a cycle chain. The accumulated fragmentalist approach favoured by many scientists removes individual links from the chain and examines them in isolation. Examining links in isolation ‘dissects the frog’ sacrificing function to explore structure. Below, we examine each of the links in a little more detail whilst maintaining functional connections.
The External Environment
We begin with the external environment because without this, there would be nothing for our senses to detect. The external environment consists of the uneven distribution of energy/matter in space and time. Note that we consider energy and matter together. Many ‘real’ scientists, as some like to describe themselves, adopt a purely materialist view. Einstein’s greatest contribution was not to state that E=mc2 but to recognise that “energy and matter are simply different manifestations of same thing.”
Although uneven, the distribution of energy and matter is not entirely random since patterns are repeated. Science explores these patterns but those who study nature’s patterns in isolation can soon loose the overall picture. We must keep an eye on how the little pictures fit within the big one if we are to avoid accumulated fragentalism. This is what we call the ‘Andromedan’ approach.
Sensation
Our senses detect the presence of energy and matter in space at the present time only. Reference to any other time is impossible without memory. This is not to say that “time does not exist as such” or that “time is merely a figment of the human imagination” as many are seduced into believing. Time exists but we create our own temporal context using memory. The ability to create temporal context necessarily demands a pre-existing biological capacity to create, store and retrieve information about past events. This capacity is a function of our nervous system. Memory is not simply about the past. It necessarily includes plans for the future.
Memory
Without memory life would be meaningless. Every sensation must be compared with memory before we can attribute meaning. This page, for example, is meaningless to those with no knowledge stored in memory of the English language or the alphabet. We must necessarily compare current sensory information with memory in order to make ‘sense’ of our surroundings.
Sensations are experienced but because memories are created, memories are not always accurate. We create memories by combining information about space and time, space and matter or time and matter. (Note that we must always include energy in the term ‘matter’). This means we create three types of memory. Tulving calls them episodic, procedural and semantic.
• Episodic memory combines information about space and time and is sometimes called spatio-temporal memory.
• Procedural memory combines information about time and matter since it stores information on when (time) to do what (matter).
• Semantic memory combines information about matter and space storing information about why what (matter) goes where (space).
Time space and matter provide the when, where and what. Episodic, procedural and semantic memories provide the who, how and why. These three types of memory form the foundations of Freud’s three aspects of personality – id, superego and ego respectively.
Episodic memories and id are self-centred, personal and based on unity (one). They focus on feelings. Procedural memories and superego emphasise behaviour. They are group orientated and based on plurality (some). Semantic memories and ego focus on thinking, logic and reason. These emphasise universality, necessity and totality (all). Psychologists rejecting Freud’s view that there are three aspects to the personality as “merely Freud’s opinion” have failed to understand Freud. Memory and personality are inextricably linked. Some personality theories ignore this link and in so doing, they loose touch with reality and step into the realms of speculation.
Perception
Perception begins when we make comparisons between sensation and memory or between one memory and another. Perceptions are often inaccurate because information is stored in memory is often inaccurate. Unlike electronic memory, organic memory is prone to error. Information is sometimes recorded inaccurately or misinterpreted. It can be retrieved incorrectly, mixed up or wholly or partially lost. Furthermore, since it is our perceptions we add to memory rather than an accurate recording of events, any errors that already exist in memory are compounded and perpetuated.
Thinking
Comparing, combining and separating ideas are the three basic processes of thinking. All are impossible without memory. Understanding the memory/personality link is central to understanding all human activity. Some psychologists have not fully grasped this idea.
At the fundamental level, all thinking consists of comparing, combining and dividing of ideas. We can apply any of these three processes to any topic we choose. Analysis, be it chemical, political or psychological, is essentially about division. Creativity is about combining and/or dividing ideas in novel ways. Comparison always precedes combination or division. Human beings are capable of comparing, combining and separating ideas in more ways than any other creature on earth.
Intention
We compare possible courses of action combining and separating ideas stored in memory to create intentions. Whether conscious or unconscious, intention precedes all actions and behaviour. This must be the case because without intent, all our behaviour would be entirely random. The reverse of this argument is that any behaviour that is not random must follow some form of pattern or template that can only be held in memory. We use memory of past experiences to formulate and plan all intentional activities and behaviour.
Behaviour
Behaviour is the interaction between each individual and the external environment. It includes interaction with other people either individually or in groups. All behaviour is guided by intention since without intent no neural impulse is generated so nothing happens. All behaviour is motivated by what the individual wants and individual priorities change from time to time and from one situation to another.
The only accurate way of finding out other what people want is to ask them. Any attempt to explore motivation in isolation is therefore misguided. In the present paranoid climate, partially created by psychologists, people will often look for a hidden intention when asked what they want. Some may initially give the answer they think the questioner wants to hear rather than an honest answer. Game on!
In inventing countless theories of motivation, psychologists extend and perpetuate these mind games. They then charge you to untangle the web of distrust and confusion they are partially responsible for creating. All theories of motivation can be replaced with a single word – ask. It seems the ‘Emperor of Motivation’ has no clothes. He is not wearing robes made of a rich, complex beautiful fabric visible only to psychologists. He is as naked as the day he was born.
People behave in the ways they choose for their own reasons (conscious or unconscious), in accordance with their own feelings, moderated by the social pressures they feel at the time. The only hope of understanding what motivates any individual at the present time is to ask them. We can now discard all theories of motivation.
All we really need to know about psychology is this. The way we feel affects the way we think. What we think directs our behaviour. How we behave and the way others behave towards us affects the way we feel. This feeling – thinking – behaviour cycle was described by Galen almost 2000 years ago. It seems the smartest animal on Earth can sometimes be awfully slow to learn.
Personality
Personality is another area of psychology often taken out of context. It is an area where speculation soars to incredible heights. Whilst it is accepted that reading the mind of another person or accessing their thoughts and memories is impossible, 70% of employers seem prepared to believe that psychologists can reveal hidden personality traits and measure them. The behavioural consistent patterns we call personality is entirely dependent upon memory, which we accept is unreadable. Why then, are we prepared to believe that psychologists can ‘read’ personality?
Personality is based on qualitative differences whilst measurement is based on quantitative differences. What is the quantitative difference between an egg and a tree? Of course, it is plain for all to see that this question is meaningless. Why then are we prepared to believe that psychologists can use quantitative methods to measure qualitative personality differences with psychometric tests? Maybe the time has come to send these pseudo-scientists packing.
From United Kingdom, Huddersfield
Recent surveys indicate that 70% of employers now use psychometric testing as a recruitment aid. Whatever the actual figure, psychology has increasingly infiltrated industry over the last century. Personality, intelligence, individual and group behaviour, team structures, cognition and conflict resolution are all areas that have provided psychologists with a lucrative income. But is our money well spent or are psychometrics merely the height of speculation posing as deep learning?
Few commercial users of psychometric tests have any real grasp of the underlying theory. It is difficult to think of any other area in which buyers will part with cash so readily with so little knowledge of the product. The prevailing attitude throughout industry seems to be to trust psychologists to provide tools that can be used without having to engage with the theory. This naïve attitude practically invites exploitation. Unless buyers have basic knowledge of how the products work, how will they know if they are not working? How can they tell which products are based on fact and which on fantasy? Psychologists selling the tests are not necessarily the most reliable source of information. Buyers need independent guidance.
In reality, psychology is not as complex as many psychologists would have us believe. In fact, any psychologist claiming that psychology beyond the comprehension of the layman is trying to imply that they are cleverer than you – a dishonest and egocentric little manoeuvre psychologists themselves call ‘invalidation’. Provided we look at the basics systematically, psychology is no dark art.
Psychologists and indeed most scientists begin by taking their own specialist field as the area of study. Chemists specialise in chemistry, biologists in biology, psychologists in psychology and occupational psychologists in occupational psychology. Once these mental divisions are created, few consider the relevance of ideas from outside their own field. Professionals of every discipline become territorial, arrogant, blinkered and defensive.
To describe this lack of interchange between disciplines George Kelly uses the term “accumulated fragmentalism”. Accumulated fragmentalism is a fundamental error of the modern ‘scientific’ model. In order to represent external reality accurately, any mental model must take account of how all the fragments fit together. The system used here examines a chain of ideas without losing sight of how the links of the chain are connected. These are the links: -
• The External Environment – time, space and energy/matter.
• Sensation – the ways in which our five senses experience objects and events in the external environment.
• Memory – the ways in which we create, store and retrieve information about previous experiences.
• Perception – derived from comparison between sensation and memory or between one memory and another.
• Thinking – comparing, combining and dividing ideas.
• Intention – how we use information about past experiences to formulate and plan current or future activities and behaviour.
• Behaviour – our interactions with the external environment and with one another.
• Personality – regularly used behavioural patterns that determine our identity and how others perceive us.
• Feedback – We observe the effects of our actions and behaviour on the external environment and on other people. We respond to external stimuli and the behaviour of others. We have experiences and we experiment. The results of our observations and experiments are fed back via the senses and stored in memory so we can modify and improve our actions next time. We call it ‘learning’ or ‘education’.
We do not always make observations or conduct experiments in the real world. We can imagine events and anticipate outcomes adding this self-generated knowledge to memory. The difficulty we then face is differentiating between actual events and self-generated ones until, in the words of Immanuel Kant, “with the long practice of attention, we become skilled at separating them.”
The bulleted list above forms an endless loop like a cycle chain. The accumulated fragmentalist approach favoured by many scientists removes individual links from the chain and examines them in isolation. Examining links in isolation ‘dissects the frog’ sacrificing function to explore structure. Below, we examine each of the links in a little more detail whilst maintaining functional connections.
The External Environment
We begin with the external environment because without this, there would be nothing for our senses to detect. The external environment consists of the uneven distribution of energy/matter in space and time. Note that we consider energy and matter together. Many ‘real’ scientists, as some like to describe themselves, adopt a purely materialist view. Einstein’s greatest contribution was not to state that E=mc2 but to recognise that “energy and matter are simply different manifestations of same thing.”
Although uneven, the distribution of energy and matter is not entirely random since patterns are repeated. Science explores these patterns but those who study nature’s patterns in isolation can soon loose the overall picture. We must keep an eye on how the little pictures fit within the big one if we are to avoid accumulated fragentalism. This is what we call the ‘Andromedan’ approach.
Sensation
Our senses detect the presence of energy and matter in space at the present time only. Reference to any other time is impossible without memory. This is not to say that “time does not exist as such” or that “time is merely a figment of the human imagination” as many are seduced into believing. Time exists but we create our own temporal context using memory. The ability to create temporal context necessarily demands a pre-existing biological capacity to create, store and retrieve information about past events. This capacity is a function of our nervous system. Memory is not simply about the past. It necessarily includes plans for the future.
Memory
Without memory life would be meaningless. Every sensation must be compared with memory before we can attribute meaning. This page, for example, is meaningless to those with no knowledge stored in memory of the English language or the alphabet. We must necessarily compare current sensory information with memory in order to make ‘sense’ of our surroundings.
Sensations are experienced but because memories are created, memories are not always accurate. We create memories by combining information about space and time, space and matter or time and matter. (Note that we must always include energy in the term ‘matter’). This means we create three types of memory. Tulving calls them episodic, procedural and semantic.
• Episodic memory combines information about space and time and is sometimes called spatio-temporal memory.
• Procedural memory combines information about time and matter since it stores information on when (time) to do what (matter).
• Semantic memory combines information about matter and space storing information about why what (matter) goes where (space).
Time space and matter provide the when, where and what. Episodic, procedural and semantic memories provide the who, how and why. These three types of memory form the foundations of Freud’s three aspects of personality – id, superego and ego respectively.
Episodic memories and id are self-centred, personal and based on unity (one). They focus on feelings. Procedural memories and superego emphasise behaviour. They are group orientated and based on plurality (some). Semantic memories and ego focus on thinking, logic and reason. These emphasise universality, necessity and totality (all). Psychologists rejecting Freud’s view that there are three aspects to the personality as “merely Freud’s opinion” have failed to understand Freud. Memory and personality are inextricably linked. Some personality theories ignore this link and in so doing, they loose touch with reality and step into the realms of speculation.
Perception
Perception begins when we make comparisons between sensation and memory or between one memory and another. Perceptions are often inaccurate because information is stored in memory is often inaccurate. Unlike electronic memory, organic memory is prone to error. Information is sometimes recorded inaccurately or misinterpreted. It can be retrieved incorrectly, mixed up or wholly or partially lost. Furthermore, since it is our perceptions we add to memory rather than an accurate recording of events, any errors that already exist in memory are compounded and perpetuated.
Thinking
Comparing, combining and separating ideas are the three basic processes of thinking. All are impossible without memory. Understanding the memory/personality link is central to understanding all human activity. Some psychologists have not fully grasped this idea.
At the fundamental level, all thinking consists of comparing, combining and dividing of ideas. We can apply any of these three processes to any topic we choose. Analysis, be it chemical, political or psychological, is essentially about division. Creativity is about combining and/or dividing ideas in novel ways. Comparison always precedes combination or division. Human beings are capable of comparing, combining and separating ideas in more ways than any other creature on earth.
Intention
We compare possible courses of action combining and separating ideas stored in memory to create intentions. Whether conscious or unconscious, intention precedes all actions and behaviour. This must be the case because without intent, all our behaviour would be entirely random. The reverse of this argument is that any behaviour that is not random must follow some form of pattern or template that can only be held in memory. We use memory of past experiences to formulate and plan all intentional activities and behaviour.
Behaviour
Behaviour is the interaction between each individual and the external environment. It includes interaction with other people either individually or in groups. All behaviour is guided by intention since without intent no neural impulse is generated so nothing happens. All behaviour is motivated by what the individual wants and individual priorities change from time to time and from one situation to another.
The only accurate way of finding out other what people want is to ask them. Any attempt to explore motivation in isolation is therefore misguided. In the present paranoid climate, partially created by psychologists, people will often look for a hidden intention when asked what they want. Some may initially give the answer they think the questioner wants to hear rather than an honest answer. Game on!
In inventing countless theories of motivation, psychologists extend and perpetuate these mind games. They then charge you to untangle the web of distrust and confusion they are partially responsible for creating. All theories of motivation can be replaced with a single word – ask. It seems the ‘Emperor of Motivation’ has no clothes. He is not wearing robes made of a rich, complex beautiful fabric visible only to psychologists. He is as naked as the day he was born.
People behave in the ways they choose for their own reasons (conscious or unconscious), in accordance with their own feelings, moderated by the social pressures they feel at the time. The only hope of understanding what motivates any individual at the present time is to ask them. We can now discard all theories of motivation.
All we really need to know about psychology is this. The way we feel affects the way we think. What we think directs our behaviour. How we behave and the way others behave towards us affects the way we feel. This feeling – thinking – behaviour cycle was described by Galen almost 2000 years ago. It seems the smartest animal on Earth can sometimes be awfully slow to learn.
Personality
Personality is another area of psychology often taken out of context. It is an area where speculation soars to incredible heights. Whilst it is accepted that reading the mind of another person or accessing their thoughts and memories is impossible, 70% of employers seem prepared to believe that psychologists can reveal hidden personality traits and measure them. The behavioural consistent patterns we call personality is entirely dependent upon memory, which we accept is unreadable. Why then, are we prepared to believe that psychologists can ‘read’ personality?
Personality is based on qualitative differences whilst measurement is based on quantitative differences. What is the quantitative difference between an egg and a tree? Of course, it is plain for all to see that this question is meaningless. Why then are we prepared to believe that psychologists can use quantitative methods to measure qualitative personality differences with psychometric tests? Maybe the time has come to send these pseudo-scientists packing.
From United Kingdom, Huddersfield
Hello Mark 51:
>Recent surveys indicate that 70% of employers now use psychometric testing as a recruitment aid... But is our money well spent...<
All assessments are not created equal. Our 50,000+ clients report that using assessments yields them at least a 300% ROI and many report ROIs well above 1,000% so the answer is yes their money is well spent.
Bob Gately
From United States, Chelsea
45>Recent surveys indicate that 70% of employers now use psychometric testing as a recruitment aid... But is our money well spent...<
All assessments are not created equal. Our 50,000+ clients report that using assessments yields them at least a 300% ROI and many report ROIs well above 1,000% so the answer is yes their money is well spent.
Bob Gately
From United States, Chelsea
Hi,
I agree with Bob that Assessments are Imperative and effective. The important facts companies should take care to be aware about are:
1. What kind of Assessment tools are they in need of?
2. What is the reliability of the Assessment tool?
3. Can a division of the company be trained so as to handle the debriefing
of the assessment tool?this would ensure that the company has a clear
understanding of the Psychometric representation involved.
4. What is the history of the company providing the assessment?
5. Is the tool unbiased about gender/geographical location etc.Behavior based tools are generally this.
6. A Very Important Factor is to find out what exactly the tool measures. It is not enough if it identifies what is the category the person in concern falls into. Such kind is only for speculation and does not seem to have much relevance in Corporates. So the ideal tool would be one that can classify one for His/Her individual traits/Values/Behavior/Interests.This kind of tool will help an individual to understand their behavior pattern, how He/She views it and how others view the same.This can give a very good insight and actually help individuals and Corporates to work together to bring out the productive best in an individual and / or place him/her in their ideal job fit.This way i see many benefits. Most significant could be:
a. Individual fits his/her job implying even under stressful situations ownership will be experienced by the individual and so mostly we have found individuals being self starters in their ideal job.
b. Attrition will really be addressed.
c. This above point will mean fewer badmouthing about the company by the ones who have just left.
d. The company will be spending lesser on recruitment and can actually spend that money on welfare schemes.
Mainly stress levels on individuals, their respective families, & their Managers will actually reduce...
Happy Friendship Day 2 each of U
Sujatha Suresh
98408 54301
Head- Assessments & Training Div
Pravarra
From India, Bhilai
I agree with Bob that Assessments are Imperative and effective. The important facts companies should take care to be aware about are:
1. What kind of Assessment tools are they in need of?
2. What is the reliability of the Assessment tool?
3. Can a division of the company be trained so as to handle the debriefing
of the assessment tool?this would ensure that the company has a clear
understanding of the Psychometric representation involved.
4. What is the history of the company providing the assessment?
5. Is the tool unbiased about gender/geographical location etc.Behavior based tools are generally this.
6. A Very Important Factor is to find out what exactly the tool measures. It is not enough if it identifies what is the category the person in concern falls into. Such kind is only for speculation and does not seem to have much relevance in Corporates. So the ideal tool would be one that can classify one for His/Her individual traits/Values/Behavior/Interests.This kind of tool will help an individual to understand their behavior pattern, how He/She views it and how others view the same.This can give a very good insight and actually help individuals and Corporates to work together to bring out the productive best in an individual and / or place him/her in their ideal job fit.This way i see many benefits. Most significant could be:
a. Individual fits his/her job implying even under stressful situations ownership will be experienced by the individual and so mostly we have found individuals being self starters in their ideal job.
b. Attrition will really be addressed.
c. This above point will mean fewer badmouthing about the company by the ones who have just left.
d. The company will be spending lesser on recruitment and can actually spend that money on welfare schemes.
Mainly stress levels on individuals, their respective families, & their Managers will actually reduce...
Happy Friendship Day 2 each of U
Sujatha Suresh
98408 54301
Head- Assessments & Training Div
Pravarra
From India, Bhilai
Thank you both for your responses. It was not a short post but hopefully one that aroused your interest.
I agree entirely that assessment is imperative since selection is the name of the game. What I cannot accept is that nomothetic psychometric tests can ever be an effective tool for selecting an individual best suited to a particular post.
There are some ideographic tools like George Kelly’s ‘Repertory Grid’ or Carl Rogers’ ‘Q-Sort’ that allow us to explore an individual’s value systems and thought processes. These tools are occasionally used for selection for senior posts. They provide incredibly accurate individual profiles. However, they are very time consuming, expensive and cannot be used to compare candidates directly with one another.
My objection is to the nomothetic tools that typically use only 4 to 16 bipolar dimensions to categorise all individuals. My objection is based on the following reasoning.
There are three types of factors influence personality.
• The first are factors common to ALL people. We all have appetites or ‘drives’ for food, drink, warmth, shelter, social contact, love, sex etc. However, since these universal factors are common to all human beings, there is no point in measuring them.
• The second apply to SOME people. Cultural influences are common to a particular social group or social class. As Eric Berne observed, these vary from time to time and from culture to culture. These factors are the fodder of psychometric tests.
• The third are unique to the individual or to ONE person. These individual factors are of most interest to any prospective employer. Unfortunately, nomothetic psychometric testing reveals nothing about these unique individual factors.
Summarising the above, some personality factors are common to ALL people, some are shared by SOME people and some are unique to ONE person. There are no other options. This is why Freud was undoubtedly right to claim that there are three basic personality dimensions. He used the terms ‘ego’, ‘superego’ and ‘id’ respectively. Despite this, Freud is often dimissed as 'unscientific' by many psychologists.
Psychometric personality tests can only ever be used to explore ‘group sameness’ (the ideas and values shared by some people) rather than the ‘individual differences’, the dimensions of personality that make each of us unique.
I am aware that I am expressing a minority view. However, the majority view is often mistaken. Until the mid 1600s, the majority of ‘educated’ people in the Western world believed that the Sun orbits the Earth. They were wrong. Indian and Incan texts show that these civilisations knew of the heliocentric solar system many centuries before ‘western civilisation’ – something Ghandi thought would be a very good idea.
On the issue of reliability, Professor of Psychometrics at the University of Exeter, the late Paul Kline says “while intelligence is a relatively good predictor of occupational performance with correlations of around 0.6 (36%), personality tests were far more limited with correlations rarely exceeding 0.25 (6.25%). This lack of predictive power suggests that all is not well with psychometric measurement.”
The key point here is tah there is an enormous disparity between what psychologists claim psychometric tests are capable of telling us and what many HR professionals believe psychometric tests are capable of telling us. Kline’s figures imply that only one personality test in 16 is a reliable indicator of occupational performance. The question now is given 16 personality test results, which one is accurate?
To answer Sujatha’s points
1. Only ideographic tools explore individual differences. Psychometrics measure group sameness.
2. In nomothetic personality tests “correlation rarely exceeds 0.25”. They are unreliable fifteen times out of sixteen.
3. Since tests are so unreliable, why are we using them? Is ‘training’ simply ‘brainwashing’ into the cult of group sameness? Is this ethical? Innovators think differently from others. If we ever succeed in establishing ‘group sameness’, where will innovation come from?
4. If the product itself is unreliable, the hx of the company is irrelevant. In deciding WHAT to believe, it is important not to be seduced into deciding WHO to believe. Reputation is a poor guide in deciding WHAT to believe.
5. Most psychometric tests are based on largely Western, largely middle American, largely white, largely male, largely Christian, largely university student group norms (16PF or MMPI for example). Although modifications have been made, the bias still exists since it is built into the very foundations of test design. If we cannot ‘read’ the memory of a candidate we can never predict behaviour since all intentions underlying behaviour are created using information stored in memory. Test designers inevitably impose bias so no nomothetic test can ever be unbiased.
6. What the tool is measuring is the key point here. If we cannot ‘measure’ the difference between an egg and a tree, how can anyone ‘measure’ personality differences? Personality differences, like the differences between eggs and trees, are qualitative – not quantitative. I agree entirely with the rest of your comments– but this type of information can only be elicited using ideographic instruments – not psychometric tests.
Call me a sad old traditionalist but I would like to see psychometric tests consigned to the scrap heap so we can return to the old method of talking to one another. I have more faith in human beings than in tests.
Kind Regards
Mark 51
From United Kingdom, Huddersfield
I agree entirely that assessment is imperative since selection is the name of the game. What I cannot accept is that nomothetic psychometric tests can ever be an effective tool for selecting an individual best suited to a particular post.
There are some ideographic tools like George Kelly’s ‘Repertory Grid’ or Carl Rogers’ ‘Q-Sort’ that allow us to explore an individual’s value systems and thought processes. These tools are occasionally used for selection for senior posts. They provide incredibly accurate individual profiles. However, they are very time consuming, expensive and cannot be used to compare candidates directly with one another.
My objection is to the nomothetic tools that typically use only 4 to 16 bipolar dimensions to categorise all individuals. My objection is based on the following reasoning.
There are three types of factors influence personality.
• The first are factors common to ALL people. We all have appetites or ‘drives’ for food, drink, warmth, shelter, social contact, love, sex etc. However, since these universal factors are common to all human beings, there is no point in measuring them.
• The second apply to SOME people. Cultural influences are common to a particular social group or social class. As Eric Berne observed, these vary from time to time and from culture to culture. These factors are the fodder of psychometric tests.
• The third are unique to the individual or to ONE person. These individual factors are of most interest to any prospective employer. Unfortunately, nomothetic psychometric testing reveals nothing about these unique individual factors.
Summarising the above, some personality factors are common to ALL people, some are shared by SOME people and some are unique to ONE person. There are no other options. This is why Freud was undoubtedly right to claim that there are three basic personality dimensions. He used the terms ‘ego’, ‘superego’ and ‘id’ respectively. Despite this, Freud is often dimissed as 'unscientific' by many psychologists.
Psychometric personality tests can only ever be used to explore ‘group sameness’ (the ideas and values shared by some people) rather than the ‘individual differences’, the dimensions of personality that make each of us unique.
I am aware that I am expressing a minority view. However, the majority view is often mistaken. Until the mid 1600s, the majority of ‘educated’ people in the Western world believed that the Sun orbits the Earth. They were wrong. Indian and Incan texts show that these civilisations knew of the heliocentric solar system many centuries before ‘western civilisation’ – something Ghandi thought would be a very good idea.
On the issue of reliability, Professor of Psychometrics at the University of Exeter, the late Paul Kline says “while intelligence is a relatively good predictor of occupational performance with correlations of around 0.6 (36%), personality tests were far more limited with correlations rarely exceeding 0.25 (6.25%). This lack of predictive power suggests that all is not well with psychometric measurement.”
The key point here is tah there is an enormous disparity between what psychologists claim psychometric tests are capable of telling us and what many HR professionals believe psychometric tests are capable of telling us. Kline’s figures imply that only one personality test in 16 is a reliable indicator of occupational performance. The question now is given 16 personality test results, which one is accurate?
To answer Sujatha’s points
1. Only ideographic tools explore individual differences. Psychometrics measure group sameness.
2. In nomothetic personality tests “correlation rarely exceeds 0.25”. They are unreliable fifteen times out of sixteen.
3. Since tests are so unreliable, why are we using them? Is ‘training’ simply ‘brainwashing’ into the cult of group sameness? Is this ethical? Innovators think differently from others. If we ever succeed in establishing ‘group sameness’, where will innovation come from?
4. If the product itself is unreliable, the hx of the company is irrelevant. In deciding WHAT to believe, it is important not to be seduced into deciding WHO to believe. Reputation is a poor guide in deciding WHAT to believe.
5. Most psychometric tests are based on largely Western, largely middle American, largely white, largely male, largely Christian, largely university student group norms (16PF or MMPI for example). Although modifications have been made, the bias still exists since it is built into the very foundations of test design. If we cannot ‘read’ the memory of a candidate we can never predict behaviour since all intentions underlying behaviour are created using information stored in memory. Test designers inevitably impose bias so no nomothetic test can ever be unbiased.
6. What the tool is measuring is the key point here. If we cannot ‘measure’ the difference between an egg and a tree, how can anyone ‘measure’ personality differences? Personality differences, like the differences between eggs and trees, are qualitative – not quantitative. I agree entirely with the rest of your comments– but this type of information can only be elicited using ideographic instruments – not psychometric tests.
Call me a sad old traditionalist but I would like to see psychometric tests consigned to the scrap heap so we can return to the old method of talking to one another. I have more faith in human beings than in tests.
Kind Regards
Mark 51
From United Kingdom, Huddersfield
Hi,
CiteHR is one place we are able to express without feeling threatened. First of all I must thank Mark51 for having generated a very "in use" topic discussion. Guess the way discussions are going the topic may become more difficult for all to follow....
Many have many opinions about this topic, I'm sure. I have gained experience in the field ( and believe me I too had all the same doubts about metrics etc), It would indeed be welcome if we can discuss the same.
If we relied only on the human emotions. The pain is, many would agree, we are finding it increasingly difficult to understand and interact in positive ways. One finds more and more are conning some one somewhere all the time. Being a soft skills Trainer, I have had the experiences where:
1. individuals have literally cried that they are under extreme stress,
2. they do not know what is expected at times and
3. some find it difficult to understand why others cannot understand them!
4. Some say their families are affected by what they are going through...
We have had cases of youngsters and mid level employees hurt because of what is going on.... what happens is Individuals are experiencing lowered level of Self Confidence, increased levels of experiencing shame and being ridiculed and families getting broken up.Here proper metrics help one to identify their qualities and locate problem areas.Once bias is scientifically removed, individuals become more willing to accept change.Our whole generation is seeing and undergoing Change at all levels and in many ways positive and negative...
I agree, only Humans can help each other but to find this happen ideally all the time is proving more and more challenging. In fact Cite HR is the one and only site where in case someone expresses stress, we immediately rush to recognize that Humans matter and 10's of us rush to advice and give material and emotional support. like Abu Ben Adam poem...May this Tribe increase!
See the flip side of the day to day happenings,
HR employ a new person "A", since they find "A" good. Everything is fine till suddenly "A" chooses to leave without giving adequate reasons....the ones who employed "A' are questioned, suspected or concluded ineffective in identifying right individuals.
Recently (as I had mentioned in my "new comer" thread) I had the opportunity to attend an Assessment program. It was intriguing and also painful at the same time, when I heard the trouble HR go through when following the bell theory or bleed theory 5% of employees are made to leave irrespective of their capabilities. The individual in context may be good but rule says if individual falls in lowest 5% knock him or her...
When right metrics are used immense benefits are got by the one who Hires, The One who manages the hired and the one who is hired. Good to hear Bob that there is a good percentage of ROI...
From India, Bhilai
CiteHR is one place we are able to express without feeling threatened. First of all I must thank Mark51 for having generated a very "in use" topic discussion. Guess the way discussions are going the topic may become more difficult for all to follow....
Many have many opinions about this topic, I'm sure. I have gained experience in the field ( and believe me I too had all the same doubts about metrics etc), It would indeed be welcome if we can discuss the same.
If we relied only on the human emotions. The pain is, many would agree, we are finding it increasingly difficult to understand and interact in positive ways. One finds more and more are conning some one somewhere all the time. Being a soft skills Trainer, I have had the experiences where:
1. individuals have literally cried that they are under extreme stress,
2. they do not know what is expected at times and
3. some find it difficult to understand why others cannot understand them!
4. Some say their families are affected by what they are going through...
We have had cases of youngsters and mid level employees hurt because of what is going on.... what happens is Individuals are experiencing lowered level of Self Confidence, increased levels of experiencing shame and being ridiculed and families getting broken up.Here proper metrics help one to identify their qualities and locate problem areas.Once bias is scientifically removed, individuals become more willing to accept change.Our whole generation is seeing and undergoing Change at all levels and in many ways positive and negative...
I agree, only Humans can help each other but to find this happen ideally all the time is proving more and more challenging. In fact Cite HR is the one and only site where in case someone expresses stress, we immediately rush to recognize that Humans matter and 10's of us rush to advice and give material and emotional support. like Abu Ben Adam poem...May this Tribe increase!
See the flip side of the day to day happenings,
HR employ a new person "A", since they find "A" good. Everything is fine till suddenly "A" chooses to leave without giving adequate reasons....the ones who employed "A' are questioned, suspected or concluded ineffective in identifying right individuals.
Recently (as I had mentioned in my "new comer" thread) I had the opportunity to attend an Assessment program. It was intriguing and also painful at the same time, when I heard the trouble HR go through when following the bell theory or bleed theory 5% of employees are made to leave irrespective of their capabilities. The individual in context may be good but rule says if individual falls in lowest 5% knock him or her...
When right metrics are used immense benefits are got by the one who Hires, The One who manages the hired and the one who is hired. Good to hear Bob that there is a good percentage of ROI...
From India, Bhilai
Hi Sujatha,
The greatest strength of Cite HR is that this is possibly the only HR forum in the world where it is possible to express views openly without feeling threatened. Open debate is impossible in any other public forum so this policy of free and open debate must never be lost. No HR Journal, for example, will ever publish material opposing psychometrics simply because this opposition is seen as threatening the livelihood of many in the HR industry. I see rejecting psychometrics as an opportunity for the HR industry to progress to something better.
I am pleased to hear that others have had similar doubts about psychometrics but suspect that few have had the same doubts. Those with the same doubts never use these tests again.
I don’t necessarily agree that the arguments for and against psychometrics are too difficult for the majority on this site to follow. Cite HR obviously has an extremely knowledgeable base of subscribers who are perfectly capable of making up their own minds once given the arguments.
Psychometric theory is demonstrably false. In a nutshell, all psychometric tests are based on the notion that the personality traits, intelligence factors or capabilities of candidates are distributed throughout the population in accordance with the laws of normal distribution. In accepting this notion, we must also accept that the suitability of any pool of applicants for a particular post will also follow the same rules of normal distribution. The most suitable applicants fall on the extreme right of the curve – the least suitable fall on the extreme left.
All psychometric tests identify candidates who fall closest to the norm indicated by the peak in the centre of the curve. All psychometrics therefore deliberately select ‘average’ candidates by design. They always reject both the most suitable and the least suitable candidates in equal proportion. They may ensure standardisation but standardisation is achieved by selecting mediocre candidates at the expense of exceptional ones. Psychometric selection therefore perpetuates what J S Mill calls “the tyranny of the majority”.
There are many more arguments against psychometric testing but none is too difficult for most people on this forum to understand. For those interested, these arguments can be found in ‘Opposing Psychometrics’ on our website at patterson-powell.com
On ‘soft skills’, I am afraid I am unable to comment. I have a weakness. I never use a shovel if I can find a bulldozer and never use a bulldozer if I can find dynamite. My partner in crime, Colin Patterson, balances my hard edge.
The idea that HR professionals are blamed when people leave ‘without reason’ is disturbing. No individual can ever be held responsible for the actions of another and nobody leaves ‘without reason’ - although the individual concerned may think it prudent to keep their reasons for leaving private.
The hostility and intimidation HR professionals are subjected to in the aftermath of someone leaving may be the very reason the individual left the company refusing to give ‘adequate reasons’. Hostile working environments are extremely stressful. Stressed workers never perform to their full capability and will jump ship at the first opportunity. They jump not because of HR mistakes but because of bad management practices that deliberately create conflict and hostility, which brings me to your final point.
I am unfamiliar with ‘bell theory’ or ‘bleed theory’ but I think you are suggesting that 5% of employees are ‘culled’ (for want of a better word) at regular intervals regardless of their performance. This is Voltaire’s ‘English’ way as described in ‘Candide’. The English like to shoot an Admiral from time to time to encourage the others. It still happens – though the ‘shooting’ is usually metaphorically nowadays. It seems the English have infected India with this terrible ‘disease’.
If I understand bell theory and bleed theory correctly, this alarms me greatly. Are rules the master or the servant of man? I find this practice especially disturbing if psychometrics is used in selecting those to be ‘culled’ since this would ensure that the ‘best’ employees are culled along with the ‘worst’ (see above). Such draconian management practices produce a hostile climate of fear and stress. Even if psychometric tests are not used, this is both unethical and counter-productive. Deming’s red bead experiment will help to demonstrate why. I suggest all HR professionals and managers familiarise themselves with this extremely persuasive experiment.
Deliberate intimidation and unnecessarily producing a climate of fear and stress is never a good way to run a business since your best employees will inevitably jump ship to somewhere better at the first opportunity.
Kind regards and thanks for your interest in the topic. I too would welcome further discussion.
Mark 51
[/b]
From United Kingdom, Huddersfield
The greatest strength of Cite HR is that this is possibly the only HR forum in the world where it is possible to express views openly without feeling threatened. Open debate is impossible in any other public forum so this policy of free and open debate must never be lost. No HR Journal, for example, will ever publish material opposing psychometrics simply because this opposition is seen as threatening the livelihood of many in the HR industry. I see rejecting psychometrics as an opportunity for the HR industry to progress to something better.
I am pleased to hear that others have had similar doubts about psychometrics but suspect that few have had the same doubts. Those with the same doubts never use these tests again.
I don’t necessarily agree that the arguments for and against psychometrics are too difficult for the majority on this site to follow. Cite HR obviously has an extremely knowledgeable base of subscribers who are perfectly capable of making up their own minds once given the arguments.
Psychometric theory is demonstrably false. In a nutshell, all psychometric tests are based on the notion that the personality traits, intelligence factors or capabilities of candidates are distributed throughout the population in accordance with the laws of normal distribution. In accepting this notion, we must also accept that the suitability of any pool of applicants for a particular post will also follow the same rules of normal distribution. The most suitable applicants fall on the extreme right of the curve – the least suitable fall on the extreme left.
All psychometric tests identify candidates who fall closest to the norm indicated by the peak in the centre of the curve. All psychometrics therefore deliberately select ‘average’ candidates by design. They always reject both the most suitable and the least suitable candidates in equal proportion. They may ensure standardisation but standardisation is achieved by selecting mediocre candidates at the expense of exceptional ones. Psychometric selection therefore perpetuates what J S Mill calls “the tyranny of the majority”.
There are many more arguments against psychometric testing but none is too difficult for most people on this forum to understand. For those interested, these arguments can be found in ‘Opposing Psychometrics’ on our website at patterson-powell.com
On ‘soft skills’, I am afraid I am unable to comment. I have a weakness. I never use a shovel if I can find a bulldozer and never use a bulldozer if I can find dynamite. My partner in crime, Colin Patterson, balances my hard edge.
The idea that HR professionals are blamed when people leave ‘without reason’ is disturbing. No individual can ever be held responsible for the actions of another and nobody leaves ‘without reason’ - although the individual concerned may think it prudent to keep their reasons for leaving private.
The hostility and intimidation HR professionals are subjected to in the aftermath of someone leaving may be the very reason the individual left the company refusing to give ‘adequate reasons’. Hostile working environments are extremely stressful. Stressed workers never perform to their full capability and will jump ship at the first opportunity. They jump not because of HR mistakes but because of bad management practices that deliberately create conflict and hostility, which brings me to your final point.
I am unfamiliar with ‘bell theory’ or ‘bleed theory’ but I think you are suggesting that 5% of employees are ‘culled’ (for want of a better word) at regular intervals regardless of their performance. This is Voltaire’s ‘English’ way as described in ‘Candide’. The English like to shoot an Admiral from time to time to encourage the others. It still happens – though the ‘shooting’ is usually metaphorically nowadays. It seems the English have infected India with this terrible ‘disease’.
If I understand bell theory and bleed theory correctly, this alarms me greatly. Are rules the master or the servant of man? I find this practice especially disturbing if psychometrics is used in selecting those to be ‘culled’ since this would ensure that the ‘best’ employees are culled along with the ‘worst’ (see above). Such draconian management practices produce a hostile climate of fear and stress. Even if psychometric tests are not used, this is both unethical and counter-productive. Deming’s red bead experiment will help to demonstrate why. I suggest all HR professionals and managers familiarise themselves with this extremely persuasive experiment.
Deliberate intimidation and unnecessarily producing a climate of fear and stress is never a good way to run a business since your best employees will inevitably jump ship to somewhere better at the first opportunity.
Kind regards and thanks for your interest in the topic. I too would welcome further discussion.
Mark 51
[/b]
From United Kingdom, Huddersfield
Hello Mark 51:
My comments about assessments are limited to my experience and reading. Too many people presume that the MBTI, 16PF, MMPI and DISC types are the only psychometrics in use. Our clients do not use any of these assessments for selection. By the way, the MBTI is not to be used for selection according its publisher.
>The greatest strength of Cite HR is that this is possibly the only HR forum in the world where it is possible to express views openly without feeling threatened.<
A very good point.
>Open debate is impossible in any other public forum so this policy of free and open debate must never be lost.<
Our clients say the debate about assessments is over for them. Their employee turnover rate declines, new hire productivity increases, costs go down and the bottom line goes up.
An executive vice president of a large insurance company said that "Every time we pay to use your assessments our EBIT (Earnings Before Interest and Taxes) goes up by ten times the dollar amount. The money we pay is chump change."
>No HR Journal, for example, will ever publish material opposing psychometrics simply because this opposition is seen as threatening the livelihood of many in the HR industry.<
Perhaps the debate you want to have has been over for a long time?
If you are correct, there must be dozens of studies that support your views. Suggesting that everyone who disagrees with you is only concerned with their livelihood is silly and even if it were true, it wouldn't mean that you are correct.
>I see rejecting psychometrics as an opportunity for the HR industry to progress to something better.<
Yes, you have expressed a desire to go back to the good old days when hiring managers were free to hire and fire whomever they want regardless of their suitability to the job.
>I am pleased to hear that others have had similar doubts about psychometrics but suspect that few have had the same doubts. Those with the same doubts never use these tests again. <
Your presumption that psychometrics do not work is not supported by the data. Our clients achieve ROIs well above 300% and often well above 1000% so their data contradicts your assertion that psychometrics cannot work. And yes, their new hires are more successful when they hire for talent than when they did not hire for talent.
>I don’t necessarily agree that the arguments for and against psychometrics are too difficult for the majority on this site to follow.<
I agree, when managers hire for talent they quickly learn that psychometrics work and they work very well, see the two examples below.
---------------------
Design/Build firm:
---------------------
A design-build firm had a turnover problem with their project managers.
The hiring manager Dave agonized over firing project managers.
He wasn't happy about firing employees, the employees were not
happy when they were fired especially those that were hired away
from their former employers where they had been successfully
employed for years. Dave asked me for help.
Dave continued to screen his applicants as he always did but he
added a step before he made the job offer, he assessed all
qualified to be hired job applicants for their degree of
job suitability.
Within two years his turnover went from 33% per year to 0%.
Dave said that "everyone we hire exceeds our expectations."
Hiring for talent works.
--------------------------
Software Development:
---------------------------
Sonja, the HR Manager for a software development company,
hired for talent for over two years and then stopped for two years
because the two owners thought they could do a better job of
selecting good Technical Support Analysts (TSA) than she could.
TSAs go to the customers' workplace and identify the programming
errors and then reprogram the software.
The two owners felt too restricted by the talent selection process.
After two years of doing it themselves they went to Sonja's office
and told her "We s... at this, start using the your talent method
again, you are much more successful at hiring good TSAs than
we are." The owners could not duplicate her success rate, not
bad for just an HR Manager. Sonja took a risk when she first
tried the talent selection process.
---------------------------------------
Employers over rely on interviews and qualifications. The goal should be to hire competent people, not necessarily the most competent, who will become successful employees. The best I can tell a resume never actually does any work.
> Cite HR obviously has an extremely knowledgeable base of subscribers who are perfectly capable of making up their own minds once given the arguments.<
Subscribers should be convinced by data not just by arguments since data is available and arguments are endless.
>Psychometric theory is demonstrably false.<
Then please share with us all the court cases won by job applicants and employees who sued over the use of psychometric assessments. Do not include the MBTI since it is not a preemployment assessment. In the US some employers have misused the 16PF and MMPI.
>In a nutshell, all psychometric tests are based on the notion that the personality traits, intelligence factors or capabilities of candidates are distributed throughout the population in accordance with the laws of normal distribution.<
Doesn't a normal distribution describe rather proscribe?
>In accepting this notion, we must also accept that the suitability of any pool of applicants for a particular post will also follow the same rules of normal distribution. The most suitable applicants fall on the extreme right of the curve – the least suitable fall on the extreme left. <
Oh my, your comments are at odds with our process since "The most suitable applicants fall on the extreme right of the curve – the least suitable fall on the extreme left." does not apply to our process.
Sometimes the most suitable fall on the extreme right, sometimes on the extreme left and sometimes right in the middle and at other times slightly to the left of center and sometimes slightly to the right of center.
>All psychometric tests identify candidates who fall closest to the norm indicated by the peak in the centre of the curve. All psychometrics therefore deliberately select ‘average’ candidates by design.<
Not true for our process. Sometimes our clients hire from the left side, sometimes they hire from the right side, and sometimes they hire from the middle. It all depends on the demands of the job. Perhaps you don't know enough about all psychometric assessments?
>They always reject both the most suitable and the least suitable candidates in equal proportion.<
That is not true for our process which demonstrates that using the word always without knowing everything is a dangerous practice.
>They may ensure standardisation but standardisation is achieved by selecting mediocre candidates at the expense of exceptional ones.<
That is not true for our process since the whole point is to hire very successful employees at the expense of not hiring mediocre and poor employees. Our clients hire exceptional employees not the exceptional candidates. Sometimes they are the same but only about 20% of the time.
>Psychometric selection therefore perpetuates what J S Mill calls “the tyranny of the majority”. <
Hiring managers that use our method hire people who become successful employees which means they screen out about 80% of the qualified to be hired job applicants.
>There are many more arguments against psychometric testing but none so too difficult for most people on this forum to understand. For those interested, these arguments can be found in ‘Opposing Psychometrics’ on our website at patterson-powell.com<
Are you open to changing your mind or is your business wedded to the proposition that psychometric testing cannot work?
>It seems the English have infected India with this terrible ‘disease’.<
Didn't General Electric do the same thing?
>Kind regards and thanks for your interest in the topic. I too would welcome further discussion.<
Why, it seems to me you are not trying to help people understand assessments but rather convince them not to use assessments.
If you are incorrect, you are doing a disservice. If you are correct, please offer a list of US court cases that support your views. Courts in the US do not agree with you.
I'm always willing to learn but I am unwilling to trust an opinion unless I can also verify it. As far as I can tell you are offering your opinions unsubstantiated by data. I'll stay with the experience of the 50,000+ employers that use our assessments rather than accept your opinion at face value.
Employers do not need perfect information about applicants but they do need better information so they can reduce their chances of making bad hiring decisions. Our clients are adamant that they reduce their bad hires substantially. I hope you won't argue this point without first knowing what our clients actually do.
Bob
From United States, Chelsea
My comments about assessments are limited to my experience and reading. Too many people presume that the MBTI, 16PF, MMPI and DISC types are the only psychometrics in use. Our clients do not use any of these assessments for selection. By the way, the MBTI is not to be used for selection according its publisher.
>The greatest strength of Cite HR is that this is possibly the only HR forum in the world where it is possible to express views openly without feeling threatened.<
A very good point.
>Open debate is impossible in any other public forum so this policy of free and open debate must never be lost.<
Our clients say the debate about assessments is over for them. Their employee turnover rate declines, new hire productivity increases, costs go down and the bottom line goes up.
An executive vice president of a large insurance company said that "Every time we pay to use your assessments our EBIT (Earnings Before Interest and Taxes) goes up by ten times the dollar amount. The money we pay is chump change."
>No HR Journal, for example, will ever publish material opposing psychometrics simply because this opposition is seen as threatening the livelihood of many in the HR industry.<
Perhaps the debate you want to have has been over for a long time?
If you are correct, there must be dozens of studies that support your views. Suggesting that everyone who disagrees with you is only concerned with their livelihood is silly and even if it were true, it wouldn't mean that you are correct.
>I see rejecting psychometrics as an opportunity for the HR industry to progress to something better.<
Yes, you have expressed a desire to go back to the good old days when hiring managers were free to hire and fire whomever they want regardless of their suitability to the job.
>I am pleased to hear that others have had similar doubts about psychometrics but suspect that few have had the same doubts. Those with the same doubts never use these tests again. <
Your presumption that psychometrics do not work is not supported by the data. Our clients achieve ROIs well above 300% and often well above 1000% so their data contradicts your assertion that psychometrics cannot work. And yes, their new hires are more successful when they hire for talent than when they did not hire for talent.
>I don’t necessarily agree that the arguments for and against psychometrics are too difficult for the majority on this site to follow.<
I agree, when managers hire for talent they quickly learn that psychometrics work and they work very well, see the two examples below.
---------------------
Design/Build firm:
---------------------
A design-build firm had a turnover problem with their project managers.
The hiring manager Dave agonized over firing project managers.
He wasn't happy about firing employees, the employees were not
happy when they were fired especially those that were hired away
from their former employers where they had been successfully
employed for years. Dave asked me for help.
Dave continued to screen his applicants as he always did but he
added a step before he made the job offer, he assessed all
qualified to be hired job applicants for their degree of
job suitability.
Within two years his turnover went from 33% per year to 0%.
Dave said that "everyone we hire exceeds our expectations."
Hiring for talent works.
--------------------------
Software Development:
---------------------------
Sonja, the HR Manager for a software development company,
hired for talent for over two years and then stopped for two years
because the two owners thought they could do a better job of
selecting good Technical Support Analysts (TSA) than she could.
TSAs go to the customers' workplace and identify the programming
errors and then reprogram the software.
The two owners felt too restricted by the talent selection process.
After two years of doing it themselves they went to Sonja's office
and told her "We s... at this, start using the your talent method
again, you are much more successful at hiring good TSAs than
we are." The owners could not duplicate her success rate, not
bad for just an HR Manager. Sonja took a risk when she first
tried the talent selection process.
---------------------------------------
Employers over rely on interviews and qualifications. The goal should be to hire competent people, not necessarily the most competent, who will become successful employees. The best I can tell a resume never actually does any work.
> Cite HR obviously has an extremely knowledgeable base of subscribers who are perfectly capable of making up their own minds once given the arguments.<
Subscribers should be convinced by data not just by arguments since data is available and arguments are endless.
>Psychometric theory is demonstrably false.<
Then please share with us all the court cases won by job applicants and employees who sued over the use of psychometric assessments. Do not include the MBTI since it is not a preemployment assessment. In the US some employers have misused the 16PF and MMPI.
>In a nutshell, all psychometric tests are based on the notion that the personality traits, intelligence factors or capabilities of candidates are distributed throughout the population in accordance with the laws of normal distribution.<
Doesn't a normal distribution describe rather proscribe?
>In accepting this notion, we must also accept that the suitability of any pool of applicants for a particular post will also follow the same rules of normal distribution. The most suitable applicants fall on the extreme right of the curve – the least suitable fall on the extreme left. <
Oh my, your comments are at odds with our process since "The most suitable applicants fall on the extreme right of the curve – the least suitable fall on the extreme left." does not apply to our process.
Sometimes the most suitable fall on the extreme right, sometimes on the extreme left and sometimes right in the middle and at other times slightly to the left of center and sometimes slightly to the right of center.
>All psychometric tests identify candidates who fall closest to the norm indicated by the peak in the centre of the curve. All psychometrics therefore deliberately select ‘average’ candidates by design.<
Not true for our process. Sometimes our clients hire from the left side, sometimes they hire from the right side, and sometimes they hire from the middle. It all depends on the demands of the job. Perhaps you don't know enough about all psychometric assessments?
>They always reject both the most suitable and the least suitable candidates in equal proportion.<
That is not true for our process which demonstrates that using the word always without knowing everything is a dangerous practice.
>They may ensure standardisation but standardisation is achieved by selecting mediocre candidates at the expense of exceptional ones.<
That is not true for our process since the whole point is to hire very successful employees at the expense of not hiring mediocre and poor employees. Our clients hire exceptional employees not the exceptional candidates. Sometimes they are the same but only about 20% of the time.
>Psychometric selection therefore perpetuates what J S Mill calls “the tyranny of the majority”. <
Hiring managers that use our method hire people who become successful employees which means they screen out about 80% of the qualified to be hired job applicants.
>There are many more arguments against psychometric testing but none so too difficult for most people on this forum to understand. For those interested, these arguments can be found in ‘Opposing Psychometrics’ on our website at patterson-powell.com<
Are you open to changing your mind or is your business wedded to the proposition that psychometric testing cannot work?
>It seems the English have infected India with this terrible ‘disease’.<
Didn't General Electric do the same thing?
>Kind regards and thanks for your interest in the topic. I too would welcome further discussion.<
Why, it seems to me you are not trying to help people understand assessments but rather convince them not to use assessments.
If you are incorrect, you are doing a disservice. If you are correct, please offer a list of US court cases that support your views. Courts in the US do not agree with you.
I'm always willing to learn but I am unwilling to trust an opinion unless I can also verify it. As far as I can tell you are offering your opinions unsubstantiated by data. I'll stay with the experience of the 50,000+ employers that use our assessments rather than accept your opinion at face value.
Employers do not need perfect information about applicants but they do need better information so they can reduce their chances of making bad hiring decisions. Our clients are adamant that they reduce their bad hires substantially. I hope you won't argue this point without first knowing what our clients actually do.
Bob
From United States, Chelsea
"Ever since they started to think of themselves as scientists, measurement has been at the forefront of psychologists' minds. One of the overriding problems in the field of personality has concerned the question "what to measure?". [Bannister D. Fransella F. (1985) -'Inquiring Man' p 48.]
So - the first question we must answer when using a psychometric test is 'precisely what is this test measuring?'. If unable to answer the question, we shouldn't be using the test. If we say personality, we should be able to define personality in such a way that all would agree. Good luck if you want to give it a try. In 1935 Allport listed over 50 definitions of personality. There are now many more. If you are running intelligence tests you should be able to define 'intelligence' in such a way that all would agree. How can anyone measure precisely what they cannot define precisely? Operational definitions, by definition, are not precise but speculative - yet psychometric tests are claimed to be objective tests rather than merely speculative ones.
It seems possible that the word 'psychometrics' is sometimes used in different ways by psychologists and some HR professionals. This needs clarification. Psychologists differentiate between 'psychometric' tests and other methods of psychological assessment. Those in HR who have a looser interpretation have failed to understand the psychologists. This misunderstanding underlies the unrealistic beliefs of many in HR.
Any test presenting the same questions to a number of candidates and comparing answers with those of a standardised norm group is a 'psychometric', 'normative' or 'nomothetic' test. These tests apply the same set bipolar constructs to all participants (e.g. introvert v extravert ¨C stable v neurotic). Even Raymond Cattell, designer of the 16PF test, realised that his initial 16 bipolar constructs were insufficient. By the time he died in 1997 he had increased the number to 35. This was effectively an admission that despite earlier claims, factor analysis had failed to produce a set of universal personality dimensions on which all could be assessed. In reality we all use thousands of different bipolar constructs so there is no 'universal set'.
In contrast, any method of assessment exploring participants in terms of their own bipolar constructs is an 'ipsitive' or 'ideographic' test. Characteristically there are no set questions but there are specified procedures. Psychologists do not include this type of test under the banner of 'psychometrics'. These instruments are very accurate but very time consuming and very expensive indeed. In HR, these tests are sometimes included as 'psychometrics'. Applying the principle of charity I wll assume this may be the point of where Bob and I diverge. If using ideographic methods, then his customers are quite rightly satisfied. If not, they are mistaken.
George Kelly argues thus. If we wish to understand another individual, we must allow them to express themselves in terms of their own constructs. We cannot understand another individual by imposing constructs of our own making or the constructs of those who devise a particular test. On p 61 Bannister & Fransella say, "Psychologists failure derives from our habit of asking people to answer OUR questions rather than noting the nature of the questions THEY are asking."
I wish to make it clear that like Bannister and Fransella, I hold psychologists largely responsible for the misconceptions of modern occupational test methods. However, if we have been deceived, we have allowed ourselves to be deceived. HR professionals must undoubtedly shoulder a share of the blame. Like carpenters, all HR professionals should endeavour to familiarise themselves with the tools of their trade. Those who do not can expect to loose a finger or two.
My own personal position is relevant. I have been forced into retirement by deafness. My computer is my main link with the outside world. I am not in business so I have no business interests to protect. I have no potential customers to impress with evidence of ROI from former satisfied customers. As demonstrated by Piaget, I know that evidence only ever provides provisional knowledge. I know that the number of believers supporting an idea is NEVER a measure of truth or validity so ROI figures do not persuade me one way or the other. Nor should they persuade anyone else. I know that only old ideas have many supporters and that all new ideas begin with only one supporter. If we rely on numbers of supporters to establish the validity of ideas, nothing would ever change.
I can afford to be independent since I owe no allegiance to any employer or to any organization. From now until I die I am free of all shackles. I can afford the luxury of time to read and to think. I openly admit my preference is for ideographic methods of psychological assessment and make no secret of the fact that I see 'psychometrics' (as defined above) as "voodoo science" - a term used in a case against the Australian railway company Railcorp. (Google 'railcorp voodoo science' for the benefit of those who need evidence.)
Despite the wishes of others, I certainly do not see the debate about psychometrics as "being over long ago" any more than I see the debates about racism, sexism or the origin of the universe to be over. The debate will continue for as long as psychometric tests continue to be used. Psychometrics is just a fad. The fad for phrenology lasted for 170 years but there are still some believers. Provided their beliefs cause no harm to others, residual phrenologists are welcome to continue in their beliefs.
The psychometric fad will probably last for a similar period of time. However, psychometrics can cause considerable harm to others so believers must expect continuing opposition. The debate will certainly not be resolved in my lifetime so I have nothing personal to gain. Those making a living from psychometrics have much to gain and much to loose so are unlikely to be impartial. I am absolutely convinced that there are no worse or more unjust ways of hiring or firing people. Psychometrics is based on conformity. Innovators do not conform and industry needs innovators.
ROI is an entirely subjective rather than an objective measure of performance. All objective scientific evaluations conclude that no psychometric personality test has ever achieved a correlation between predicted and actual occupational performance greater than 0.3. Professor Paul Kline was a professor of psychometrics at the University of Exeter until his death in 1999. In a magazine article written shortly before his death Kline says -
"A huge analysis of the predictive power of tests in occupational psychology by Schmidt & Hunter (1998) indicated that while intelligence was a relatively good predictor of occupational performance, with correlations of about 0.6, personality tests were of far more limited value, with correlations rarely exceeding 0.25. This lack of predictive power again suggests that all is not well with psychometric measurement."
For those unfamiliar with correlations, 0.6 = 36% and 0.25 = 6.25%.
This statement from a professor of psychometrics is clearly at odds with the wildly extravagant and exaggerated claims of many in the HR industry.
Many disbelieve me when I quote Kline and I am frequently accused of making this quote up so for the doubters, the quote comes from an article in 'Psychology Review' 1999, Vol 6; Number 2 pp 6-10. It can be ordered online from the British Library and is identified by the following
ISSN 1354-1129
Unique Item Number RN068843637
Shelfmark 6946.536350.
We now each have a decision to make. Which evidence about the reliability of psychometric testing is more credible? The evidence offered by those selling psychometric tests or the evidence from the pen of a professor of psychometrics.
From my own impartial position the decision is easy. For those deeply involved in HR, the decision is one that will cause considerable anxiety and cognitive dissonance for many years to come. I make no apology for any anxiety caused. As Bacon, Descartes and Kant all suggest, "It is prudent to check the foundations before building our magnificent edifices." Psychometrics may appear a magnificent edifice to some but it has very weak foundations.
Good luck to all of you with this thorny problem. Me? I'm off to the allotment to do some therapeutic digging.
Kind regards
Mark 51
From United Kingdom, Huddersfield
So - the first question we must answer when using a psychometric test is 'precisely what is this test measuring?'. If unable to answer the question, we shouldn't be using the test. If we say personality, we should be able to define personality in such a way that all would agree. Good luck if you want to give it a try. In 1935 Allport listed over 50 definitions of personality. There are now many more. If you are running intelligence tests you should be able to define 'intelligence' in such a way that all would agree. How can anyone measure precisely what they cannot define precisely? Operational definitions, by definition, are not precise but speculative - yet psychometric tests are claimed to be objective tests rather than merely speculative ones.
It seems possible that the word 'psychometrics' is sometimes used in different ways by psychologists and some HR professionals. This needs clarification. Psychologists differentiate between 'psychometric' tests and other methods of psychological assessment. Those in HR who have a looser interpretation have failed to understand the psychologists. This misunderstanding underlies the unrealistic beliefs of many in HR.
Any test presenting the same questions to a number of candidates and comparing answers with those of a standardised norm group is a 'psychometric', 'normative' or 'nomothetic' test. These tests apply the same set bipolar constructs to all participants (e.g. introvert v extravert ¨C stable v neurotic). Even Raymond Cattell, designer of the 16PF test, realised that his initial 16 bipolar constructs were insufficient. By the time he died in 1997 he had increased the number to 35. This was effectively an admission that despite earlier claims, factor analysis had failed to produce a set of universal personality dimensions on which all could be assessed. In reality we all use thousands of different bipolar constructs so there is no 'universal set'.
In contrast, any method of assessment exploring participants in terms of their own bipolar constructs is an 'ipsitive' or 'ideographic' test. Characteristically there are no set questions but there are specified procedures. Psychologists do not include this type of test under the banner of 'psychometrics'. These instruments are very accurate but very time consuming and very expensive indeed. In HR, these tests are sometimes included as 'psychometrics'. Applying the principle of charity I wll assume this may be the point of where Bob and I diverge. If using ideographic methods, then his customers are quite rightly satisfied. If not, they are mistaken.
George Kelly argues thus. If we wish to understand another individual, we must allow them to express themselves in terms of their own constructs. We cannot understand another individual by imposing constructs of our own making or the constructs of those who devise a particular test. On p 61 Bannister & Fransella say, "Psychologists failure derives from our habit of asking people to answer OUR questions rather than noting the nature of the questions THEY are asking."
I wish to make it clear that like Bannister and Fransella, I hold psychologists largely responsible for the misconceptions of modern occupational test methods. However, if we have been deceived, we have allowed ourselves to be deceived. HR professionals must undoubtedly shoulder a share of the blame. Like carpenters, all HR professionals should endeavour to familiarise themselves with the tools of their trade. Those who do not can expect to loose a finger or two.
My own personal position is relevant. I have been forced into retirement by deafness. My computer is my main link with the outside world. I am not in business so I have no business interests to protect. I have no potential customers to impress with evidence of ROI from former satisfied customers. As demonstrated by Piaget, I know that evidence only ever provides provisional knowledge. I know that the number of believers supporting an idea is NEVER a measure of truth or validity so ROI figures do not persuade me one way or the other. Nor should they persuade anyone else. I know that only old ideas have many supporters and that all new ideas begin with only one supporter. If we rely on numbers of supporters to establish the validity of ideas, nothing would ever change.
I can afford to be independent since I owe no allegiance to any employer or to any organization. From now until I die I am free of all shackles. I can afford the luxury of time to read and to think. I openly admit my preference is for ideographic methods of psychological assessment and make no secret of the fact that I see 'psychometrics' (as defined above) as "voodoo science" - a term used in a case against the Australian railway company Railcorp. (Google 'railcorp voodoo science' for the benefit of those who need evidence.)
Despite the wishes of others, I certainly do not see the debate about psychometrics as "being over long ago" any more than I see the debates about racism, sexism or the origin of the universe to be over. The debate will continue for as long as psychometric tests continue to be used. Psychometrics is just a fad. The fad for phrenology lasted for 170 years but there are still some believers. Provided their beliefs cause no harm to others, residual phrenologists are welcome to continue in their beliefs.
The psychometric fad will probably last for a similar period of time. However, psychometrics can cause considerable harm to others so believers must expect continuing opposition. The debate will certainly not be resolved in my lifetime so I have nothing personal to gain. Those making a living from psychometrics have much to gain and much to loose so are unlikely to be impartial. I am absolutely convinced that there are no worse or more unjust ways of hiring or firing people. Psychometrics is based on conformity. Innovators do not conform and industry needs innovators.
ROI is an entirely subjective rather than an objective measure of performance. All objective scientific evaluations conclude that no psychometric personality test has ever achieved a correlation between predicted and actual occupational performance greater than 0.3. Professor Paul Kline was a professor of psychometrics at the University of Exeter until his death in 1999. In a magazine article written shortly before his death Kline says -
"A huge analysis of the predictive power of tests in occupational psychology by Schmidt & Hunter (1998) indicated that while intelligence was a relatively good predictor of occupational performance, with correlations of about 0.6, personality tests were of far more limited value, with correlations rarely exceeding 0.25. This lack of predictive power again suggests that all is not well with psychometric measurement."
For those unfamiliar with correlations, 0.6 = 36% and 0.25 = 6.25%.
This statement from a professor of psychometrics is clearly at odds with the wildly extravagant and exaggerated claims of many in the HR industry.
Many disbelieve me when I quote Kline and I am frequently accused of making this quote up so for the doubters, the quote comes from an article in 'Psychology Review' 1999, Vol 6; Number 2 pp 6-10. It can be ordered online from the British Library and is identified by the following
ISSN 1354-1129
Unique Item Number RN068843637
Shelfmark 6946.536350.
We now each have a decision to make. Which evidence about the reliability of psychometric testing is more credible? The evidence offered by those selling psychometric tests or the evidence from the pen of a professor of psychometrics.
From my own impartial position the decision is easy. For those deeply involved in HR, the decision is one that will cause considerable anxiety and cognitive dissonance for many years to come. I make no apology for any anxiety caused. As Bacon, Descartes and Kant all suggest, "It is prudent to check the foundations before building our magnificent edifices." Psychometrics may appear a magnificent edifice to some but it has very weak foundations.
Good luck to all of you with this thorny problem. Me? I'm off to the allotment to do some therapeutic digging.
Kind regards
Mark 51
From United Kingdom, Huddersfield
PART 1:
Hello Mark 51:
>… we should be able to define personality in such a way that all would agree.
…. If you are running intelligence tests you should be able to define intelligence in such a way that all would agree.<
Why? Do scientists really wait until every skeptic agrees? Of course not, so why would you make such a statement? It seems to me you are making the argument that as long you disagree the rest of us should accept your opinions as fact which is not a convincing argument at all.
>How can anyone measure precisely what they cannot define precisely?<
Where is it written that employers must have precise information about job applicants and employees? Do you write this stuff hoping that the reader will think precise information is a requirement for doing business? It isn’t.
Perhaps I missed that lecture in business school but I never missed a class so I suggest to the reader precise information is not a requirement of doing business or managing employees.
>Operational definitions, by definition, are not precise but speculative - yet psychometric tests are claimed to be objective tests rather than merely speculative ones.<
Again, employers do not need nor should they wait for, require, demand or use precise information about people as a basis for job selection because precise information does not predict job success.
>Psychologists differentiate between ¡®psychometric¡¯ tests and other methods of psychological assessment. Those in HR who have a looser interpretation have failed to understand the psychologists. This misunderstanding underlies the unrealistic beliefs of many in HR.<
Case 1: For you to be correct about psychometric assessments, all those in HR who do a better job of selecting successful employees by using assessments must be incorrect.
Case 2: If you are wrong about psychometric assessments, then only you need to be wrong.
I opt for Case 2.
>…Even Raymond Cattell, designer of the 16PF test, realised that his initial 16 bipolar constructs were insufficient.<
It is insufficient for what purpose?
>In reality we all use thousands of different bipolar constructs so there is no ¡®universal set¡¯.<
Are you saying that users of well designed psychometric assessments cannot improve their employee selection process? If your answer is yes, you are factually incorrect.
>In contrast, any method of assessment exploring participants in terms of their own bipolar constructs is an ipsitive or ideographic test.<
The 50,000+ employers that use our assessments for selection do not use an ipsative assessment.
>… This point of may be where Bob and I diverge. If using ideographic methods, then his customers are quite rightly satisfied.<
We do not provide ipsative assessments for selection. Our clients are satisfied because their retention rates go up, productivity increases and employee problems decline.
>George Kelly argues thus. If we wish to understand another individual, we must allow them to express themselves in terms of their own constructs.<
Where is it required that hiring managers “must allow them to express themselves in terms of their own constructs”? There is no such requirement. Managers want to hire successful employees who do their jobs for a long time and who enjoy their jobs as well.
>We cannot understand another individual by imposing constructs of our own making or the constructs of those who devise a particular test.<
You miss the point of an employee selection process. It is not about understanding individuals but rather identifying which qualified to be hired job applicants have the best chance of becoming a successful employee.
>On p 61 Bannister & Fransella say, ¡°Psychologists failure derives from our habit of asking people to answer OUR questions rather than noting the nature of the questions THEY are asking.¡± <
Failure at doing what? Was Bannister writing about hiring employees or counseling people who wanted or needed help?
>I wish to make it clear that like Bannister and Fransella, I hold psychologists largely responsible for the misconceptions of modern occupational test methods.<
It seems to me you are doing it as well.
continued in PART 2
From United States, Chelsea
Hello Mark 51:
>… we should be able to define personality in such a way that all would agree.
…. If you are running intelligence tests you should be able to define intelligence in such a way that all would agree.<
Why? Do scientists really wait until every skeptic agrees? Of course not, so why would you make such a statement? It seems to me you are making the argument that as long you disagree the rest of us should accept your opinions as fact which is not a convincing argument at all.
>How can anyone measure precisely what they cannot define precisely?<
Where is it written that employers must have precise information about job applicants and employees? Do you write this stuff hoping that the reader will think precise information is a requirement for doing business? It isn’t.
Perhaps I missed that lecture in business school but I never missed a class so I suggest to the reader precise information is not a requirement of doing business or managing employees.
>Operational definitions, by definition, are not precise but speculative - yet psychometric tests are claimed to be objective tests rather than merely speculative ones.<
Again, employers do not need nor should they wait for, require, demand or use precise information about people as a basis for job selection because precise information does not predict job success.
>Psychologists differentiate between ¡®psychometric¡¯ tests and other methods of psychological assessment. Those in HR who have a looser interpretation have failed to understand the psychologists. This misunderstanding underlies the unrealistic beliefs of many in HR.<
Case 1: For you to be correct about psychometric assessments, all those in HR who do a better job of selecting successful employees by using assessments must be incorrect.
Case 2: If you are wrong about psychometric assessments, then only you need to be wrong.
I opt for Case 2.
>…Even Raymond Cattell, designer of the 16PF test, realised that his initial 16 bipolar constructs were insufficient.<
It is insufficient for what purpose?
>In reality we all use thousands of different bipolar constructs so there is no ¡®universal set¡¯.<
Are you saying that users of well designed psychometric assessments cannot improve their employee selection process? If your answer is yes, you are factually incorrect.
>In contrast, any method of assessment exploring participants in terms of their own bipolar constructs is an ipsitive or ideographic test.<
The 50,000+ employers that use our assessments for selection do not use an ipsative assessment.
>… This point of may be where Bob and I diverge. If using ideographic methods, then his customers are quite rightly satisfied.<
We do not provide ipsative assessments for selection. Our clients are satisfied because their retention rates go up, productivity increases and employee problems decline.
>George Kelly argues thus. If we wish to understand another individual, we must allow them to express themselves in terms of their own constructs.<
Where is it required that hiring managers “must allow them to express themselves in terms of their own constructs”? There is no such requirement. Managers want to hire successful employees who do their jobs for a long time and who enjoy their jobs as well.
>We cannot understand another individual by imposing constructs of our own making or the constructs of those who devise a particular test.<
You miss the point of an employee selection process. It is not about understanding individuals but rather identifying which qualified to be hired job applicants have the best chance of becoming a successful employee.
>On p 61 Bannister & Fransella say, ¡°Psychologists failure derives from our habit of asking people to answer OUR questions rather than noting the nature of the questions THEY are asking.¡± <
Failure at doing what? Was Bannister writing about hiring employees or counseling people who wanted or needed help?
>I wish to make it clear that like Bannister and Fransella, I hold psychologists largely responsible for the misconceptions of modern occupational test methods.<
It seems to me you are doing it as well.
continued in PART 2
From United States, Chelsea
PART 2:
>However, if we have been deceived, we have allowed ourselves to be deceived. HR professionals must undoubtedly shoulder a share of the blame. Like carpenters, all HR professionals should endeavour to familiarise themselves with the tools of their trade.<
HR professionals do not need a Ph.D. to use such tools.
>… I am not in business so I have no business interests to protect.<
We all have biases, agendas and egos.
>I have no potential customers to impress with evidence of ROI from former satisfied customers.<
You cannot help HR professionals without such knowledge
>As demonstrated by Piaget, I know that evidence only ever provides provisional knowledge. <
But you are convinced beyond doubt that you are correct. Do you mean that to be correct we must first believe and then ignore evidence to the contrary? If provisional knowledge improves the selection process, then provisional knowledge is a competitive advantage.
>I know that the number of believers supporting an idea is NEVER a measure of truth or validity so ROI figures do not persuade me one way or the other.<
Are you persuadable?
> I know that only old ideas have many supporters and that all new ideas begin with only one supporter.<
Are you offering new ideas or rehashing old ideas?
>If we rely on numbers of supporters to establish the validity of ideas, nothing would ever change.<
Didn’t you suggest that we need to all agree before we can use an assessment? I’m glad to see you are recanting that silly idea.
>I can afford to be independent since I owe no allegiance to any employer or to any organization. From now until I die I am free of all shackles. I can afford the luxury of time to read and to think.<
That does not mean your judgment is not flawed.
>I openly admit my preference is for ideographic methods of psychological assessment and make no secret of the fact that I see psychometrics¯ (as defined above) as ¡°voodoo science¡± ¨C a term used in a case against the Australian railway company Railcorp. (Google ¡®railcorp voodoo science¡¯ for the benefit of those who need evidence.)
Who used the term? I agree, many employers use the wrong assessments and/or misuse the right assessments but that does not mean all assessment should be tossed out. I agree with Kearney who said “employers should only use job-relevant measures that are proven to predict on-the-job success and performance, such as occupational testing.”
The best I can tell from reading some Googled articles is that the RailCorp did not use appropriate assessments. The following is from http://www.abc.net.au/pm/content/2004/s1231109.htm
-----------------------------------------------
The Secretary of the New South Wales State Labor Council, John Robertson.
“JOHN ROBERTSON: There are three tests that go under the heading of psychometric testing. One's called the RAAT (Rules Acquisition Aptitude Test) Test, and it involves crossing out letters on a page, as instructed, using speed and accuracy. The next part of that test is a personality test, where you answer questions about what was most like about you in these questions and what was least like you. The other test in this area is called the SCAT (Safe Concentration And Attention Test) Test, where you've got to pick out patterns and lines in these particular objects, and it's effectively like a giant Gameboy or computer game. Now, we just think that these things are absurd.
---------------------------
I agree with John Robertson.
>Despite the wishes of others, I certainly do not see the debate about psychometrics as ¡®being over long ago¡¯ any more than I see the debates about racism, sexism or the origin of the universe to be over.<
I wrote that for our 50,000+ clients say the debate is over them. For the RailCorp they made a huge error in judgment. If you suggest that all assessments have the same problems as the ones used by RailCorp you do not understand all assessments.
> The debate will continue for as long as psychometric tests continue to be used.<
As long as employers use the wrong tests you are correct.
>Psychometrics is just a fad.<
Fad or not some assessments work very well.
> The fad for phrenology lasted for 170 years but there are still some believers. Provided their beliefs cause no harm to others, residual phrenologists are welcome to continue in their beliefs.<
You dismiss all assessments even though all assessments are not the same? That seems to me to be the hallmark of a true believer, evidence be damned.
>The psychometric fad will probably last for a similar period of time. However, psychometrics can cause considerable harm to others so believers must expect continuing opposition.<
You are in fact a true believer in your position since there is much evidence that disproves your assertion that assessments cannot work.
> The debate will certainly not be resolved in my lifetime so I have nothing personal to gain.<
For true believers there is always something to gain, acceptance of their position comes to mind.
>Those making a living from psychometrics have much to gain and much to loose so are unlikely to be impartial.<
That is fallacious argument and readers should reject it as such. That type of argument is often used by true believers who do not have evidence to support their claims. Using that type of argument we should reject the advice of surgeons because they too have something to gain from operating and let us not forget lawyers, engineers, accountants, etc.
>I absolutely convinced that there are no worse or more unjust ways of hiring or firing people. Psychometrics is based on conformity. Innovators do not conform and industry needs innovators. <
Oh my, you are “absolutely convinced” so therefore you are right? I think not.
>ROI is an entirely subjective rather than an objective measure of performance.<
Except in sales, see pilot study results below.
Hiring for Talent Pilot Study Results
==========================
The Company:
===========
- National and international offices
- Insurance and financial services
- 15,000+ employees worldwide
- 8 Divisions
Before Hiring for Talent:
=================
- Company-wide turnover was 34% the preceding year
- Company-wide sales averaged 101% of sales quota
- An average US Division was selected for a pilot study
After Hiring for Talent for 6 months:
==========================
- Employee turnover reduced from 34% to 19%.
- New salespeople who had a Benchmark Suitability of...
85% or above averaged 916% of sales quota
84% or below averaged 187% of sales quota
end of pilot study
When they calculated the ROI they started hiring for talent in the other divisions.
continued in PART 3
From United States, Chelsea
>However, if we have been deceived, we have allowed ourselves to be deceived. HR professionals must undoubtedly shoulder a share of the blame. Like carpenters, all HR professionals should endeavour to familiarise themselves with the tools of their trade.<
HR professionals do not need a Ph.D. to use such tools.
>… I am not in business so I have no business interests to protect.<
We all have biases, agendas and egos.
>I have no potential customers to impress with evidence of ROI from former satisfied customers.<
You cannot help HR professionals without such knowledge
>As demonstrated by Piaget, I know that evidence only ever provides provisional knowledge. <
But you are convinced beyond doubt that you are correct. Do you mean that to be correct we must first believe and then ignore evidence to the contrary? If provisional knowledge improves the selection process, then provisional knowledge is a competitive advantage.
>I know that the number of believers supporting an idea is NEVER a measure of truth or validity so ROI figures do not persuade me one way or the other.<
Are you persuadable?
> I know that only old ideas have many supporters and that all new ideas begin with only one supporter.<
Are you offering new ideas or rehashing old ideas?
>If we rely on numbers of supporters to establish the validity of ideas, nothing would ever change.<
Didn’t you suggest that we need to all agree before we can use an assessment? I’m glad to see you are recanting that silly idea.
>I can afford to be independent since I owe no allegiance to any employer or to any organization. From now until I die I am free of all shackles. I can afford the luxury of time to read and to think.<
That does not mean your judgment is not flawed.
>I openly admit my preference is for ideographic methods of psychological assessment and make no secret of the fact that I see psychometrics¯ (as defined above) as ¡°voodoo science¡± ¨C a term used in a case against the Australian railway company Railcorp. (Google ¡®railcorp voodoo science¡¯ for the benefit of those who need evidence.)
Who used the term? I agree, many employers use the wrong assessments and/or misuse the right assessments but that does not mean all assessment should be tossed out. I agree with Kearney who said “employers should only use job-relevant measures that are proven to predict on-the-job success and performance, such as occupational testing.”
The best I can tell from reading some Googled articles is that the RailCorp did not use appropriate assessments. The following is from http://www.abc.net.au/pm/content/2004/s1231109.htm
-----------------------------------------------
The Secretary of the New South Wales State Labor Council, John Robertson.
“JOHN ROBERTSON: There are three tests that go under the heading of psychometric testing. One's called the RAAT (Rules Acquisition Aptitude Test) Test, and it involves crossing out letters on a page, as instructed, using speed and accuracy. The next part of that test is a personality test, where you answer questions about what was most like about you in these questions and what was least like you. The other test in this area is called the SCAT (Safe Concentration And Attention Test) Test, where you've got to pick out patterns and lines in these particular objects, and it's effectively like a giant Gameboy or computer game. Now, we just think that these things are absurd.
---------------------------
I agree with John Robertson.
>Despite the wishes of others, I certainly do not see the debate about psychometrics as ¡®being over long ago¡¯ any more than I see the debates about racism, sexism or the origin of the universe to be over.<
I wrote that for our 50,000+ clients say the debate is over them. For the RailCorp they made a huge error in judgment. If you suggest that all assessments have the same problems as the ones used by RailCorp you do not understand all assessments.
> The debate will continue for as long as psychometric tests continue to be used.<
As long as employers use the wrong tests you are correct.
>Psychometrics is just a fad.<
Fad or not some assessments work very well.
> The fad for phrenology lasted for 170 years but there are still some believers. Provided their beliefs cause no harm to others, residual phrenologists are welcome to continue in their beliefs.<
You dismiss all assessments even though all assessments are not the same? That seems to me to be the hallmark of a true believer, evidence be damned.
>The psychometric fad will probably last for a similar period of time. However, psychometrics can cause considerable harm to others so believers must expect continuing opposition.<
You are in fact a true believer in your position since there is much evidence that disproves your assertion that assessments cannot work.
> The debate will certainly not be resolved in my lifetime so I have nothing personal to gain.<
For true believers there is always something to gain, acceptance of their position comes to mind.
>Those making a living from psychometrics have much to gain and much to loose so are unlikely to be impartial.<
That is fallacious argument and readers should reject it as such. That type of argument is often used by true believers who do not have evidence to support their claims. Using that type of argument we should reject the advice of surgeons because they too have something to gain from operating and let us not forget lawyers, engineers, accountants, etc.
>I absolutely convinced that there are no worse or more unjust ways of hiring or firing people. Psychometrics is based on conformity. Innovators do not conform and industry needs innovators. <
Oh my, you are “absolutely convinced” so therefore you are right? I think not.
>ROI is an entirely subjective rather than an objective measure of performance.<
Except in sales, see pilot study results below.
Hiring for Talent Pilot Study Results
==========================
The Company:
===========
- National and international offices
- Insurance and financial services
- 15,000+ employees worldwide
- 8 Divisions
Before Hiring for Talent:
=================
- Company-wide turnover was 34% the preceding year
- Company-wide sales averaged 101% of sales quota
- An average US Division was selected for a pilot study
After Hiring for Talent for 6 months:
==========================
- Employee turnover reduced from 34% to 19%.
- New salespeople who had a Benchmark Suitability of...
85% or above averaged 916% of sales quota
84% or below averaged 187% of sales quota
end of pilot study
When they calculated the ROI they started hiring for talent in the other divisions.
continued in PART 3
From United States, Chelsea
Community Support and Knowledge-base on business, career and organisational prospects and issues - Register and Log In to CiteHR and post your query, download formats and be part of a fostered community of professionals.
Psychometrics - Tests & HR.pdf
Ch 31 psychometrics.p pt