Experts and expertise

Preamble

In 2012 I explored the nature of expertise in an article that was mainly focused on expert witness expertise. A deeper exploration was triggered today when I discovered a statement made by the Care Quality Commission. Scroll down for more. This article will delve deep into the issues of experts and expertise, and prove that the CQC is misguided.

Expertise refers to a high level of knowledge or skill in a particular field or area. Experts are often recognised for their ability to understand complex information, solve problems, make informed decisions, and perform tasks more effectively and efficiently than non-experts.

Expertise is usually acquired through extensive study, training, and experience. But none of the latter is guaranteed to create expertise. The process of developing expertise is far more complex.

Expertise can be demonstrated in many different fields, including but not limited to science, art, sports, business, and education. It can be measured in various ways, such as through certifications, degrees, professional experience, or the recognition of peers. An expert at trading stock markets or forex will be judged largely by profitability-performance over long periods. Hedge funds and the pensions markets depend on these people.

In medical circles we can also expect performance of experts and expertise to be much better than ordinary people who may think they experts, through sheer experience alone.  It is important to note that expertise is not static. It requires continuous learning and adaptation, especially in fields where knowledge and best practices evolve rapidly, such as technology and medicine. Experts are expected to have expertise. It would be ridiculous to claim to be an expert without expertise.

Controversy

The Care Quality Commission believes that there can be ‘Experts by experience’. See here where the CQC states, “Experts by Experience are people who have recent personal experience (within the last eight years) of using or caring for someone who uses health, mental health and/or social care services that we regulate.” The CQC-effect is often seen when persons who suffered depression or trauma sets themselves up to share knowledge, write books, or start a following through blogs or YouTube channels. Whilst such persons may not claim to be experts, they are often quoted as if experts.

Experience is an essential part of creating expertise but on its own it does not create expertise. Why?

Here are a few reasons why experience alone will not be enough:

1. Quality of experience: Not all experience is equal. Simply spending time in a field doesn’t guarantee expertise. The quality of the experience matters. This includes the variety of situations encountered, the challenges overcome, and the lessons learned.
2. Reflection and learning: Experience is most valuable when it is paired with reflection and learning. Experts don’t just accumulate experiences; they learn from them. They analyse their successes and failures, draw lessons from their experiences, and apply these lessons to future situations.
3. Up-to-date knowledge: In many fields, knowledge evolves rapidly. What was considered best practice a few years ago might be outdated today. Therefore, experts need to continuously update their knowledge, which often involves learning from sources other than their own experience, such as research, training, and the experiences of others.
4. Skill development: Expertise involves not just knowledge but also skills. These skills often need to be deliberately practiced and developed over time. This can involve training, education, and feedback from others, not just personal experience.
5. Cognitive biases: Relying solely on personal experience can lead to cognitive biases. For example, people might overgeneralise from their own experiences, or they might be influenced by confirmation bias, where they pay more attention to experiences that confirm their existing beliefs. It is important to balance personal experience with other forms of evidence and perspectives.

In conclusion, while experience is a vital component of expertise, it is not the only factor. Expertise typically involves a combination of high-quality experience, continuous learning, up-to-date knowledge, skill development, and the ability to overcome cognitive biases.

The CQC – which is largely perceived as a politically driven organisation (by me) –  is wrong to hold much value about ‘experts by experience’. Furthermore, expertise has a has to have objective components and parameters – else anybody can claim to be an expert.

Defining features of expertise

1. Deep knowledge: Experts have a comprehensive understanding of their field. This includes both theoretical knowledge and practical, hands-on experience. They understand not only the basics but also the nuances and complexities of their subject matter.
2. Experience: Expertise often comes with years of experience. This experience allows experts to recognise patterns, apply knowledge in practical contexts, and make informed decisions quickly and accurately.
3. Skill: Experts have developed high-level skills through practice and application of their knowledge. They can perform tasks related to their field more effectively and efficiently than non-experts.
4. Problem-solving ability: Experts are adept at solving problems in their field. They can identify problems, generate solutions, and implement these solutions effectively. They can also often anticipate potential issues and mitigate them before they become problems.
5. Recognition: Experts are usually recognised by others in their field. This can be through awards, publications, positions of authority, or simply the respect and acknowledgment of their peers.
6. Continuous learning: Experts understand that their field is constantly evolving and that they need to continue learning to maintain their expertise. They stay up-to-date with the latest research, trends, and developments in their field.
7. Teaching and communication skills: While not always the case, many experts are also good teachers. They can explain complex concepts in ways that non-experts can understand, and they can effectively communicate their ideas and knowledge to others.
8. Intuition: Over time, experts develop an intuitive understanding of their field. They can often make good decisions or come up with creative solutions based on this intuition, even when they can’t fully articulate the process they used to arrive at these decisions or solutions.

Intuition in expertise refers to the ability of experts to make quick, effective decisions or judgments that seem to bypass the typical analytical thought process. This is often based on their extensive experience and deep knowledge in a particular field. Over time, experts accumulate a vast amount of knowledge and experience, which they can draw upon almost unconsciously. This allows them to recognise patterns, make connections, and come up with solutions more quickly than non-experts. They may not be able to fully articulate how they arrived at a decision or solution, but their intuition often proves to be accurate.

Here are a few key points about intuition in expertise:

1. Pattern recognition: Experts often have an intuitive ability to recognise patterns or see connections that others might miss. This is based on their deep knowledge and extensive experience. For example, a seasoned chess player might intuitively recognise a winning strategy upon glancing at the board, or a skilled doctor might have a gut feeling about a diagnosis based on a complex combination of symptoms.
2. Automaticity: Experts can often perform certain tasks automatically, without conscious thought, freeing up cognitive resources for more complex problems. This automaticity is a result of extensive practice and experience.
3. Adaptive decision-making: Experts can make quick, adaptive decisions in complex and uncertain situations. They can draw on their intuition to make judgments when there is no clear right or wrong answer.
4. Tacit knowledge: Experts often possess tacit knowledge, which is knowledge that’s difficult to transfer to another person by means of writing it down or verbalising it. This type of knowledge is often gained from personal experience and intuition.

While intuition can be powerful, it is not infallible. Experts can still make mistakes, especially when faced with unfamiliar situations. Furthermore, intuition can sometimes be influenced by cognitive biases. Therefore, it is important for experts to also use analytical thinking and evidence-based methods.

Developmental pathway

The development of expertise is a complex process that involves the integration and refinement of experience, knowledge, and skill. Here is a general outline of how these elements come together:

1. Acquisition of fundamental knowledge and skills: The journey towards expertise begins with the acquisition of fundamental knowledge and skills in a particular domain. This involves learning the basic concepts, principles, and techniques that are essential to the field.
2. Practical experience: As individuals gain more experience in the field, they begin to apply their knowledge and skills in practical situations. This allows them to see how the concepts and techniques they’ve learned work in the real world. It also exposes them to a variety of situations and challenges, helping them to broaden their understanding and improve their skills.
3. Reflection and learning from experience: Experience alone is not enough to develop expertise. It is also important to reflect on experiences, learn from them, and use them to refine knowledge and skills. This might involve analysing successes and failures, seeking feedback, and identifying areas for improvement.
4. Advanced learning and skill development: As individuals progress in their field, they continue to learn and develop their skills. This might involve pursuing advanced education or training, learning from experts, and staying up-to-date with the latest research and developments in the field.
5. Deliberate practice: Deliberate practice involves focused, goal-oriented practice with immediate feedback. It is designed to improve performance and is essential for refining skills and developing expertise.
6. Development of intuition and tacit knowledge: Over time, as individuals gain more experience and practice their skills, they begin to develop intuition and tacit knowledge. These are important aspects of expertise that allow experts to make quick, effective decisions and perform tasks more efficiently.
7. Continuous learning and adaptation: Even after achieving a level of expertise, it is important to continue learning and adapting. This involves staying up-to-date with changes in the field, continuously refining skills, and learning from new experiences.

Skill

A skill is the ability to perform an action with determined results often within a given amount of time, energy, or both. Skills can be divided into domain-general and domain-specific skills. For example, in the domain of work, some general skills would include time management, teamwork and leadership, self-motivation and others, whereas domain-specific skills would be useful only for a certain job.

Skills are acquired and developed through a process that involves learning, practice, and experience. Here’s a general outline of how this process works:

1. Learning: The first step in acquiring a skill is to learn about it. This can involve studying the theory behind the skill, observing others who are proficient in the skill, and getting instruction from a teacher or mentor. For example, if you wanted to learn how to play the piano, you might start by learning about music theory and watching skilled pianists.
2. Practice: Once you have a basic understanding of the skill, the next step is to practice it. This involves repeatedly performing the skill in order to improve your ability. Practice is often most effective when it is deliberate, meaning that you focus on improving specific aspects of the skill and get feedback on your performance. For example, a budding pianist might practice scales or specific pieces of music, and get feedback from a teacher or from listening to their own performance.
3. Experience: Over time, as you gain experience with the skill, you’ll likely start to improve. You’ll become more efficient, make fewer mistakes, and be able to perform the skill in more complex or challenging situations. Experience also helps you to develop intuition and tacit knowledge, which are important aspects of skill that can be difficult to teach directly.
4. Reflection and adjustment: An important part of skill acquisition is reflecting on your performance and making adjustments based on what you’ve learned. This might involve changing your approach, focusing on different aspects of the skill, or seeking out new learning resources or methods of practice.
5. Consistency and persistence: Skills often take time to develop, and progress may be slow at times. Consistency and persistence are key. Even when progress seems slow, continued practice and experience can lead to gradual improvements.
6. Learning from mistakes: Mistakes are a natural part of skill acquisition. They provide valuable feedback and learning opportunities. Rather than being discouraged by mistakes, it is important to learn from them and use them to improve your skill.

Remember, the process of acquiring a skill can vary depending on the nature of the skill and the individual’s learning style, prior experience, and other factors. But in general, it involves a combination of learning, practice, experience, reflection, and adjustment.

Models for skill acquisition

1. Fitts and Posner’s Three-Stage Model: This model proposes that skill acquisition occurs in three stages: the cognitive stage, the associative stage, and the autonomous stage.
   • In the cognitive stage, the learner develops an understanding of the skill and what is required to perform it. This stage involves a lot of trial and error and conscious thought about the skill.
   • In the associative stage, the learner begins to refine the skill, reducing errors and becoming more consistent. The skill becomes more integrated and fluid.
   • In the autonomous stage, the skill becomes automatic and can be performed with little conscious thought. The learner can also perform the skill while doing other tasks.
2. Dreyfus and Dreyfus’ Five-Stage Model: This model proposes that skill acquisition occurs in five stages: novice, advanced beginner, competent, proficient, and expert.
   • Novices are at the initial stage of skill acquisition, where they follow rules and guidelines to perform the skill.
   • Advanced beginners have gained some experience and can start to make decisions based on context.
   • Competent performers can manage complexity through conscious planning and decision making.
   • Proficient performers have developed intuition and can make decisions based on a holistic view of the situation.
   • Experts have a deep understanding of the skill and can make intuitive decisions based on a deep, tacit understanding of the situation.
3. Anders Ericsson’s Deliberate Practice Model: This model emphasises the importance of deliberate practice in skill acquisition. Deliberate practice involves focused, goal-oriented practice with immediate feedback. According to Ericsson, it is not just the amount of practice that matters, but the quality of the practice. Deliberate practice is designed to improve performance, emphasises repetition of skill-related tasks, involves continuous feedback, and is highly demanding mentally.

These models provide a framework for understanding how skills are acquired and developed. They highlight the importance of practice, experience, feedback, and the progression from conscious, deliberate performance of a skill to more automatic and intuitive performance.

Time

The “10,000-hour rule” is a popular idea that suggests that it takes roughly 10,000 hours of practice to achieve mastery in a field. This concept was popularised by Malcolm Gladwell in his book “Outliers,” based on research by psychologist K. Anders Ericsson.

However, the validity of the 10,000-hour rule is a subject of debate. While it is true that achieving expertise generally requires a significant amount of practice, there are several reasons why the 10,000-hour rule might not be a hard-and-fast rule:

1. Variation across fields: The amount of practice needed to achieve expertise can vary greatly depending on the field. Some skills or fields may require more or less practice to achieve mastery.
2. Individual differences: People learn at different rates, so the amount of practice needed to achieve expertise can vary from person to person. Factors such as innate talent, learning strategies, and the quality of instruction can all influence how quickly someone can become an expert.
3. Quality of practice: Ericsson’s research emphasises not just the quantity of practice, but also the quality. He argues that “deliberate practice,” which involves focused, goal-oriented practice with immediate feedback, is more important than sheer hours of practice.
4. Other factors: While practice is important, it is not the only factor that contributes to expertise. Other factors, such as the quality of instruction, the learning environment, and the learner’s motivation and persistence, can also play a role.

The original research by K. Anders Ericsson that inspired the 10,000-hour rule actually focused on the concept of “deliberate practice,” which is a specific type of focused, goal-oriented practice with immediate feedback. Ericsson found that top performers in fields like music and chess had typically accumulated thousands of hours of deliberate practice. However, he did not claim that 10,000 hours was a magic number or that it was sufficient to achieve expertise in any field.

Subsequent research has shown that the amount of deliberate practice needed to achieve expertise can vary greatly depending on the field and the individual. For example, a study by Macnamara et al published in the journal “Intelligence” in 2014 found that deliberate practice could explain between 26% and 1% of the variance in performance in games, music, sports, education, and professions, respectively. This suggests that while deliberate practice is important, it is not the only factor that contributes to expertise.

Individual differences can play a significant role. People have different learning rates, and factors such as innate talent, motivation, and the quality of instruction and feedback can all influence how quickly someone can become an expert.

In summary, while the 10,000-hour rule is a catchy concept and a useful reminder of the importance of practice, it is not a definitive or universally valid rule. Achieving expertise typically involves a combination of high-quality deliberate practice, innate talent, motivation, and other factors, and the amount of practice needed can vary depending on the field and the individual. Doing the time does not guarantee the achievement of expertise.

Against the odds

Experts, by virtue of their deep knowledge, experience, and honed skills, can often “beat the odds” in their respective fields. This means they can achieve outcomes that are significantly better than what would be expected based on chance or average performance. Here are a few examples:

1. Finance and Investing: Expert investors and financial analysts, who have a deep understanding of market trends, economic indicators, and financial analysis techniques, can often achieve returns that beat the market average. However, it is important to note that investing always involves risk, and even experts can’t guarantee success.
2. Medicine: Expert doctors can often diagnose and treat conditions more effectively than less experienced practitioners. For example, research has shown that surgeons who perform a high volume of a particular procedure often have better patient outcomes than those who perform it less frequently.
3. Sports: Expert athletes often outperform their peers, not only because of their physical skills but also because of their understanding of strategy, their ability to read the game, and their mental toughness. Coaches with a deep understanding of the sport can devise strategies that give their team an edge.
4. Science and engineering: Experts in these fields can often solve problems or develop innovations that others can’t. For example, expert engineers might be able to design more efficient systems or structures, and expert scientists might be able to make breakthrough discoveries.
5. Art and music: Expert artists and musicians can often create works that are more technically proficient, original, or emotionally resonant than those created by less experienced individuals.

There is of course no guarantee that experts will beat the odds every time. Beating the odds means only that the chances of success are better in the hands of recognised experts. A 20% improvement over the odds can be good in an area where failure among non-experts is 90%.  Let us be sensible about this; no one is suggesting that experts will never fail.

Artificial intelligence and robotics

Artificial intelligence (AI) and those working in robotics can develop a form of “expertise” in the sense that it can learn to perform specific tasks at a high level of proficiency, often surpassing human performance. This is particularly true for tasks that involve pattern recognition, prediction, and optimisation, such as image recognition, natural language processing, game playing, and route planning.

AI develops this proficiency through machine learning, a process that involves training an algorithm on a large amount of data. The algorithm learns to recognise patterns in the data and make predictions or decisions based on these patterns. In some ways, this is similar to how humans develop expertise through learning and practice.

However, there are also important differences between human expertise and AI “expertise” e.g.

1. Generalisation: Human experts can often generalize their knowledge and skills to new situations that they haven’t encountered before. In contrast, AI systems are typically more limited in their ability to generalize. They perform best on tasks that are similar to those they were trained on.
2. Understanding: Human experts typically have a deep understanding of their field. They can explain the principles and concepts behind their decisions, and they can adapt their approach based on this understanding. In contrast, AI systems often lack this kind of understanding. They can make accurate predictions or decisions, but they can’t explain the underlying reasons in a meaningful way.

Large language models, like OpenAI’s GPT-4, represent a significant advancement in the field of AI, particularly in natural language processing. These models are trained on vast amounts of text data and can generate remarkably coherent and contextually appropriate text. They can answer questions, write essays, summarize text, translate languages, and even generate creative writing, among other tasks.

In a sense, these models exhibit a form of “expertise” in handling language-based tasks. They can often perform these tasks at a level that can seem comparable to, or in some cases even surpass, human performance. They can “learn” from the patterns in the data they were trained on and apply this learning to generate responses to new inputs.

While AI systems can achieve remarkable proficiency in specific tasks, they also have significant limitations:

1. Narrow Specialisation: AI systems are typically very good at the specific tasks they were designed and trained for, but they struggle to generalize their learning to new tasks or situations. They lack the broad, flexible intelligence that humans have.
2. Lack of Understanding: AI systems can recognise patterns and make predictions based on these patterns, but they don’t have a conscious understanding of what they’re doing. They struggle to explain the underlying reasons for their decisions in a meaningful way.
3. Data Dependence: AI systems, particularly those based on machine learning, require large amounts of data to learn effectively. They struggle to learn from few examples, a capability that humans excel at.
4. Potential for Bias: Because these models learn from data, they can also learn and reproduce the biases present in that data. This can lead to outputs that are biased or offensive.
5. Inability to Verify Information: These models don’t have the ability to verify the information they generate. They can’t access or evaluate real-time information.
6. Ethical and Societal Considerations: AI systems don’t have values or ethics. They can be programmed to follow certain rules, but they don’t understand the meaning or importance of these rules. This raises a host of ethical and societal issues, such as how to ensure that AI systems are used responsibly and how to deal with the impacts of AI on jobs and society.
7. Brittleness: AI systems can be brittle, meaning they can fail in unexpected ways when they encounter situations that differ from those they were trained on. They also struggle to adapt to new situations without additional training.
8. Lack of Creativity and Emotional Intelligence: Current AI systems lack the ability to truly create or understand emotions in the way humans do. While there are AI systems that can generate music, art, or text that may seem creative, these are based on patterns in the data the AI was trained on, rather than any inherent creativity. Similarly, while some AI systems can recognise and respond to human emotions to a certain extent, they don’t actually understand or experience these emotions.

Take away points

Experience is a key factor in developing expertise. However, it is not the defining feature of expertise.

Expertise is a high level of knowledge and skill a field, gained through study, training, and experience. It involves deep knowledge, skill, problem-solving, recognition, continuous learning, and intuition.

Skills are developed through learning, practice, and experience. Several models explain this process, including Fitts and Posner’s Three-Stage Model, Dreyfus and Dreyfus’ Five-Stage Model, and Ericsson’s Deliberate Practice Model.

Expertise has significant practical value in various aspects of life and work, including problem-solving, decision-making, performance, innovation, teaching and mentoring, leadership, and providing trustworthy and credible information or advice.

Experts, by virtue of their deep knowledge, experience, and honed skills, can often “beat the odds” in their respective fields, achieving outcomes that are significantly better than what would be expected based on chance or average performance. This can be seen in fields such as finance and investing, medicine, sports, science and engineering, and art and music.

It is highly unlikely that the CQC’s ‘Experts by Experience‘ would have gone through the processes towards achieving true expertise. For all the above reasons, the CQC is fundamentally misguided in conflating and confusing experience with expertise.

Artificial Intelligence (AI) systems simulate a form of expertise but there are some important limitations.