Measuring Clinician Confidence to Show Educational Performance: Is It Valid?

Track: Research

Session Number: 2004
Date: Wed, Sep 27th, 2017
Time: 11:35 AM - 11:50 AM

Description:

Recent educational research has gone back and forth on the value of measuring clinician confidence as a metric to show improved skill and performance. Evans et al. (2014) found that neither commitment to change nor confidence strongly predicted whether an educational participant implemented change in their practice after a live symposium. However, they suggested that confidence may be a valuable metric to continue to explore. Many educators include a confidence metric within their outcomes reports, but are often unsure on how to interpret findings. Increased confidence leads to perceptions that clinicians are showing increased mastery of content; decreased confidence can show that clinicians are reflecting on content that they previously did not understand and may be more willing to change. Can confidence be better explored as a measurement of performance?

We took a retrospective look at 45 educational outcomes assessments that included confidence scoring as well as evidence-based decision-making scoring to view the relationship between confidence and performance. The study design for all 45 assessments was similar: case-vignette based survey distributed 30-45 days after the educational activity, participant scores were matched to a nonparticipant control, physician/NP/PA participants only, surveys were scored based on percent alignment with evidence, 10-point confidence question regarding management of that particular therapeutic area. Overall, 4,298 individual responses were included in the data set (2,048 participants and 2,250 matched nonparticipant control).

The results showed that while participants overall scored higher and were more confident than nonparticipants, confidence and performance were not highly correlated (r = .287). Thus, the higher performance does not always mean higher confidence and vice versa.

Based on research conducted by Cash et al. (JCEHP 2011), clinicians can be divided into 4 quadrants based on their confidence and knowledge/performance scores: mastery (high confidence, high performance); misinformed mistakes (high confidence, low performance); doubt and hesitation (low confidence, high performance); and uninformed paralysis (low confidence, low performance). We split the respondents into these 4 groups based on median nonparticipant scores: 8-10 on 10-point confidence scale = high confidence; 55%+ on evidence-based performance = high performance. Based on this metric, participants are significantly more likely than nonparticipants to fall in the mastery quadrant (47% vs 35%, P < .001). Much of this increase comes from decrease in the uninformed paralysis and misinformed mistake quadrant. However, there was not as much difference between participants and nonparticipants in the doubt and hesitation quadrant (21% vs 18%, respectively).

This session will highlight this new research as well as conduct an open discussion for input on the value on confidence measurement in outcomes research. Do educators/supporters view confidence as a valid metric? Do educators attempt to increase clinician confidence? How can these metrics be used to show value and success of education?
Session Type: Presentation

Learning Objective 1: Understand the strengths and weaknesses of assessing clinician confidence
Learning Objective 2: Develop new ideas for utilizing confidence in outcomes measurement
Learning Objective 3: Determine ways to improve clinician confidence through CME
Session Type: Presentation

Learning Objective 1: Understand the strengths and weaknesses of assessing clinician confidence
Learning Objective 2: Develop new ideas for utilizing confidence in outcomes measurement
Learning Objective 3: Determine ways to improve clinician confidence through CME