Integrating Meta-Analysis into Your Educational Outcomes Portfolio to Demonstrate and Drive Success
Track: Full Session (30 minutes)
Most medical education providers focus on outcomes for single activities. While this is helpful for gauging the success of a single activity, it does not provide information regarding whether or not the provider’s instructional models and overall strategies are having a positive impact on healthcare provider (HCP) performance and patient health. Meta-analyses can provide that insight, and should become part of standard procedures for medical education in order to better understand the overall impact of multiple formats and activities. Meta-analyses in the context of medical education can be thought of as akin to clinical guidelines, which pull information from multiple sources to develop a comprehensive evaluation and strategy for tackling a particular problem, in this case, how to best educate HCPs.
In addition to the overall benefit of gaining insight regarding an organization’s success as a medical education provider, there are 3 distinct advantages to conducting meta-analyses. First, they increase statistical power due to having multiple sources of input. Second, they resolve uncertainty regarding an overall educational program if results from individual activities disagree. Third, they can inform strategic planning for future activities by prompting an evaluation of patterns or outliers. This can be achieved for each of Moore’s Levels, mainly knowledge, competence, and behavior. For example, if the results of a meta-analysis were not favorable (i.e., the overall effect size was low), drilling down to data from individual activities can help determine if the poor results were due to content, design, format, or other barriers. Conversely, if the results are favorable, elements of success can be determined and continued. In order for the meta-analysis to be representative of the provider’s programs, it is important to establish criteria for which activities to include, while avoiding selection bias or “cherry picking.”
The two main goals for this workshop are to provide learners with an appreciation for the value of meta-analysis as well as the skills to conduct a meta-analysis. It is hoped that following this workshop, participants will be motivated to take what they have learned back to their workplaces, begin conducting meta-analyses, and use the information obtained from the meta-analysis to guide strategy in developing future activities to maximize success.
It is not the intention of this presentation to provide a full understanding of the mathematics and statistical theory behind meta-analysis, but rather a basic overview.