Meta-analysis for Supporting Empirical Theories in Educational Sciences

Sławomir Pasikowski

doi:10.18778/2450-4491.20.05

Authors

Sławomir Pasikowski University of Lodz https://orcid.org/0000-0002-0768-1596

DOI:

https://doi.org/10.18778/2450-4491.20.05

Keywords:

evidence-based approach, meta-analysis, theory, systematic literature review

Abstract

This article explores the role of meta-analysis and systematic review in developing and refining empirical theories in educational sciences. It highlights the method’s value in synthesizing research findings, identifying patterns, and improving the explanatory power and coherence of theories. It also underscores the skepticism present in academic circles, especially concerning meta-analysis. While meta-analysis is widely used in evidence-based approaches, its adoption in educational research sometimes remains locally limited due to concerns about data quality, methodological heterogeneity, publication bias, and perceived epistemic incompatibility with constructivist or interpretive paradigms. The author argues that these challenges can be addressed through methodological rigor, data transparency, proper contextualization, and interdisciplinary training in statistics, epistemology, and logic. Meta-analysis is presented not only as a statistical tool, but as a means of supporting intellectual inquiry and collaborative theory-building. The article calls for greater integration of meta-analytic methods into education research, emphasizing their potential to enhance the quality, comparability, and transparency of scientific knowledge.

Author Biography

Sławomir Pasikowski, University of Lodz

Sławomir Pasikowski – Ph.D. in Education Sciences, specializations: the methodology of scientific research in education. Author of publications on the methodology of educational research and the use of statistics in social research. Vice-chair of the Section for Methodology of Research on Education at the Committee on Pedagogical Sciences of the Polish Academy of Sciences (PAN). Member of the Laboratory of Research Tools at the Committee on Pedagogical Sciences of the PAN.

References

Audigier V., White I. R., Jolani S., Debray T. P. A., Quartagno M., Carpenter J., Resche-Rigon M. (2018) Multiple Imputation for Multilevel Data with Continuous and Binary Variables, “Statistical Science”, no. 33, pp. 160–183, https://doi.org/10.1214/18-STS646 DOI: https://doi.org/10.1214/18-STS646

Bartoš F., Maier M., Wagenmakers E., Doucouliagos H., Stanley T. D. (2023) Robust Bayesian Meta‐Analysis: Model‐Averaging Across Complementary Publication Bias Adjustment Methods, Research Synthesis Methods, no. 14(1), pp. 99–116, https://doi.org/10.1002/jrsm.1594 DOI: https://doi.org/10.1002/jrsm.1594

Cumming G., Calin-Jageman R. (2017) Introduction to the New Statistics. Estimation, Open Science, and Beyond, New York, NY – London, Routledge, https://doi.org/10.4324/9781315708607 DOI: https://doi.org/10.4324/9781315708607

Gajda A., Karwowski M., Beghetto R.A. (2017) Creativity and Academic Achievement: A Meta-Analysis, Journal of Educational Psychology, no. 109, pp. 269–299, https://doi.org/10.1037/edu0000133 DOI: https://doi.org/10.1037/edu0000133

Gosling C.J., Solanes A., Fusar-Poli P., Radua J. (2023) Metaumbrella: the First Comprehensive Suite to Perform Data Analysis in Umbrella Reviews with Stratification of the Evidence, BMJ Mental Health, no. 26, pp. 1–8, https://doi.org/10.1136/bmjment-2022-300534 DOI: https://doi.org/10.1136/bmjment-2022-300534

Gough D., Oliver S., Thomas J. (2017) An Introduction to Systematic Reviews, London, Sage.

Grund S., Ludtke O., Robitzsch A. (2018) Multiple Imputation of Missing Data for Multilevel Models: Simulations and Recommendations, Organizational Research Methods, no. 21, pp. 111–149, https://doi.org/10.1177/1094428117703686 DOI: https://doi.org/10.1177/1094428117703686

Harrer M., Cuijpers P., Furukawa T. A., Ebert D. D. (2022) Doing Meta-Analysis with R. A Hands-On Guide, London – New York, NY, Taylor & Francis Group, https://doi.org/10.1201/9781003107347 DOI: https://doi.org/10.1201/9781003107347

Hattie J. (2008) Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement, London – New York, NY, Routledge.

Hempel C. G. (1964) Fundamentals of Concept Formation in Empirical Science, Chicago, IL – London, University of Chicago Press.

Hempel C. G. (1965) Aspects of Scientific Explanation and Other Essays in the Philosophy of Science, New York, NY, Free Press.

Heyvaert M., Maes B., Onghena P. (2011) Applying Mixed Methods Research at the Synthesis Level: An Overview, Research in the Schools, no. 18(1), pp. 12–24.

Hong Q. N., Pluye P., Bujold M., Wassef M. (2017) Convergent and Sequential Synthesis Designs: Implications for Conducting and Reporting Systematic Reviews of Qualitative and Quantitative Evidence, Systematic Reviews, no. 6(1):61, https://doi.org/10.1186/s13643-017-0454-2 DOI: https://doi.org/10.1186/s13643-017-0454-2

Ioannidis J. P. A. (2009) Integration of Evidence from Multiple Meta-Analyses: A Primer on Umbrella Reviews, Treatment Networks and Multiple Treatments Meta-Analyses, Canadian Medical Association Journal, no. 181(8), pp. 488–493, https://doi.org/10.1503/cmaj.081086 DOI: https://doi.org/10.1503/cmaj.081086

Lakatos I. (1989) The Methodology of Scientific Research Programmes, Cambridge, Cambridge University Press.

León S. P., Garrido M. G., Martínez I. G., Lipnevich A. (2025) Passing TPACK through the Metascience Circle: A Meta-analytic Systematic Review of the Application of the Model, May 14, preprint, https://doi.org/10.35542/osf.io/c5qsr_v1 DOI: https://doi.org/10.35542/osf.io/c5qsr_v1

León S. P., Lipnevich A., Garrido M. G. (2025) Is All That Glitters Gold? Meta-analytical Systematic Review of the Effect of the Montessori, April 7, preprint, https://doi.org/10.35542/osf.io/mf9ec_v1 DOI: https://doi.org/10.35542/osf.io/mf9ec_v1

León S. P., Panadero E., García-Martínez I. (2023) How Accurate Are Our Students? A Meta-Analytic Systematic Review on Self-assessment Scoring Accuracy, Educational Psychology Review, no. 35:106, https://doi.org/10.1007/s10648-023-09819-0. DOI: https://doi.org/10.1007/s10648-023-09819-0

Merton R. K. (2017) Social Theory and Social Structure, New York, NY, Free Press.

Moher D., Liberati A., Tetzlaff J., Altman D. G., Prisma Group (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA Statement, PLoS Medicine, no. 6(7), e1000097, https://doi.org/10.1371/journal.pmed.1000097 DOI: https://doi.org/10.1371/journal.pmed.1000097

Page M. J., Higgins J. P. T., Sterne J. A. C. (2019) Assessing Risk of Bias Due to Missing Results in a Synthesis, in: Cochrane Handbook for Systematic Reviews of Interventions, J. P. T. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, V. A. Welch (eds.), Oxford, The Cochrane Collaboration and John Wiley & Sons, https://doi.org/10.1002/9781119536604.ch13 DOI: https://doi.org/10.1002/9781119536604.ch13

Przełęcki M. (1988) Logika teorii empirycznych, Warszawa, Wydawnictwo Naukowe PWN.

Pigott T. D., Polanin J. R. (2019) Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review, Review of Educational Research, no. 90(1), pp. 24–46, https://doi.org/10.3102/0034654319877153 DOI: https://doi.org/10.3102/0034654319877153

Popper K. R. (2002) The Logic of Scientific Discovery, London – New York, NY, Routledge.

Reichenbach H. (1963) The Rise of Scientific Philosophy, Berkeley & Los Angeles, University of California Press.

Sandelowski M., Barroso J. (2007) Handbook for Synthesizing Qualitative Research, New York, NY, Springer.

Schmidt F. L. (1992) What Do Data Really Mean? Research Findings, Meta-Analysis, and Cumulative Knowledge in Psychology, American Psychologist, no. 47(10), pp. 1173–1181, https://doi.org/10.1037//0003-066X.47.10.1173 DOI: https://doi.org/10.1037//0003-066X.47.10.1173

Shea B. J., Reeves B. C., Wells G., Thuku M., Hamel C., Moran J., Moher D., Tugwell P., Welch V., Kristjansson E., Henry D. A. (2017) AMSTAR 2: A Critical Appraisal Tool for Systematic Reviews That Include Randomised or Non-Randomised Studies of Healthcare Interventions, or Both, BMJ, 358:j4008, https://doi.org/10.1136/bmj.j4008 DOI: https://doi.org/10.1136/bmj.j4008

Slim K., Marquillier T. (2022) Umbrella Reviews: A New Tool to Synthesize Scientific Evidence in Surgery, Journal of Visceral Surgery, no. 159(2), pp. 144–149, https://doi.org/10.1016/j.jviscsurg.2021.10.001 DOI: https://doi.org/10.1016/j.jviscsurg.2021.10.001

van Fraassen B. C. (2004) The Scientific Image, Oxford, Oxford University Press.

Vevea J. L., Hedges L. V. (1995) A General Linear Model for Estimating Effect Size in the Presence of Publication Bias, Psychometrika, no. 60, pp. 419–435, https://doi.org/10.1007/BF02294384 DOI: https://doi.org/10.1007/BF02294384

Wiśniewska E., Karwowski M. (2007) Efektywność treningów twórczości – podejście metaanalityczne, Ruch Pedagogiczny, no. 3–4, pp. 31–50.

Wittgenstein L. (1958) Philosophical Investigations, New York, NY, Macmillan Publishing Co.