Search results
Feb 2, 2024 · We then identify potential strategies to address limitations related to current approaches and propose 5 key considerations for the collection, organization, and analysis of text data from...
- Step 1: Defining The Research Question
- Step 2: Literature Search
- Step 3: Choice of The Effect Size Measure
- Step 4: Choice of The Analytical Method Used
- Step 5: Choice of Software
- Step 6: Coding of Effect Sizes
- Step 7: Analysis
- Step 8: Reporting Results
The first step in conducting a meta-analysis, as with any other empirical study, is the definition of the research question. Most importantly, the research question determines the realm of constructs to be considered or the type of interventions whose effects shall be analyzed. When defining the research question, two hurdles might develop. First, ...
2.2.1 Search strategies
Similar to conducting a literature review, the search process of a meta-analysis should be systematic, reproducible, and transparent, resulting in a sample that includes all relevant studies (Fisch and Block 2018; Gusenbauer and Haddaway 2020). There are several identification strategies for relevant primary studies when compiling meta-analytical datasets (Harari et al. 2020). First, previous meta-analyses on the same or a related topic may provide lists of included studies that offer a good...
2.2.2 Study inclusion criteria and sample composition
Next, researchers must decide which studies to include in the meta-analysis. Some guidelines for literature reviews recommend limiting the sample to studies published in renowned academic journals to ensure the quality of findings (e.g., Kraus et al. 2020). For meta-analysis, however, Steel et al. (2021) advocate for the inclusion of all available studies, including grey literature, to prevent selection biases based on availability, cost, familiarity, and language (Rothstein et al. 2005), or...
2.3.1 Types of effect sizes
The two most common meta-analytical effect size measures in management studies are (z-transformed) correlation coefficients and standardized mean differences (Aguinis et al. 2011a; Geyskens et al. 2009). However, meta-analyses in management science and related fields may not be limited to those two effect size measures but rather depend on the subfield of investigation (Borenstein 2009; Stanley and Doucouliagos 2012). In economics and finance, researchers are more interested in the examinatio...
2.3.2 Conversion of effect sizes to a common measure
After having defined the primary effect size measure for the meta-analysis, it might become necessary in the later coding process to convert study findings that are reported in effect sizes that are different from the chosen primary effect size. For example, a study might report only descriptive statistics for two study groups but no correlation coefficient, which is used as the primary effect size measure in the meta-analysis. Different effect size measures can be harmonized using conversion...
Choosing which meta-analytical method to use is directly connected to the research question of the meta-analysis. Research questions in meta-analyses can address a relationship between constructs or an effect of an intervention in a general manner, or they can focus on moderating or mediating effects. There are four meta-analytical methods that are...
Software solutions to perform meta-analyses range from built-in functions or additional packages of statistical software to software purely focused on meta-analyses and from commercial to open-source solutions. However, in addition to personal preferences, the choice of the most suitable software depends on the complexity of the methods used and th...
2.6.1 Coding sheet
The first step in the coding process is the design of the coding sheet. A universal template does not exist because the design of the coding sheet depends on the methods used, the respective software, and the complexity of the research design. For univariate meta-analysis or meta-regression, data are typically coded in wide format. In its simplest form, when investigating a correlational relationship between two variables using the univariate approach, the coding sheet would contain a column...
2.6.2 Inclusion of moderator or control variables
It is generally important to consider the intended research model and relevant nontarget variables before coding a meta-analytic dataset. For example, study characteristics can be important moderators or function as control variables in a meta-regression model. Similarly, control variables may be relevant in a MASEM approach to reduce confounding bias. Coding additional variables or constructs subsequently can be arduous if the sample of primary studies is large. However, the decision to incl...
2.6.3 Treatment of multiple effect sizes in a study
A long-debated issue in conducting meta-analyses is whether to use only one or all available effect sizes for the same construct within a single primary study. For meta-analyses in management research, this question is fundamental because many empirical studies, particularly those relying on company databases, use multiple variables for the same construct to perform sensitivity analyses, resulting in multiple relevant effect sizes. In this case, researchers can either (randomly) select a sing...
2.7.1 Outlier analysis and tests for publication bias
Before conducting the primary analysis, some preliminary sensitivity analyses might be necessary, which should ensure the robustness of the meta-analytical findings (Rudolph et al. 2020). First, influential outlier observations could potentially bias the observed results, particularly if the number of total effect sizes is small. Several statistical methods can be used to identify outliers in meta-analytical datasets (Aguinis et al. 2013; Viechtbauer and Cheung 2010). However, there is a deba...
2.7.2 Model choice
After controlling and correcting for the potential presence of impactful outliers or publication bias, the next step in meta-analysis is the primary analysis, where meta-analysts must decide between two different types of models that are based on different assumptions: fixed-effects and random-effects (Borenstein et al. 2010). Fixed-effects models assume that all observations share a common mean effect size, which means that differences are only due to sampling error, while random-effects mod...
2.8.1 Reporting in the article
The final step in performing a meta-analysis is reporting its results. Most importantly, all steps and methodological decisions should be comprehensible to the reader. DeSimone et al. (2020) provide an extensive checklist for journal reviewers of meta-analytical studies. This checklist can also be used by authors when performing their analyses and reporting their results to ensure that all important aspects have been addressed. Alternative checklists are provided, for example, by Appelbaum et...
2.8.2 Open-science practices
Another increasingly important topic is the public provision of meta-analytical datasets and statistical codes via open-source repositories. Open-science practices allow for results validation and for the use of coded data in subsequent meta-analyses (Polanin et al. 2020), contributing to the development of cumulative science. Steel et al. (2021) refer to open science meta-analyses as a step towards “living systematic reviews” (Elliott et al. 2017) with continuous updates in real time. MRQ su...
how to write up a meta-analysis. What is meta-analysis and how do I do it? Psychologists are typically interested in finding general answers to questions across this diverse discipline. Some examples are whether cognitive behaviour therapy (CBT) is efficacious for treating anxiety in children and adolescents (Cartwright-
- 1MB
- 30
Nov 30, 2021 · PDF | On Nov 30, 2021, Christopher Hansen and others published How to conduct a meta-analysis in eight steps: a practical guide | Find, read and cite all the research you need on ResearchGate
Key Points: Meta-analysis is the statistical combination of results from two or more separate studies. Potential advantages of meta-analyses include an improvement in precision, the ability to answer questions not posed by individual studies, and the opportunity to settle controversies arising from conflicting claims.
Feb 2, 2024 · This research note focuses predominantly on the approach we adopted for a Facebook Group, specifically on recruitment, planning for and mitigating for risk, Group membership, and data collection and analysis.
People also ask
Which meta-analysis should be used for Internet-based research?
How do you conduct a meta-analysis?
What are the advantages and disadvantages of a meta-analysis?
What can a meta-analysis tell us?
We now present a meta-analysis example step by step to introduce the reader to adequate data collection and analysis for meta-analyses on Internet-based research. For this example, we use the database from the meta-analysis by Villar et al. ( 2013 ), as this example represents a typical case.
