Exact Data

Begin by gathering all the data produced throughout the review process and apply a structured form. In this phases, your team will decide what information will be extracted, select a collection method, and apply it. Having clear methods and using tools will help ensure consistancy and accuracy.

Key Actions:

• Have access to the full text for all of the included studies. Try searching the library catalog, use the browser extension LibKey Nomad, or request articles the library does not have access to through Interlibrary Loan.
• Decide what information to extract. Fields may include study design, population, intervention, outcomes, and others depending on your question.
• Create and test your data extraction table. This may include adding a small number of studies and seeing if the process runs smoothly and all data fields have been correctly entered.
• Have at least two reviews extract data. This will ensure that nothing is missed and that the selected tools are working correctly. Resolve and review any discrepancies between the two reviewers.

Compile and Describe Included Studies

Once data is extracted, organize the pieces of information into structured evidence tables and summary tables. These make it easier to compare studies and identify patterns or inconsistencies.

Suggested Tables Might Include:

• Study Characteristics Table: design, sample size, setting, population, interventions, and outcomes.

• Evidence Table: a more detailed summary including statistical significance, quality rating (e.g., Jadad score), benefit magnitude, and measures like Absolute Risk Reduction (ARR) or Number Needed to Treat (NNT.) This is a great tool for helping readers quickly understand the scope and context of the studies included.

Evidence Table Columns:

1. Condition

2. Study Design

3. Author, Year

4. N (sample size)

5. Statistically Significant?

6. Quality of Study

7. Magnitude of Benefit

8. Absolute Risk Reduction (ARR)

9. Number Needed to Treat (NNT)

10. Comments

These tables help ensure transparency and set the stage for synthesis.

Synthesize Results

Once your data is organized, determine whether a quantitative or qualitative synthesis would be most appropriate. The synthesis should be chosen according to the kind of data at hand, their consistency across studies, and your review question.

Types of Synthesis:

• Quantitative Synthesis (Meta-Analysis):

  • Use when studies are similar enough (in design, population, outcomes, etc.) that their results can be combined

  • Pooling data from different databases and calculate the overall effect

  • Report metrics like:

    • Pooled effect size or how strong the effect is overall

    • Confidence intervals or the range in which the true effect most likely falls

  • Tools include: RevMan Web or R Source / Python coding with specific meta-analysis packages

• Qualitative/Narrative Synthesis:

  • Use when studies are too different (in design, measures, populations, or outcomes) to combine or if the data is descriptive and not numerical

  • This involves:

    • Describing findings across studies

    • Highlighting trends, patterns, and differences

    • Idetifying where studies agree or conflict

Assess Publication Bias

In quantitative syntheses, it’s important to consider whether publication bias may be influencing your results. Publication bias stems from solely including works with significant results making the results of a meta-analysis overestimate the real effect of an intervention or exposure.

Common Methods:

• Funnel plots: a scatter plot of each study's effect size versus its precision (standard error or sample size) and should ideally looks like a symmetrical inverted tunnel

• Egger’s test : a statistical test that detects funnel plot asymmetry and can help determine relationships between effect size and precision

• Other tests like Trim-and-Fill Method and Bregg's Test

Publication bias can impact the reliability of your conclusions, especially if smaller, negative studies are underrepresented. Always assess bias in both visual (funnel plot) and statistical (Egger's test) ways. If bias is present, you will need to discuss the impact on results.