The Future of Research: AI-Driven Automation in Systematic Reviews
The systematic literature review (SLR) is the gold standard that provides firm scientific evidence to support decision-making. SLRs play a vital role in offering a holistic assessment of efficacy, safety, and cost-effectiveness of a diagnostic aid or therapy by synthesizing data from various clinical studies. The process of conducting an SLR begins with formulating an unbiased search strategy to identify pertinent research articles for a thorough examination. In today’s digital age, literature searches often uncover numerous publications, necessitating intensive manual review and analysis, thereby potentially resulting in findings becoming outdated by the time an SLR is completed and published. Artificial intelligence (AI) has the potential to greatly enhance the efficiency and precision of SLRs. AI significantly enhances the efficiency and accuracy of SLRs by automating the literature search, screening, and data extraction processes, reducing the time and potential for human error. Tools like Rayyan and DistillerSR leverage machine learning and natural language processing to streamline and standardize these tasks, making SLRs more scalable and less biased. In 2024, Rayyan and DistillerSR have both introduced significant updates to enhance their functionality for systematic reviews. Rayyan has incorporated a beta version of the PRISMA guideline integration, an auto-resolver feature to automatically handle conflicts between reviewer decisions, and significant improvements to its mobile app, including offline capabilities and team progress monitoring. Additionally, Rayyan has enhanced its advanced filtration and de-duplication tools and offers comprehensive training sessions to facilitate easier onboarding for new users . DistillerSR has focused on boosting its AI-powered automation for tasks like data extraction and reference screening, improving integration with reference management tools such as EndNote and Mendeley, and upgrading its user interface for a more intuitive experience. Furthermore, DistillerSR ensures compliance with various regulatory standards, making it suitable for clinical trials and sensitive research areas. These updates reflect a continued commitment to improving the efficiency, usability, and compliance of systematic review processes through advanced technological solutions.
There are certain guidelines to increase rigor, transparency, and replicability of SLRs. AI and Machine Learning Techniques (MLTs) developed with computer programming languages can provide methods to increase SLRs’ speed, rigor, transparency, and repeatability. Aimed towards researchers who want to utilize AI and MLTs to synthesise and abstract data obtained through a SLR, this article sets out how computer languages can be used to facilitate unsupervised machine learning for synthesising and abstracting data sets extracted during a SLR. Utilizing an already known qualitative method, Deductive Qualitative Analysis, this article illustrates the supportive role that AI and MLTs can play in the coding and categorisation of extracted SLR data, and in synthesising SLR data. Using a data set extracted during a SLR as a proof-of-concept, this article will include the coding used to create a well-established MLT, Topic Modelling using Latent Dirichlet allocation. This technique provides a working example of how researchers can use AI and MLTs to automate the data synthesis and abstraction stage of their SLR, and aide in increasing the speed, frugality, and rigor of research projects.
Typically, SLRs involve the following steps:
- Product Development
- Literature search
- Literature screening
- Evidence Generation
- Quality assessment
- Preparation of SLR
AI can efficiently assist in carrying out certain steps within SLRs such as,
Automated Search and Screening
AI is primarily utilized in SLRs to expedite the initial phases by automating literature search and article screening based on predefined criteria. Search engines increasingly employ AI, by enhancing Retrieval-Augmented Generation (RAG) frameworks with large language models. These frameworks enable the formulation of complex search queries, surpassing the limitations of conventional keyword-based searches.
ML classifiers are utilized to discover more relevant articles. These classifiers undergo training on an initial set of user-selected papers. Then, through iterative processes, they utilize automatic classifications to refine and improve their ability to identify further pertinent literature.
Automated tools leverage AI techniques analyze various components of an article such as its title, abstract, or full text. Natural Language Processing (NLP) algorithms dissect abstracts, titles, and keywords to gauge their relevance to the research topic. Additionally, these AI techniques can incorporate statistical selection processes to identify key terms characterizing each cluster. This involves scoring each citation based on the presence of keywords, aiding screeners in making more informed decisions regarding their relevance. Consequently, resulting clusters highlight the most representative terms, facilitating better judgment regarding the inclusion or omission of a publication from the analysis.
Data Extraction and Evidence Generation
In health research, researchers apply various protocols for literature review depending on the type of report to be generated. These include PICO (population, intervention, comparison, outcome), PCC (population, context, concept), PICODR (elements of PICO plus duration and results), PIBOSO (population, intervention, background, outcome, study design, and others).
AI processes data from predetermined fields in interventional, diagnostic, or prognostic SLRs. NLP algorithms can extract crucial details like study methodologies, results, and statistical information which are subsequently synthesized and analyzed to derive valuable insights. AI technologies leverage domain ontology to structure the data, providing a formal depiction of variable types and their interrelationships.
Quality Assessment
Minimizing selection bias and enhancing both the external and internal validity of selected publications in a SLR is crucial. Evaluating the quality of an SLR provides insight into its overall robustness and credibility. AI can assist quality assessment by analyzing various factors such as study design, sample size, and methodology of included studies.
Machine learning algorithms, when trained on available datasets, can identify patterns suggestive of high-quality research, thus aiding researchers in efficiently assessing evidence reliability. Many validated checklists, like PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), advise independent reviewer assessment of bias in literature search and selection. Integrating manual quality checks with automated screening is crucial for identifying gaps and inconsistencies, which can then be addressed by reconciling differences between the screener’s and reviewer’s decisions.
Analysis, Data Visualization, and Preparation of the Report
AI-powered tools can aid in meta-analyzing data derived from multiple studies, allowing researchers to synthesize findings quantitatively and evaluate overall effect sizes. Through semantic analysis and clustering techniques, AI can facilitate the organization and categorization of extensive literature volumes. By discerning common themes and relationships among studies, AI algorithms assist researchers in gaining profound insights into existing literature, assessing the current research landscape, and pinpointing areas for further exploration.
AI-driven visualization techniques can streamline the presentation of intricate information, making it more understandable and facilitating decision-making. Leveraging algorithms and models, AI technology can identify patterns, trends, outliers, and correlations within diverse data sets. Insights and recommendations gleaned from SLR data can aid researchers in comprehending the implications of knowledge gaps, processes, research methodologies, and policies.
By integrating feedback from researchers and refining algorithms based on new data, AI holds the potential to continually enhance the accuracy and efficiency of SLRs, thereby elevating the quality of research outcomes.
Challenges and Future Directions
Despite the transformative potential of AI in SLR and decision making, several challenges remain. Ensuring the transparency, interpretability, and ethical use of AI algorithms is paramount to fostering trust and acceptance within the healthcare community. Additionally, addressing issues related to data quality, interoperability, and bias in AI-driven analyses is essential for safeguarding the integrity of evidence-based medicine.
Looking ahead, continued research and innovation are necessary to harness the full potential of AI in healthcare. Collaborative efforts between interdisciplinary teams comprising clinicians, researchers, data scientists, and policymakers will be instrumental in overcoming challenges and unlocking new opportunities for leveraging AI in evidence synthesis and decision-making.
In conclusion, the integration of AI in Systematic Literature Reviews and Health Technology Assessment decision-making represents a paradigm shift in evidence-based medicine. By harnessing the power of AI, we can streamline processes, enhance decision quality, and ultimately improve healthcare outcomes for patients worldwide. As we navigate this transformative journey, it is imperative to prioritize ethics, transparency, and collaboration to realize the full benefits of AI in healthcare.
References:
- Francisco B, Angelo S, Osborne F, et al. Artificial Intelligence for Literature Reviews: Opportunities and Challenges doi: https://doi.org/10.48550/arXiv.2402.08565
- Marshall, C., & Wallace, D. (2019). Toward automated systematic reviews: A study of the precision and recall of AI-assisted screening. Journal of the American Medical Informatics Association, 26(11), 1215-1222. doi: 10.1186/s13643-019-1074-9.
- O’Mara-Eves, A., et al. (2015). Using text mining for study identification in systematic reviews: A systematic review of current approaches. Systematic Reviews, 4(1), 5. DOI: 1186/2046-4053-4-5
Authors:
Asif Syed, PhD.
Senior Scientific Writer II
Connect with Asif on LinkedIn
Raghuraj Puthige, PhD.
Function Head, Medical Communications – Enago Life Sciences
Connect with Raghuraj on LinkedIn
Dr. Anupama Kapadia
General Manager, Enago Life Sciences
Connect with Anupama on LinkedIn