Systematic Reviews

by Bruce Abbott, Amy Studer, Nicole A Capdarest-Arest, Megan G. Van Noord, Erik Davis Fausak, Lorri Zipperer — November 20, 2020

Systematic Reviews

by Bruce Abbott, Amy Studer, Nicole A Capdarest-Arest, Megan G. Van Noord, Erik Davis Fausak, Lorri Zipperer — November 20, 2020

Systematic Reviews

by Bruce Abbott, Amy Studer, Nicole A Capdarest-Arest, Megan G. Van Noord, Erik Davis Fausak, Lorri Zipperer – November 20, 2020

Courtesy of the National Library of Medicine

This guide provides an overview of the systematic review process, definitions, best practices related to database selection, developing and documenting the search strategy, and managing publication citation data. Resources and strategies for finding existing systematic reviews are also included.

Are you looking to collaborate with a librarian on a systematic review? Complete our intake form to request a consult.

Also see our guide on Systematic Reviews – Tools, which covers tools to help researchers perform and manage health sciences systematic reviews.

Bruce Abbott


Health Sciences Librarian

btabbott@ucdavis.edu

916-734-3529

In this Guide:

What is a Systematic Review?

A systematic review is a summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies. (CEBM).

Key characteristics of a systematic review:

  • a clearly stated set of objectives with pre-defined eligibility criteria for studies
  • an explicit, reproducible methodology
  • a systematic search that attempts to identify all studies that would meet the eligibility criteria
  • an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias
  • a systematic presentation, and synthesis, of the characteristics and findings of the included studies. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 6)

Meta analysis is a systematic review that uses quantitative methods to synthesize and summarize the results.

Similarities and Differences Between Systematic and Literature Reviews

Reproduced from: Bettany-Saltikov, J. (2010). Learning how to undertake a systematic review: Part 1. Nursing Standard, 24(40): 47-55.

How to Books

  • Finding What Works in Health Care by Jill Eden; Laura Levit; Alfred Berg; Sally Morton (Editors); Committee on Standards for Systematic Reviews of Comparative Effectiveness Research; Institute of Medicine
    Institute of Medicine (IOM) recommends 21 standards for developing high-quality systematic reviews of comparative effectiveness research. The standards address the entire systematic review process from the initial steps of formulating the topic and building the review team to producing a detailed final report that synthesizes what the evidence shows and where knowledge gaps remain.
    Call Number: Blaisdell Medical Library, W 84.3 I595f 2011
    Publication Date: 2011-06-20
    Electronic Access
    .
  • Reviewing Research Evidence for Nursing Practice: Systematic Reviews
    Electronic Access

  • Systematic Reviews in Health Care: Meta-Analysis in Context
    Call Number: Carlson Health Sci Library, WA20.5 S995 2001
  • Systematic Reviews in Health Care: A Practical Guide
    Call Number: Carlson Health Sci Library, W20.5 S9955 2001
  • Systematic Reviews to Support Evidence-Based Medicine: How to Review and Apply Findings of Healthcare Research
    Call Number: Blaisdell Medical Library, WB 102 S97 2011

Guidelines

 

Levels of Evidence

Evidence Pyramid

The EVIDENCE PYRAMID is often used to illustrate the development of evidence. At the base of the pyramid is animal research and laboratory studies – this is where ideas are first developed. As you progress up the pyramid the amount of information available decreases in volume, but increases in relevance to the clinical setting.

Meta Analysis – systematic review that uses quantitative methods to synthesize and summarize the results.

Systematic Review – summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Randomised Controlled Trial – Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Cohort Study – Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Case Control Study – study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Case Series – report on a series of patients with an outcome of interest. No control group is involved.  (Definitions from CEBM)

 

Other Levels of Evidence

NHMRC – How to Use the Evidence: Assessment and Application of Scientific Evidence


Joanna Briggs Institute – Levels of Evidence – One of four levels of evidence are assigned to all conclusions drawn in JBI Systematic Reviews.

Centre for Evidence Based Medicine (UK) – Levels of Evidence (March 2009)

Locating Systematic Reviews

Before starting work on a systematic review check to ensure that a systematic review does not already exist, or is currently under way.

A check of what systematic reviews already exist in your discipline may also provide a starting point for selecting a review topic.

Systematic Review Databases

The following databases extensively index the systematic review literature in their discipline.

General

  • Cochrane Database of Systematic Reviews (CDSR) – Cochrane Reviews
    Includes all Cochrane Reviews (and protocols) prepared by Cochrane Review Groups. Each Cochrane Review is a peer-reviewed systematic review that has been prepared and supervised by a Cochrane Review Group according to the Cochrane Handbook for Systematic Reviews of Interventions or Cochrane Handbook for Diagnostic Test Accuracy Reviews.
  • Database of Abstracts of Reviews of Effects (DARE)
    Database of Abstracts of Reviews of Effects seeks to identify the best quality systematic reviews being published. It complements the Cochrane Database of Systematic Reviews by offering a selection of quality-assessed reviews in those subjects where there is currently no Cochrane Review.
  • Systematic Reviews Journal
    An online-only open-access peer-reviewed medical journal published by BioMed Central that focuses on systematic reviews. Articles are either about specific systematic reviews or about their protocols, methodologies, findings, followup, etc.

Search Strategy

Reporting Bias

Types of reporting bias include:

  • Publication bias
  • Time lag bias
  • Multiple (duplicate) publication bias
  • Location bias
  • Citation bias
  • Language bias
  • Outcome reporting bias.

For details see: Cochrane Handbook for Systematic Reviews of Interventions (2008, Chapter 10)

Search Filters

The following sites include examples of pre-tested search filters:

AHRQ – Making Healthcare Safer III Patient Safety Practices – a set of systematic reviews on patient safety issues including search strategies.

McMaster University – HEDGES Project – including MEDLINE, EMBASE, and PsycINFO

SIGN – Scottish Intercollegiate Guidelines Network

Centre for Reviews and Dissemination – Search Filter Resource

NCBI – Clinical Queries Table

PubMed Search Strategies Blog
“This blog has been created to share PubMed search strategies. Search strategies posted here are not perfect. They are posted in the hope that others will benefit from the work already put into their creation and/or will offer suggestions for improvements.”

The InterTASC Information Specialists’ Sub-Group Search Filter Resource
The InterTASC Information Specialists’ Sub-Group (ISSG) is the group of information professionals supporting research groups within England and Scotland providing technology assessments to the National Institute for Health and Clinical Excellence (NICE) and other associated Information Specialists.

OvidSP Medline Expert Searches

PubMed Special Queries Searches

Translating search strategies from one platform to another

Database Syntax Guide – from Cochrane–search syntax from multiple platforms

MEDLINE Transpose – translates searches from PubMed to  Ovid MEDLINE and vice versa.

Embase Search Tips

Embase Search Tips

PsycInfo Search Blog

PsycInfo Topics Guide – includes suggested search terms

For more information …

Centre for Evidence-Based Medicine – Formulating Answerable Clinical Questions

HealthLinks, University of Washington – Construct Well-Built Clinical Questions Using PICO

Evidence-Based Medicine: How to Practice and Teach EBM
Call Number:
Carlson Health Sci Library WB 102 E933 2005
Blaisdell Medical Library WB 102 E933 2005

Searching Systematically

The Institute of Medicine recommends in their publication Finding What Works in Health Care: Standards for Systematic Reviews, working with a librarian or other information specialist to plan out your search strategy and to peer-review the final strategy used: Standard 3.1.1 and 3.1.3.

Searching is a critical part of conducting the systematic review, as errors in the search process potentially result in a biased or otherwise incomplete evidence base. Searches for systematic reviews need to be constructed to maximise recall and deal effectively with a number of potentially biasing factors. (McGowan, 2005, p. 75)

You should aim to be as extensive as possible when conducting searches for systematic reviews. However, it may be necessary to strike a balance between the sensitivity and precision of your search.

Sensitivity – the number of relevant results identified divided by the total number of relevant results in existence

Precision – the number of relevant results identified divided by the total number of results identified.

Increasing the comprehensiveness of a search will reduce its precision and will retrieve more non-relevant results. However,

… at a conservatively-estimated reading rate of two abstracts per minute, the results of a database search can be ‘scanread’ at the rate of 120 per hour (or approximately 1000  over an 8-hour  period), so the high yield and low precision associated with systematic review searching is not as  daunting as it might at first appear in comparison with the total  time to be invested in the review. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 130)

A useful technqiue is to check the search strategies used in other systematic reviews for hints on terms and combinations to use. Several groups have also developed pre-tested search filters.

1. Developing Answerable Questions

A technique often used in health research for formulating a clinical question is the PICO Model.

Using PICO, a clinical question will have 4 elements – Patient, Intervention, Comparison and Outcome.

The Cochrane Handbook for Systematic Reviews of Interventions (2008, p. 85-89) includes the following factors to consider when developing criteria for your PICO elements.

Patient, Population or Problem
  • How is the disease/condition defined?
  • What are the most important characteristics that describe the people?
  • Are there any relevant demographic factors (eg. age, sex, ethnicity)?
  • What is the setting (eg. hospital, community, etc)?
  • Who should make the diagnosis?
  • Are there any other types of people who should be excluded from the review (because they are likely to react to the intervention in a different way)?
  • How will studies involving only a subset of relevant participants be handled?
Interventions and Comparisons
  • What are the experimental and control (comparator) interventions of interest?
  • Does the intervention have variations (eg. dosage/intensity, mode of delivery, personnel who deliver it, frequency of delivery, duration of delivery, timing of delivery)?
  • Are all variations to be included (for example is there a critical dose below which the intervention may not be clinically appropriate)?
  • How will trials including only part of the intervention be handled?
  • How will trials including the intervention of interest combined with another intervention (co-intervention) be handled?
Outcomes
  • Main outcomes, for inclusion in the ‘Summary of findings’ table, are those that are essential for decision-making, and should usually have an emphasis on patient-important outcomes.
  • Primary outcomes are the two or three outcomes from among the main outcomes that the review would be likely to be able to address if sufficient studies are identified, in order to reach a conclusion about the effects (beneficial and adverse) of the intervention(s).
  • Secondary outcomes include the remaining main outcomes (other than primary outcomes) plus additional outcomes useful for explaining effects.
  • Ensure that outcomes cover potential as well as actual adverse effects.
  • Consider outcomes relevant to all potential decision makers, including economic data.
  • Consider the type and timing of outcome measurements.

We have two resources which provide a PICO search interface (both require being on the campus network, including the VPN):

Ovid MEDLINE PICO search: http://access.ovid.com/custom/ucdavispico/

Embase: https://embase.com/a/#/picoSearch 

 

2. Identify Synonyms and Related Terms

It is important not to overlook this stage in the search process. Time spent identifying all possible synonyms and related terms for each of your PICO elements or concepts will ensure that your search retrieves as many relevant records as possible.

It might be useful to check relevant dictionaries, encyclopedias and key texts for alternate terms.

Build a list of each of the search terms you identify. For example, if you were searching for:

Exercise-based rehabilitation for coronary heart disease
Your list of synonyms and related terms might include:

3. Use Truncation and Wildcards

The definition of ‘truncation’ is to shorten or cut-off at the end. Truncation is used in database searches to ensure the retrieval of all possible variations of a search term.  All databases allow truncation, but the symbols used may vary, so it is best to check the database help for details.

Databases usually allow words to be truncated either at the end, or internally:

  • Truncating a word at the end ensures that all variations of the word, beginning with a specific root, will be retrieved. This is particularly useful for retrieving singular and plural versions of words.

Be careful not to truncate terms too early, or you may retrieve a high number of irrelevant documents.

Most databases use an asterisk (*) to find alternate endings for terms. For example:

  • Truncating a word internally ensures that any variations of spelling of a word can be retrieved. For example, pediatrics or paedetrics

Internal truncation is available in some databases, allowing you to search for alternate spellings of words – extremely useful when searching for American and English spellings of words.

For example, using the OVID databases (MEDLINE, EMBASE, PsycINFO, etc), a question mark included within a word can designate zero or one character in that place:

4. Combining Terms

Boolean Operators

Boolean operators allow you to link terms together, either to widen a search or to exclude terms from your search results.

Proximity Operators

Proximity (sometimes called “adjacency”) searching is similar to using Boolean operators in that you are specifying relationships between 2 or more terms. This feature is available in Ovid, but not PubMed. However, proximity searching allows you to specify the proximity of words to each other.

Nesting

Nest search terms to control the logic of your search. For example:

(rehabilitation OR exercise OR exercise therapy OR sports OR physical training) AND (Coronary heart bypass OR myocardial ischemia OR myocardial infarction OR coronary disease OR coronary thrombosis)

 

5. Identify Search Limits/Exclusion Criteria

You should next think about the limits you intend to apply to your search.

Criteria Questions to Ask
 Advice from the Cochrane Handbook for Systematic Reviews of Interventions  (2008, p. 134)
Time Period Will your review be restricted by year of publication, or is it important that you cover all years? “Date restrictions should be applied only if it is known that relevant studies could only have been reported during a specific time period, for example if the intervention was only available after a certain time point.”
Language Should you restrict to English language publications only? “Whenever possible review authors should attempt to identify and assess for eligibility all possibly relevant reports of trials irrespective of language of publication. No language restrictions should be included in the search strategy.”
Publication Type Are you restricting your search by publication type? “Format restrictions such as excluding letters are not recommended because letters may contain important additional information relating to an earlier trial report or new information about a trial not reported elsewhere.”
Geographic Considerations Are there any geographic considerations to include in your search strategy? For example, if you were researching Chinese herbal medicine you would need to consult Chinese literature.

6. Keyword vs Subject Searching

Controlled vocabularies (such as the MESH subject headings used in Medline and EMTREE subject headings used in EMBASE) provide an organized approach to the way knowledge is described.

Their use is extremely important as they bring uniformity to the indexing of publications included within a database. Using the same terminology throughout a database creates consistency and precision and helps you to find relevant information no matter what terminology the author may have used within their publication.

Indexing is usually a manual process. Databases such as MEDLINE employ specially trained indexers to read the full-text of each publication then identify all of the concepts covered within the article. These concepts are then translated to the controlled vocabulary used within the database. It is the indexer’s job to ensure that each concept included in the article are identified and assigned a term.

Each database may use different subject headings to describe the same concept. As an example, the term “complementary medicine”:

The OVID (MEDLINE), Embase.com, and CINAHL databases provide a search option to “explode” terms. PubMed automatically explodes terms, although there is the option of choosing not to explode a term. Exploded searches retrieve indexed records for a term, plus other terms which are a derivative (more specific, narrower terms) of the search term. Exploding search terms provides a fast way to find related concepts in a single search.

For example, if a search for “complementary therapies” in MEDLINE was exploded:

The search results would also include records  indexed with the MESH headings acupuncture”, “anthroposophy”, “auriculotherapy”, and so on.

NOTE: Clicking on a MESH heading will display its tree, including the exploded terms.

Keyword searches are extremely important when conducting systematic reviews, and should be used in combination with the relevant subject headings within each of your database searches:

Where to Search

Where to Search: Contents

Bibliographic Databases: When searching for a systematic review it is important that you search across a range of databases, as no one database covers all the health-related literature. The decision regarding which databases to search depends largely upon the topic of the review. Searches of bibliographic databases usually identify the bulk of the literature to be reviewed.

Grey Literature: Includes includes a wide range of resources. Grey lit may be ephemeral (that is, of questionable relevance or quality), but it continues to have an impact.

Handsearching: Involves page-by-page examination of relevant journal issues, conference proceedings and other publications for relevant studies.

Citation Indexes: Track references authors include in the reference lists of their publications.

Sensitivity vs Precision

You should aim to be as extensive as possible when conducting searches for systematic reviews. However, it may be necessary to strike a balance between the sensitivity and precision of your search.

Sensitivity – the number of relevant results identified divided by the total number of relevant results in existence

Precision – the number of relevant results identified divided by the total number of results identified.

Increasing the comprehensiveness of a search will reduce its precision and will retrieve more non-relevant results. However,

… at a conservatively-estimated reading rate of two abstracts per minute, the results of a   database search can be ‘scanread’ at the rate of 120 per hour (or approximately 1000  over an 8-hour   period), so the high yield and low precision associated with systematic review searching is not as   daunting as it might at first appear in comparison with the total  time to be invested in the review. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 130)

Bibliographic Databases

Search Tip

Searching the PubMed, Embase, Ovid MEDLINE. and CINAHL databases is a little different than many other databases.

Follow these tips:

  • Search for one term at a time
  • Map terms to subject headings (for Ovid MEDLINE, Embase, and CINAHL)
  • Read the Scope Notes
  • Look at the Subject Headings Tree
  • When in doubt, Explode but do not Focus
  • Use Keywords when necessary
  • Combine terms with Boolean Operators
  • Limit/filter results as required
 Search Filters

The following sites include examples of pre-tested search filters:

Cochrane Handbook – Chapter 6 (Search Filters 6.4.11) filter for identifying RCTs

ISSG Search Filters Resource

SIGN – Scottish Intercollegiate Guidelines Network

Tufts University – Useful MeSH for Evidence-Based Medicine

McMaster University – HEDGES Project – including MEDLINE, EMBASE, and PsycINFO

Centre for Reviews and Dissemination – Search Filter Resource

NCBI – Clinical Queries Table

PubMed Search Strategies Blog – “This blog has been created to share PubMed search strategies. Search strategies posted here are not perfect. They are posted in the hope that others will benefit from the work already put into their creation and/or will offer suggestions for improvements.”

Ovid MEDLINE Expert Searches

Ovid Expert Filters

Palliative Care PubMed Searches – from CareSearch

Filters for Aboriginal & Torres Strait Islander health literature from Lowitja Institute.

 

Global Index Medicus — from the World Health Organization: provides access to regional medical indexes, such as LILACS, WPRO, and other regional medical indexes.

Sensitivity vs Precision

You should aim to be as extensive as possible when conducting searches for systematic reviews. However, it may be necessary to strike a balance between the sensitivity and precision of your search.

Sensitivity – the number of relevant results identified divided by the total number of relevant results in existence

Precision – the number of relevant results identified divided by the total number of results identified.

Increasing the comprehensiveness of a search will reduce its precision and will retrieve more non-relevant results. However,

… at a conservatively-estimated reading rate of two abstracts per minute, the results of a   database search can be ‘scanread’ at the rate of 120 per hour (or approximately 1000  over an 8-hour   period), so the high yield and low precision associated with systematic review searching is not as   daunting as it might at first appear in comparison with the total  time to be invested in the review. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 130)

Bibliographic Databases

When searching for a systematic review it is important that you search across a range of databases, as no one database covers all the health-related literature. The decision regarding which databases to search depends largely upon the topic of the review.

Core Databases

Other Databases of Interest

Medline

Cochrane Library

Embase

Joanna Briggs Institute (JBI) Evidence
Based Practice Database

 

 CINAHL Complete: Cumulative Index to Nursing and Allied Health Literature [via EBSCO]

PsycINFO [via Proquest]

SPORTDiscus [via EBSCO]

Web of Science Core Collection [via Web of Science]

 

For a complete list of University of California Davis databases search by subject or title with our Databases search function.

Core Databases

The following databases should be included in all health-related systematic review searches.

Medline

 Coverage of biomedical literature from 1948 to present
Updated daily
Indexes over 5,400 journals from over 80 countries in 37 languages
Abstracts included for approx 75% of records
What’s the Difference between Medline and PubMed?
Coverage of biomedical literature from 1947 to present
Updated daily
Indexes 5,000 journals from over 60 countries (60% content overlap with Medline)
More indexing of European and Asian journals
Premier database for locating meta-analyses and systematic reviews produced by the Cochrane Collaboration
Updated quarterly
Ability to search for non-Cochrane reviews via DARE – DARE records link to MEDLINE Abstracts
All Cochrane Reviews are full-text
Systematic Reviews, Evidence Summaries, Evidence-Based Recommended Practices, Best Practice Information Sheets, Consumer Information Sheets and other resources primarily in Nursing and Allied Health, although medical topics are also featured.
JBI collaborates internationally with entities (including centers and other groups) located across the world.

 

Other Databases

Most of these databases require you to be logged into the campus network. You can log in with your Kerberos username and passphrase using the “Link to this LibGuide in VPN” in the left hand column.

  • CINAHL Complete (Cumulative Index to Nursing & Allied Health Literature)
    References and abstracts on nursing, biomedical, allied health and consumer health literature. Also includes health care books, nursing dissertations, selected conference proceedings, standards of practice, educational software, audiovisuals and book chapters, as well as Evidence-Based care sheets.
    Coverage: 1982-
  • SportDiscus [via EBSCO]
    PLEASE NOTE: only 4 simultaneous user access — please EXIT when finished.SPORTDiscus is the world’s leading database in sport, health, fitness and sports medicine. SportDiscus has over 1.5 million qualified references from thousands of international periodicals, books, e-journals, conference proceedings, theses, dissertations, and websites that focus on different areas involving sport, including: sport sciences, engineering, psychology, administration, sociology, history, coaching, training, physical education, physical fitness, health and recreation.
    Coverage: 1830-Present
  • Web of Science [via Web of Knowledge]
    Provides access to the content from three Thomson Reuters ISI (Institute for Scientific Information) Citation Databases: Science Citation Index Expanded; Social Sciences Citation Index; Arts & Humanities Citation Index.Coverage: 1900-present [Science 1899-, Social Sciences 1900-, Arts & Humanities 1975-]
  • Google Scholar
    Google Scholar searches for scholarly literature on the web, including “peer-reviewed papers, theses, books, preprints, abstracts and technical reports..from a variety of academic publishers, professional societies, preprint repositories and universities.” Often articles cited in Google Scholar results are available to UC Davis users through electronic journal and database subscriptions that the Library has licensed and paid for.
  • PsycINFO [via Proquest]
    PsycINFO is a department of the American Psychological Association (APA). It provides citations to articles in professional journals, conference proceedings, books, reports, dissertations and even important internet sites in psychology and related disciplines, most with abstracts and some citations.Coverage: 1840-present
  • PAIS International Database [via Proquest]
    Indexes all types of materials in the area of public affairs and public policy – journal articles, books and book chapters, reports, government documents. Topics include agriculture, banking, demographics, education, environment, finance, government, health, law, legislation, political science, social sciences, and statistics.
    Coverage: 1972-present
  • Global Health [via OvidSP]
    “Global Health brings together the resources of two internationally renowned databases – the Public Health and Tropical Medicine (PHTM) database, previously produced by the Bureau of Hygiene and Tropical Diseases (BHTD), and the human health and diseases information extracted from CAB ABSTRACTS. Global Health provides an alternative, complementary point of reference with a broad analysis of foreign language journals, books, research reports, patents and standards, dissertations, conference proceedings, annual reports, public health, developing country information, and other difficult to obtain material.”
  • ProQuest Statistical Insight
    ProQuest Statistical Insight combines Index and abstract records previously available from print indexes: American Statistics Index (ASI) from 1973, and Index to International Statistics (IIS)from 1980. ProQuest Statistical Insight enables searching indexes and tables for statistics produced by the U.S. government and major international organizations. The database includes some statistical tables as well as abstracts, or summaries, of statistical publications. The Library has many of the publications indexed in print or electronic format.

Grey Literature

Reporting Bias

Types of reporting bias include:

  • Publication bias
  • Time lag bias
  • Multiple (duplicate) publication bias
  • Location bias
  • Citation bias
  • Language bias
  • Outcome reporting bias.

For details see: Cochrane Handbook for Systematic Reviews of Interventions (2008, Chapter 10)

What is Grey Literature?

Grey literature is “information produced  by all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing”. (GreyNet)

Grey literature includes:

Theses and dissertations, conference proceedings, newsletters, reports, government documents, informal communications, translations, census data, research reports, technical reports, standards, patents, videos, clinical trials and practice guidelines, eprints, preprints, wiki articles, emails, blogs, listserv archives, repository content.

Grey literature may present a number of challenges for the researcher:

  • Difficult to find
  • Many documents are not web published
  • Older documents may not be archived
  • Short print runs may lead to lack of availability
  • Format and citation information may be inconsistent
  • Volume of material may be over-whelming and consuming
  • Quality of evidence varies.

 

Clinical Trials Registers
  • Cochrane Central Register of Clinical Trials (CENTRAL) – Details of published articles taken from bibliographic databases (notably MEDLINE and EMBASE), and other published and unpublished sources. In addition, each Cochrane Review Group maintains and updates a collection of controlled trials relevant to its own area of interest.
  • Clinical trials.gov  – Registry of federally and privately supported clinical trials conducted in the United States and around the world. Includes details about a trial’s purpose, who may participate, locations, and phone numbers for more details.
  • Current Controlled Trials – Current Controlled Trials allows users to search, register and share information about randomised controlled trials. Access to all the information on this site is free; charges for the registration services offered by Current Controlled.
  • CenterWatch  – More than 875 clinical research centers and 600 companies offering a variety of services to the clinical trials community are profiled right here on www.centerwatch.com! Locate research centers by medical expertise or geographic region and industry providers by type of services provided. Both profiles offer detailed information about the services they provide.
  • NCIC Clinical Trials Group  – the NCIC Clinical Trials Group is a cooperative oncology group which carries out clinical trials in cancer therapy, supportive care and prevention across Canada and internationally.
  • MRC Clinical Trials Unit – Based in the UK, the MRC Clinical Trials Unit runs trials in a range of disease areas.  We specialize in trials in HIV/AIDS and cancer, but we also run trials in other areas, for example arthritis and tuberculosis.
  • NIHR Clinical Research Network – The Clinical Research Network is part of the National Institute for Health Research within the National Health Service of the United Kingdom.
  • WHO International Clinical Trials Registry Platform – Provides access to a central database containing the trial registration data sets provided by the registries from Australia, UK, USA, Japan, Korea, Brazil, China, Iran, Sri Lanka, Netherlands, Germany, Cuba, India. It also provides links to the full original records.
  • Cancer Trials – From Cancer Research UK
  • National Cancer Institute Clinical Trials – Search NCI’s list of 10,000+ clinical trials now accepting participants.
  • IFMA Clinical Trials Portal – Links to IFPMA member company websites as well as other commercial and government-sponsored websites containing information on clinical trials provided by pharmaceutical companies.

Check also the lists included in Cochrane Handbook for Systematic Reviews of Interventions (2008, p. 114-117)

Conference Presentations
Source What is included? How to search
BIOMED CENTRAL MEETINGS Includes meeting abstracts published in BioMed Central Journals, usually includes abstracts and conference name. Browse list of journals indexing conferences.
BIOSIS PREVIEWS Includes citations to individual poster/paper abstracts – rarely with abstract. May include citations to conferences as a whole (without listing individual presentations) Limit search by “Publication Type” to:

  • Meeting
  • Meeting abstract
  • Meeting Paper
EMBASE Embase contains almost 800 conferences and more than 260,000 conference abstracts, primarily from journals and journal supplements published from 2009.

Summary of conferences indexed by Embase

 Limit search by “Publication Type” to:

  • Conference abstract
  • Conference paper
  • Conference review
  • Proceeding
MEDLINE Includes some citations to individual poster/paper abstracts from HIV/AIDS meetings – may include abstracts. Includes overall citations to conference proceedings when they are published as part of an indexed journal. Limit search by “Publication Type” to:

  • Clinical conference
  • Congresses

or MESH heading “Congresses as topic”
NLM MEETING ABSTRACTS NLM Meeting Abstracts is the National Library of Medicine’s online collection of abstracts from HIV/AIDS, Health Services Research, and Space Life Sciences meetings. Search, then follow the “Meeting Abstracts” link

 

Databases indexing Grey Literature
Source What is included?
Gray Literature Database From the New York Academy of Medicine: Items indexed may include, but are not limited to the following types of materials: fact sheets, technical reports, white papers, statistical reports, market research, workshop summaries, and dissertations.  Produced by foundations, think-tanks, advocacy groups, government agencies, and academic institutions, it often offers timely, statistical analysis for state-of-the-art research.
Google Scholar
 Articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites.
MedNar One-stop search across medical societies, NIH resources, other U.S. government websites, and patents.
OAIster Catalog of millions of records representing open access resources from open access collections worldwide. OAIster includes more than 25 million records representing digital resources from more than 1,100 contributors.
OpenDOAR Directory of academic open access repositories. Search for the full-text of material held in open access repositories listed in the Directory using ‘Search Repository Contents‘, or use OpenDOAR to find repositories or groups of repositories that fit particular needs using our ‘Find‘ facility.
OpenGrey Open access to 700.000 bibliographical references of grey literature (paper) produced in Europe and allows you to export records and locate the documents.
Science.gov Gateway to over 50 million pages of authoritative selected science information provided by U.S. government agencies, including research and development results
WorldWideScience.gov Global science gateway-accelerating scientific discovery and progress through a multilateral partnership to enable federated searching of national and international scientific databases.

 

Web Searching
Google Scholar Google Scholar provides a simple way to broadly search for scholarly literature. From one place, you can search across many disciplines and sources: articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. Google Scholar helps you find relevant work across the world of scholarly research.
MedNar Mednar is a free, publicly available deep web search engine that uses advanced federated search technology to return high quality results by submitting your search query – in real-time – to other well respected search engines. Mednar then collates, ranks and drops duplicates of the results.

Use Advanced Search to specify where to search (societies, government websites, etc)

 

Handsearching

Handsearching

Handsearching involves the page-by-page examination of relevant journal issues, conference proceedings and other publications for relevant studies. In addition, the checking of reference lists of journal articles and other documents retrieved from a search.

Why is handsearching important?

  1. Locates relevant items poorly indexed or not indexed at all. Some databases do not comprehensively index all content in journal issues,  or may not index at all supplements, special issues, or conference abstracts.
  2.  Allows researchers to scan content quickly for relevant studies from high-impact journals.
    3.Ensures that relevant studies are not overlooked. (HLWIKI Canada)

 

Cochrane Review – Handsearching Versus Electronic Searching to Identify Reports of Randomized Trials

Citation Indexes

Citation Indexes

Citation indexes track references authors include in the reference lists of their publications. They provide a means to search for and analyze the literature in a way not possible through simple keyword searching.

  • Web of Science includes the Science Citation Index; Social Sciences Citation Index; Arts & Humanities Citation Index
  • 1901+
  • Coverage of over 10,000 high-impact journals in the sciences, social sciences, and arts and humanities, as well as international proceedings coverage for over 120,000 conferences.
  • 54M records. 22,000 titles. 5,000 publishers
  • Find related documents by shared references, authors and/or keywords
  • 6.4 million conference papers

 

  • Google Scholar indexes articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites.

Documenting the Search Process

Documenting the Search Process

Systematic review searches must be documented in sufficient detail to allow others to be able to assess the thoroughness of the search. You will need to keep track of:

  • Each database searched, together with the platform searched via (eg. OVID Medline, OVID PsycINFO, CSA Sociological Abstracts)
  • The date each search were conducted
  • Subject headings and keywords used – including whether terms were exploded, truncated, etc
  • Search history, including the combination of terms
  • Number of results retrieved for each search and combination
  • Total number of records
  • Duplicates identified
  • Numbers pre-screening and post-screening

In addition, all searches conducted via handsearching must identify the source (name of journal, conference proceedings, etc), plus the years.

Guides to Documenting the Search Process

 

Cochrane Handbook for Systematic Reviews of Interventions

Section 6.6 – Documenting and reporting the search process
“The search process needs to be documented in enough detail throughout the process to ensure that it can be reported correctly in the review, to the extent that ll the searches of all the databases are reproducible” (Cochrane Handbook, 2008, p. 144)

.

PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.

The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. It is an evolving document that is subject to change periodically as new evidence emerges. In fact, the PRISMA Statement is an update and expansion of the now-out dated QUOROM Statement. This website contains the current definitive version of the PRISMA Statement.

Managing & Sharing Citations

Why save citations?

Citation management software will help you keep your references in order, making it easier when you are ready to share articles with your peers, write papers or articles, and integrate the citations you’ve collected during your research process.

While there are several different brands of citation management software, if sharing references with others is your goal, then you may want to select the software that your peers are also using.

 

Tools for managing citations: EndNote

The campus has a licensed subscription to EndNote, which is a bibliographic manager program that is available as software and as a web-based version. Many people use EndNote because of its compatibility with Microsoft Word. The libraries offer help and classes on EndNote.

  • EndNote: Getting Started Guide
    This guide, by librarian Cory Craig, provides detailed instructions about how to quickly gain proficiency with EndNote.
Tools for managing citations: Other programs

There are also a wide range of other citation management programs, including web-based platforms.

Two popular and useful ones are:

Here is a chart comparing various citation management options:

Comparison of Citation Management Software

 

 

Configure EndNote for UC-eLinks

To use UC-eLinks from within EndNote, make the following modification:

1.  Open EndNote.
2.  On the top menu, select:   Edit  | Preferences |  Find Full Text (tab)
or Mac version: EndNote  |  Preferences |  Find Full Text

3.  In the OpenURL Path text box, enter:  http://ucelinks.cdlib.org:8888/sfx_local

4.  Check all 4 boxes.
5.  Click OK.

How to use UC-eLinks from within EndNote:

1.  In your EndNote Library, highlight a citation.
2.  On the top menu select: References |  URL  | OpenURL Link to reach the UC-eLinks menu to locate the full text of the article.

 

Configure EndNote for Google Docs

How to use EndNote with Google Docs

1. Have your document open in Google Docs.
2. Have an EndNote library open.
3. Drag and drop citation to the place in the document where it is wanted. (It will appear in Google Docs document unformatted {Adler, 2006 #578})
4. When finished adding citations, download as a MS Word doc.
5. Open with MS Word if not already open.
6. In MS Word 2010, select EndNote menu so that EndNote commands are visible.
7. Select the style needed and click on ‘Update Citations and Bibliography’.

The crucial piece of this is that everyone adding citations to the Google Docs document needs to use the same EndNote library because part of the citation identification is the record number.

The library could be in Dropbox or on a server to which everyone has access. As long as everyone has the EndNote application, knows the location of the EndNote library being used, and has access to it, this should work.