Bruce Abbott
Health Sciences Librarian
916-734-3529
Setting up custom PubMed (NCBI) filters can help identify relevant articles in search results. The PubMed with filters link offers an opportunity to see the benefits.
A systematic review is a summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies. (CEBM).
Meta analysis is a systematic review that uses quantitative methods to synthesize and summarize the results.
Reproduced from: Bettany-Saltikov, J. (2010). Learning how to undertake a systematic review: Part 1. Nursing Standard, 24(40): 47-55.
How to Perform a Systematic Literature Review: A Guide for Healthcare Researchers, Practitioners and Students Electronic Access
Beyond the Health Sciences
The EVIDENCE PYRAMID is often used to illustrate the development of evidence. At the base of the pyramid is animal research and laboratory studies – this is where ideas are first developed. As you progress up the pyramid the amount of information available decreases in volume, but increases in relevance to the clinical setting.
Meta Analysis – systematic review that uses quantitative methods to synthesize and summarize the results.
Systematic Review – summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.
Randomised Controlled Trial – Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.
Cohort Study – Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.
Case Control Study – study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.
Case Series – report on a series of patients with an outcome of interest. No control group is involved. (Definitions from CEBM)
Other Levels of Evidence
Joanna Briggs Institute – The JBI Model of Evidence Based Healthcare
Centre for Evidence Based Medicine (UK) – Levels of Evidence (March 2009)
Before starting work on a systematic review check to ensure that a systematic review does not already exist, or is currently under way.
A check of what systematic reviews already exist in your discipline may also provide a starting point for selecting a review topic.
The following databases extensively index the systematic review literature in their discipline.
General
Public Health
DoPHER (Database of Promoting Health Effectiveness Reviews)
DoPHER contains details of over 2,500 reviews of health promotion and public health effectiveness.
Health Evidence
Around 4000 Systematic reviews evaluating the effectiveness of public health interventions by McMaster University, Canada.
Agency for Healthcare Research and Quality (AHRQ),
Reports on common, costly medical conditions and new health care technologies and strategies.The Evidence-based Practice Centers (EPCs ) review all relevant scientific literature on a wide spectrum of clinical and health services topics. Also produce technical reports on methodological topics and other types of evidence synthesis-related reports.
Nursing & Allied Health
Joanna Briggs Institute Library
The Joanna Briggs Institute (JBI) Library is a repository for publications and information for policy makers, health professionals, health scientists and others with a practical or academic interest in evidence based healthcare. It includes:
– The JBI Database of Systematic Reviews and Implementation Reports;
– The JBI Database of Best Practice Information Sheets and Technical Reports; and
– The JBI Database of Rapid Appraisals of Published Papers (coming soon).
USDA Nutrition Evidence Systematic Review
USDA’s Nutrition Evidence Systematic Review (formerly Nutrition Evidence Library) specializes in conducting systematic reviews to inform Federal nutrition policy and programs.
OTSeeker
A database that contains abstracts of systematic reviews and randomised controlled trials relevant to occupational therapy.
PEDro
Physiotherapy Evidence Database (PEDro) is a free database of over 20,000 randomised trials, systematic reviews and clinical practice guidelines in physiotherapy.
PsycBITE
PsycBITE indexes systematic reviews, randomised controlled trials, case series on cognitive, behavioural and other treatments for psychological problems and issues occurring as a consequence of acquired brain impairment (ABI). These studies are rated for their methodological quality, evaluating various aspects of scientific rigour.
SpeechBite
SpeechBite is an evidence based practice initiative between The University of Sydney and Speech Pathology Australia. Studies on this database include Systematic Reviews (SR), Randomised Controlled Trials (RCT), Non-Randomised Controlled Trials (non-RCT), Case Series (CS), and Single Case Experimental Design (SCED).
Veterinary
Social Science
Campbell Collaboration
Campbell Collaboration (C2) helps people make well-informed decisions by preparing, maintaining and disseminating systematic reviews in education, crime and justice, and social welfare.
Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre)
The Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) is part of the Social Science Research Unit at the Institute of Education, University of London. EPPI develops systematic reviews and developing review methods in social science and public policy. They have a useful page on tools to automate the systematic review process:
http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=3677
What Works Clearinghouse
The What Works Clearinghouse (WWC) was established in 2002 as an initiative of the Institute for Education Sciences (IES) at the U.S. Department of Education. Summarize and compare the evidence of the effectiveness of interventions that address school or district’s needs.
The following databases index a range of publications and study designs, including systematic reviews.
CINAHL |
|
EMBASE |
A limit to meta analysis is also available from the drop down list. |
MEDLINE |
For meta analysis, select Meta Analysis from the |
PUBMED |
|
Types of reporting bias include:
For details see: Cochrane Handbook for Systematic Reviews of Interventions (2008, Chapter 10)
The following sites include examples of pre-tested search filters:
AHRQ – Making Healthcare Safer III Patient Safety Practices – a set of systematic reviews on patient safety issues including suggested search terms in each “Methods” section.
McMaster University – HEDGES Project – including MEDLINE, EMBASE, and PsycINFO
SIGN – Scottish Intercollegiate Guidelines Network
Centre for Reviews and Dissemination – Search Filter Resource
BestBETS Search Filter Resource
NCBI – Clinical Queries Table
PubMed Search Strategies Blog
“This blog has been created to share PubMed search strategies. Search strategies posted here are not perfect. They are posted in the hope that others will benefit from the work already put into their creation and/or will offer suggestions for improvements.”
The InterTASC Information Specialists’ Sub-Group Search Filter Resource
The InterTASC Information Specialists’ Sub-Group (ISSG) is the group of information professionals supporting research groups within England and Scotland providing technology assessments to the National Institute for Health and Clinical Excellence (NICE) and other associated Information Specialists.
OvidSP Medline Expert Searches
PubMed Special Queries Searches
Translating search strategies from one platform to another
Database Syntax Guide – from Cochrane–search syntax from multiple platforms
MEDLINE Transpose – translates searches from PubMed to Ovid MEDLINE and vice versa.
Embase Search Tips
PsycInfo Search Blog
PsycInfo Topics Guide – includes suggested search terms
Centre for Evidence-Based Medicine – Formulating Answerable Clinical Questions
Evidence-Based Medicine: How to Practice and Teach EBM
Call Number:
Carlson Health Sci Library WB 102 E933 2005
Blaisdell Medical Library WB 102 E933 2005
Searching is a critical part of conducting the systematic review, as errors in the search process potentially result in a biased or otherwise incomplete evidence base. Searches for systematic reviews need to be constructed to maximise recall and deal effectively with a number of potentially biasing factors. (McGowan, 2005, p. 75)
You should aim to be as extensive as possible when conducting searches for systematic reviews. However, it may be necessary to strike a balance between the sensitivity and precision of your search.
Sensitivity – the number of relevant results identified divided by the total number of relevant results in existence
Precision – the number of relevant results identified divided by the total number of results identified.
Increasing the comprehensiveness of a search will reduce its precision and will retrieve more non-relevant results. However,
… at a conservatively-estimated reading rate of two abstracts per minute, the results of a database search can be ‘scanread’ at the rate of 120 per hour (or approximately 1000 over an 8-hour period), so the high yield and low precision associated with systematic review searching is not as daunting as it might at first appear in comparison with the total time to be invested in the review. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 130)
A useful technique is to check the search strategies used in other systematic reviews for hints on terms and combinations to use. Several groups have also developed pre-tested search filters.
A technique often used in health research for formulating a clinical question is the PICO Model.
Using PICO, a clinical question will have 4 elements – Patient, Intervention, Comparison and Outcome.
The Cochrane Handbook for Systematic Reviews of Interventions includes the following factors to consider when developing criteria for your PICO elements.
Patient, Population or Problem |
|
Interventions and Comparisons |
|
Outcomes |
|
We have two resources which provide a PICO search interface (both require being on the campus network, including the VPN):
Ovid MEDLINE PICO search: http://access.ovid.com/custom/ucdavispico/
Embase: https://embase.com/a/#/picoSearch
It is important not to overlook this stage in the search process. Time spent identifying all possible synonyms and related terms for each of your PICO elements or concepts will ensure that your search retrieves as many relevant records as possible.
It might be useful to check relevant dictionaries, encyclopedias and key texts for alternate terms.
Build a list of each of the search terms you identify. For example, if you were searching for:
Exercise-based rehabilitation for coronary heart disease
Your list of synonyms and related terms might include:
The definition of ‘truncation’ is to shorten or cut-off at the end. Truncation is used in database searches to ensure the retrieval of all possible variations of a search term. All databases allow truncation, but the symbols used may vary, so it is best to check the database help for details.
Databases usually allow words to be truncated either at the end, or internally:
Be careful not to truncate terms too early, or you may retrieve a high number of irrelevant documents.
Most databases use an asterisk (*) to find alternate endings for terms. For example:
Internal truncation is available in some databases, allowing you to search for alternate spellings of words – extremely useful when searching for American and English spellings of words.
For example, using the OVID databases (MEDLINE, EMBASE, PsycINFO, etc), a question mark included within a word can designate zero or one character in that place:
Boolean Operators
Boolean operators allow you to link terms together, either to widen a search or to exclude terms from your search results.
Proximity Operators
Proximity (sometimes called “adjacency”) searching is similar to using Boolean operators in that you are specifying relationships between 2 or more terms. This feature is available in Ovid, but not PubMed. However, proximity searching allows you to specify the proximity of words to each other.
Nesting
Nest search terms to control the logic of your search. For example:
(rehabilitation OR exercise OR exercise therapy OR sports OR physical training) AND (Coronary heart bypass OR myocardial ischemia OR myocardial infarction OR coronary disease OR coronary thrombosis)
You should next think about the limits you intend to apply to your search.
Criteria | Questions to Ask |
Advice from the Cochrane Handbook for Systematic Reviews of Interventions |
Time Period | Will your review be restricted by year of publication, or is it important that you cover all years? | “Date restrictions should be applied only if it is known that relevant studies could only have been reported during a specific time period, for example if the intervention was only available after a certain time point.” |
Language | Should you restrict to English language publications only? | “Whenever possible review authors should attempt to identify and assess for eligibility all possibly relevant reports of trials irrespective of language of publication. No language restrictions should be included in the search strategy.” |
Publication Type | Are you restricting your search by publication type? | “Format restrictions such as excluding letters are not recommended because letters may contain important additional information relating to an earlier trial report or new information about a trial not reported elsewhere.” |
Geographic Considerations | Are there any geographic considerations to include in your search strategy? | For example, if you were researching Chinese herbal medicine you would need to consult Chinese literature. |
Controlled vocabularies (such as the MESH subject headings used in Medline and EMTREE subject headings used in EMBASE) provide an organized approach to the way knowledge is described.
Their use is extremely important as they bring uniformity to the indexing of publications included within a database. Using the same terminology throughout a database creates consistency and precision and helps you to find relevant information no matter what terminology the author may have used within their publication.
Indexing is usually a manual process. Databases such as MEDLINE employ specially trained indexers to read the full-text of each publication then identify all of the concepts covered within the article. These concepts are then translated to the controlled vocabulary used within the database. It is the indexer’s job to ensure that each concept included in the article are identified and assigned a term.
Each database may use different subject headings to describe the same concept. As an example, the term “complementary medicine”:
The OVID (MEDLINE), Embase.com, and CINAHL databases provide a search option to “explode” terms. PubMed automatically explodes terms, although there is the option of choosing not to explode a term. Exploded searches retrieve indexed records for a term, plus other terms which are a derivative (more specific, narrower terms) of the search term. Exploding search terms provides a fast way to find related concepts in a single search.
For example, if a search for “complementary therapies” in MEDLINE was exploded:
The search results would also include records indexed with the MESH headings acupuncture”, “anthroposophy”, “auriculotherapy”, and so on.
NOTE: Clicking on a MESH heading will display its tree, including the exploded terms.
Keyword searches are extremely important when conducting systematic reviews, and should be used in combination with the relevant subject headings within each of your database searches:
Bibliographic Databases: When searching for a systematic review it is important that you search across a range of databases, as no one database covers all the health-related literature. The decision regarding which databases to search depends largely upon the topic of the review. Searches of bibliographic databases usually identify the bulk of the literature to be reviewed.
Grey Literature: Includes includes a wide range of resources. Grey lit may be ephemeral (that is, of questionable relevance or quality), but it continues to have an impact.
Handsearching: Involves page-by-page examination of relevant journal issues, conference proceedings and other publications for relevant studies.
Citation Indexes: Track references authors include in the reference lists of their publications.
You should aim to be as extensive as possible when conducting searches for systematic reviews. However, it may be necessary to strike a balance between the sensitivity and precision of your search.
Sensitivity – the number of relevant results identified divided by the total number of relevant results in existence
Precision – the number of relevant results identified divided by the total number of results identified.
Increasing the comprehensiveness of a search will reduce its precision and will retrieve more non-relevant results. However,
… at a conservatively-estimated reading rate of two abstracts per minute, the results of a database search can be ‘scanread’ at the rate of 120 per hour (or approximately 1000 over an 8-hour period), so the high yield and low precision associated with systematic review searching is not as daunting as it might at first appear in comparison with the total time to be invested in the review. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 130)
Searching the PubMed, Embase, Ovid MEDLINE. and CINAHL databases is a little different than many other databases.
Follow these tips:
The following sites include examples of pre-tested search filters:
Cochrane Handbook – Chapter 6 (Search Filters 6.4.11) filter for identifying RCTs
SIGN – Scottish Intercollegiate Guidelines Network
McMaster University – HEDGES Project – including MEDLINE, EMBASE, and PsycINFO
Centre for Reviews and Dissemination – Search Filter Resource
NCBI – Clinical Queries Table
PubMed Search Strategies Blog – “This blog has been created to share PubMed search strategies. Search strategies posted here are not perfect. They are posted in the hope that others will benefit from the work already put into their creation and/or will offer suggestions for improvements.”
Palliative Care PubMed Searches – from CareSearch
Filters for Aboriginal & Torres Strait Islander health literature from Lowitja Institute.
Global Index Medicus — from the World Health Organization: provides access to regional medical indexes, such as LILACS, WPRO, and other regional medical indexes.
You should aim to be as extensive as possible when conducting searches for systematic reviews. However, it may be necessary to strike a balance between the sensitivity and precision of your search.
Sensitivity – the number of relevant results identified divided by the total number of relevant results in existence
Precision – the number of relevant results identified divided by the total number of results identified.
Increasing the comprehensiveness of a search will reduce its precision and will retrieve more non-relevant results. However,
… at a conservatively-estimated reading rate of two abstracts per minute, the results of a database search can be ‘scanread’ at the rate of 120 per hour (or approximately 1000 over an 8-hour period), so the high yield and low precision associated with systematic review searching is not as daunting as it might at first appear in comparison with the total time to be invested in the review. (Cochrane Handbook for Systematic Reviews of Interventions, 2008, p. 130)
Bibliographic Databases
When searching for a systematic review it is important that you search across a range of databases, as no one database covers all the health-related literature. The decision regarding which databases to search depends largely upon the topic of the review.
For a complete list of University of California Davis databases search by subject or title with our Databases search function.
The following databases should be included in all health-related systematic review searches.
Coverage of biomedical literature from 1948 to present Updated daily Indexes over 5,400 journals from over 80 countries in 37 languages Abstracts included for approx 75% of records What’s the Difference between Medline and PubMed? |
|
![]() |
Coverage of biomedical literature from 1947 to present Updated daily Indexes 5,000 journals from over 60 countries (60% content overlap with Medline) More indexing of European and Asian journals |
![]() |
Premier database for locating meta-analyses and systematic reviews produced by the Cochrane Collaboration Updated quarterly Ability to search for non-Cochrane reviews via DARE – DARE records link to MEDLINE Abstracts All Cochrane Reviews are full-text |
![]() |
Systematic Reviews, Evidence Summaries, Evidence-Based Recommended Practices, Best Practice Information Sheets, Consumer Information Sheets and other resources primarily in Nursing and Allied Health, although medical topics are also featured. JBI collaborates internationally with entities (including centers and other groups) located across the world. |
Most of these databases require you to be logged into the campus network. You can log in with your Kerberos username and passphrase using the “Link to this LibGuide in VPN” in the left hand column.
Types of reporting bias include:
For details see: Cochrane Handbook for Systematic Reviews of Interventions (2008, Chapter 10)
Grey literature is “information produced by all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing”. (GreyNet)
Grey literature includes:
Theses and dissertations, conference proceedings, newsletters, reports, government documents, informal communications, translations, census data, research reports, technical reports, standards, patents, videos, clinical trials and practice guidelines, eprints, preprints, wiki articles, emails, blogs, listserv archives, repository content.
Grey literature may present a number of challenges for the researcher:
Check also the lists included in Cochrane Handbook for Systematic Reviews of Interventions
Source | What is included? | How to search |
BIOMED CENTRAL Proceedings | Includes meeting abstracts published in BioMed Central Journals, usually includes abstracts and conference name. | Browse list of journals indexing conferences. |
BIOSIS PREVIEWS | Includes citations to individual poster/paper abstracts – rarely with abstract. May include citations to conferences as a whole (without listing individual presentations) | Limit search by “Publication Type” to:
|
EMBASE | Embase contains almost 800 conferences and more than 260,000 conference abstracts, primarily from journals and journal supplements published from 2009. | Limit search by “Publication Type” to:
|
MEDLINE | Includes some citations to individual poster/paper abstracts from HIV/AIDS meetings – may include abstracts. Includes overall citations to conference proceedings when they are published as part of an indexed journal. | Limit search by “Publication Type” to:
or MESH heading “Congresses as topic” |
NLM MEETING ABSTRACTS | NLM Meeting Abstracts is the National Library of Medicine’s online collection of abstracts from HIV/AIDS, Health Services Research, and Space Life Sciences meetings. | Search, then follow the “Meeting Abstracts” link |
Source | What is included? |
Gray Literature Database | From the New York Academy of Medicine: Items indexed may include, but are not limited to the following types of materials: fact sheets, technical reports, white papers, statistical reports, market research, workshop summaries, and dissertations. Produced by foundations, think-tanks, advocacy groups, government agencies, and academic institutions, it often offers timely, statistical analysis for state-of-the-art research. |
Google Scholar |
Articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. |
MedNar | One-stop search across medical societies, NIH resources, other U.S. government websites, and patents. |
OAIster | Catalog of millions of records representing open access resources from open access collections worldwide. OAIster includes more than 25 million records representing digital resources from more than 1,100 contributors. |
OpenDOAR | Directory of academic open access repositories. Search for the full-text of material held in open access repositories listed in the Directory using ‘Search Repository Contents‘. |
OpenGrey | Open access to 700.000 bibliographical references of grey literature (paper) produced in Europe and allows you to export records and locate the documents. |
Science.gov | Gateway to over 50 million pages of authoritative selected science information provided by U.S. government agencies, including research and development results |
WorldWideScience.gov | Global science gateway-accelerating scientific discovery and progress through a multilateral partnership to enable federated searching of national and international scientific databases. |
![]() |
Google Scholar provides a simple way to broadly search for scholarly literature. From one place, you can search across many disciplines and sources: articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. Google Scholar helps you find relevant work across the world of scholarly research. |
![]() |
Mednar is a free, publicly available deep web search engine that uses advanced federated search technology to return high quality results by submitting your search query – in real-time – to other well respected search engines. Mednar then collates, ranks and drops duplicates of the results.
Use Advanced Search to specify where to search (societies, government websites, etc) |
Handsearching involves the page-by-page examination of relevant journal issues, conference proceedings and other publications for relevant studies. In addition, the checking of reference lists of journal articles and other documents retrieved from a search.
Why is handsearching important?
Citation indexes track references authors include in the reference lists of their publications. They provide a means to search for and analyze the literature in a way not possible through simple keyword searching.
Systematic review searches must be documented in sufficient detail to allow others to be able to assess the thoroughness of the search. You will need to keep track of:
In addition, all searches conducted via handsearching must identify the source (name of journal, conference proceedings, etc), plus the years.
Cochrane Handbook for Systematic Reviews of Interventions
Section 6.6 – Documenting and reporting the search process
“The search process needs to be documented in enough detail throughout the process to ensure that it can be reported correctly in the review, to the extent that ll the searches of all the databases are reproducible” (Cochrane Handbook, 2008, p. 144)
.
PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.
The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram (both are downloadable files). It is an evolving document that is subject to change periodically as new evidence emerges. In fact, the PRISMA Statement is an update and expansion of the now-out dated QUOROM Statement.
Examples of Complex Search Strategy
Citation management software will help you keep your references in order, making it easier when you are ready to share articles with your peers, write papers or articles, and integrate the citations you’ve collected during your research process.
While there are several different brands of citation management software, if sharing references with others is your goal, then you may want to select the software that your peers are also using.
The campus has a licensed subscription to EndNote, which is a bibliographic manager program that is available as software and as a web-based version. Many people use EndNote because of its compatibility with Microsoft Word. The libraries offer help and classes on EndNote.
There are also a wide range of other citation management programs, including web-based platforms.
Two popular and useful ones are:
Here is a chart comparing various citation management options:
Comparison of Citation Management Software
To use UC-eLinks from within EndNote, make the following modification:
1. Open EndNote.
2. On the top menu, select: Edit | Preferences | Find Full Text (tab)
or Mac version: EndNote | Preferences | Find Full Text
3. In the OpenURL Path text box, enter: https://search.library.ucdavis.edu/openurl/01UCD_INST/01UCD_INST:UCD?
4. Check all 4 boxes.
5. Click OK.
How to use UC-eLinks from within EndNote:
1. In your EndNote Library, highlight a citation.
2. On the top menu select: References | URL | OpenURL Link to reach the UC-eLinks menu to locate the full text of the article.
How to use EndNote with Google Docs
1. Have your document open in Google Docs.
2. Have an EndNote library open.
3. Drag and drop citation to the place in the document where it is wanted. (It will appear in Google Docs document unformatted {Adler, 2006 #578})
4. When finished adding citations, download as a MS Word doc.
5. Open with MS Word if not already open.
6. In MS Word 2010, select EndNote menu so that EndNote commands are visible.
7. Select the style needed and click on ‘Update Citations and Bibliography’.
The crucial piece of this is that everyone adding citations to the Google Docs document needs to use the same EndNote library because part of the citation identification is the record number.
The library could be in Dropbox or on a server to which everyone has access. As long as everyone has the EndNote application, knows the location of the EndNote library being used, and has access to it, this should work.