Health Sciences Libraries Systematic Review Service Team
Automating the PDF Retrieval Process
by Health Sciences Libraries Systematic Review Service Team – March 29, 2021
While there are a number of scenarios in which an individual may need to acquire multiple full text journal articles (i.e., PDFs) at one time, this guide was created in response to requests for the following scenarios:
Systematic Reviews
Researchers working on comprehensive evidence syntheses such as Systematic Reviews often need several hundred PDFs related to their project in a short time span so that they can complete screening at the full text review stage while adhering to project timelines. In such situations, it is onerous and inefficient to individually look up each citation housed in their reference management tool, retrieve the PDF, and add the PDF back to the reference management tool and/or Systematic Review software (e.g., Covidence).
Supporting Frontline Clinical Care
Health System Administrators collaborate with clinicians, researchers, and librarians in developing in-house policies, clinical pathways, and critically appraised topics, which require comprehensive literature reviews and shared access to the corpus of literature.
Staying Current
Many researchers find it beneficial to work with a librarian in developing automated searches for newly published articles in specific journals. New citations are imported into a reference management tool and PDFs are retrieved and uploaded. This is an alternate approach to reviewing the table of contents on a journal’s website. It is more streamlined in that it mimics how someone might approach reviewing material in a print journal.
Journal Clubs
Some instructors find it useful to create bibliographies of references that will be presented, discussed, and critically appraised in a journal club. It is useful to be able to create a library of these references with attached PDFs that may need to be accessible on a mobile device.
Text Mining
Researchers working on topical text analyses of the literature often need to retrieve hundreds of PDFs in developing their corpus. It is especially important to be aware of publisher agreements and licensing when using full text articles for this purpose.
Summary
This guide gives an overview of how to automate download of PDFs showcasing two tools (EndNote and Zotero) that work fairly well for this purpose.
Finding Full Text with EndNote Desktop
EndNote Desktop, available for purchase through UC Davis’ On TheHub, has the ability to search for full text journal articles and conference papers which you have access to through the UC Davis library subscriptions and/or that are freely available on the Internet to attach to your existing references.
To Setup:
- Click on Edit > Preferences (or EndNote X_ > Preferences on Mac)
- Select “Find Full Text” in the left-hand column
- Check the “PubMed LinkOut” box (all boxes will now be checked)
- Insert the following OpenURL Path: https://search.library.ucdavis.edu/openurl/01UCD/01UCD_SP?
- Click “Apply”
- Select “PDF Handling” in the left-hand column
- Check the “Custom” box under PDF Auto Renaming Options
- Choose “Record Number”, then “Author”, then “Year”
- Click “OK”
To run “Find Full Text” in EndNote:
- Turn on the UC Davis Library VPN (if not connected to the campus network)
- Select (highlight) the references you would like EndNote to try and find PDFs for (limited to 250 records at a time)
- Click References > Find Full Text > Find Full Text
- EndNote Find Full Text copyright and usage notice will pop up
- Click OK
- You will see the progress of the search on the left (in the Groups pane) at the bottom
While this feature works quite well, it is not perfect and may not find everything!
To create a folder of PDFs from EndNote:
- Create a new, empty folder on your Desktop
- Open your EndNote Library’s .Data file
- Open the folder labeled PDF (EndNote automatically stores PDFs in individual folders within the .Data file)
- Search for type:.pdf
- Select all PDFs
- Copy & Paste selected PDFs into your new folder
Please refer EndNote’s Advanced Citation Management Guide for videos and additional information.
Finding Full Text with Zotero
Zotero, freely available through Zotero.org, has the ability to search for full text journal articles and conference papers which you have access to through the UC Davis library subscriptions and/or that are freely available on the Internet to attach to your existing references. It can also automatically retrieve PDFs.
To run “Find Available PDFs” in Zotero:
- Turn on the UC Davis Library VPN (if not connected to the campus network)
- Select (highlight) the references for which you would like Zotero to try and find the PDFs
- Right-click those references and select “Find Available PDFs”
- You will see the progress of the search (green check for PDFs found, red “X” for those that are not)
While this feature works well, it is not perfect and may not find everything!
PDFs Not Auto-retrieving?
Reasons why “Full-Text” might not be found using EndNote or Zotero:
- Working from off campus and UC Davis Library VPN has not been activated
- Article does not have a DOI (Digital Object Identifier)
- Article is too old to have a PDF available
- Article is too new and the PDF link is not yet available
- Journal has embargoed current articles
- UC Davis Library does not have a subscription to the journal and there is no free access
- Too many users are already in a journal with limited access
- Publisher’s website or third-party aggregator does not support or allow this capability
- Journal handles letters, editorials, correspondence differently, making it difficult to find a citation
If an article wasn’t automatically retrieved, search your favorite database or the Library’s Catalog to find it. You can add individual PDFs to your reference management tool. If an article isn’t available online, you can request it through the Library’s Interlibrary Loan Service.