alterlab-biorxiv

bioRxiv Database

Overview

Python tooling over the keyless bioRxiv API for searching and retrieving life-sciences preprints. Searches by keyword, author, date range, and category, returning structured JSON (titles, abstracts, DOIs, authors, versions), and downloads full-text PDFs.

For published, peer-reviewed literature use alterlab-pubmed; for computer-science / physics / math preprints use alterlab-arxiv. bioRxiv covers biology subjects only. The same API also serves medRxiv (swap biorxiv for medrxiv in the endpoint path, e.g. /details/medrxiv/2025-03-21/2025-03-28?category=cardiovascular%20medicine); the bundled script targets bioRxiv, so query medRxiv with curl/requests directly.

How it works (and its limits)

The bioRxiv /details endpoint returns preprints by date range, 30 records per page. It accepts a server-side subject-category filter (?category=cell_biology), but has no keyword or author filter. So this tool:

1. Paginates the date range (following the cursor until all records are retrieved), passing --category to the server so only that subject is fetched, then
2. Filters client-side by keyword (substring over title/abstract) and author (substring over the author list).

Implication: a wide date range means many API calls and a large download. Keep ranges as tight as the question allows, and use --category (cuts API calls) and --limit to bound the work.

When to Use This Skill

Use this skill when:

• Searching for recent life-sciences preprints in specific research areas
• Tracking preprints by particular authors
• Conducting systematic preprint literature reviews
• Analyzing preprint trends over time periods
• Retrieving metadata for citation management
• Downloading preprint PDFs for analysis
• Filtering papers by bioRxiv subject categories

Does NOT Trigger

Running the script

The script's only dependency is requests. Run it with uv so the dependency is provisioned on the fly:

uv run --with requests scripts/biorxiv_search.py --help

The python scripts/biorxiv_search.py ... invocations below are shorthand; substitute uv run --with requests scripts/biorxiv_search.py ... (or activate an environment that has requests).

Core Search Capabilities

1. Keyword Search

Search for preprints containing specific keywords in titles, abstracts, or author lists.

Basic Usage:

python scripts/biorxiv_search.py \
  --keywords "CRISPR" "gene editing" \
  --start-date 2024-01-01 \
  --end-date 2024-12-31 \
  --output results.json

With Category Filter:

python scripts/biorxiv_search.py \
  --keywords "neural networks" "deep learning" \
  --days-back 180 \
  --category neuroscience \
  --output recent_neuroscience.json

Search Fields: Keyword matching is a case-insensitive substring match, and a paper matches if any keyword is found (OR semantics, not AND). By default keywords are searched in both title and abstract. Customize with --search-fields:

python scripts/biorxiv_search.py \
  --keywords "AlphaFold" \
  --search-fields title \
  --days-back 365

2. Author Search

Find all papers by a specific author within a date range.

Basic Usage:

python scripts/biorxiv_search.py \
  --author "Smith" \
  --start-date 2023-01-01 \
  --end-date 2024-12-31 \
  --output smith_papers.json

Recent Publications:

# Last year by default if no dates specified
python scripts/biorxiv_search.py \
  --author "Johnson" \
  --output johnson_recent.json

3. Date Range Search

Retrieve all preprints posted within a specific date range.

Basic Usage:

python scripts/biorxiv_search.py \
  --start-date 2024-01-01 \
  --end-date 2024-01-31 \
  --output january_2024.json

With Category Filter:

python scripts/biorxiv_search.py \
  --start-date 2024-06-01 \
  --end-date 2024-06-30 \
  --category genomics \
  --output genomics_june.json

Days Back Shortcut:

# Last 30 days
python scripts/biorxiv_search.py \
  --days-back 30 \
  --output last_month.json

4. Paper Details by DOI

Retrieve detailed metadata for a specific preprint. bioRxiv DOIs come in two prefixes: 10.1101/… for older preprints and 10.64898/… for preprints posted since the move to openRxiv (December 2025), e.g. 10.64898/2026.08.28.747819. Both work with /details/ and the www.biorxiv.org/content/ URLs; the script normalizes DOIs, doi.org links, and content URLs regardless of prefix. Don't hard-code 10.1101 in regexes or validators.

Basic Usage:

python scripts/biorxiv_search.py \
  --doi "10.1101/2024.01.15.123456" \
  --output paper_details.json

Full DOI URLs Accepted (either prefix):

python scripts/biorxiv_search.py \
  --doi "https://doi.org/10.64898/2026.08.28.747819"

5. PDF Downloads

Download the full-text PDF of any preprint.

Basic Usage:

python scripts/biorxiv_search.py \
  --doi "10.1101/2024.01.15.123456" \
  --download-pdf paper.pdf

Batch Processing: For multiple PDFs, extract DOIs from a search result JSON and download each paper:

import json
from biorxiv_search import BioRxivSearcher

# Load search results
with open('results.json') as f:
    data = json.load(f)

searcher = BioRxivSearcher(verbose=True)

# Download each paper
for i, paper in enumerate(data['results'][:10]):  # First 10 papers
    doi = paper['doi']
    searcher.download_pdf(doi, f"papers/paper_{i+1}.pdf")

Valid Categories

Filter searches by bioRxiv subject categories:

• animal-behavior-and-cognition
• biochemistry
• bioengineering
• bioinformatics
• biophysics
• cancer-biology
• cell-biology
• clinical-trials
• developmental-biology
• ecology
• epidemiology
• evolutionary-biology
• genetics
• genomics
• immunology
• microbiology
• molecular-biology
• neuroscience
• paleontology
• pathology
• pharmacology-and-toxicology
• physiology
• plant-biology
• scientific-communication-and-education
• synthetic-biology
• systems-biology
• zoology

Output Format

All searches return structured JSON with the following format:

{
  "query": {
    "keywords": ["CRISPR"],
    "start_date": "2024-01-01",
    "end_date": "2024-12-31",
    "category": "genomics"
  },
  "result_count": 42,
  "results": [
    {
      "doi": "10.1101/2024.01.15.123456",
      "title": "Paper Title Here",
      "authors": "Smith, J.; Doe, J.; Johnson, A.",
      "author_corresponding": "Smith J",
      "author_corresponding_institution": "University Example",
      "date": "2024-01-15",
      "version": "1",
      "type": "new results",
      "license": "cc_by",
      "category": "genomics",
      "abstract": "Full abstract text...",
      "pdf_url": "https://www.biorxiv.org/content/10.1101/2024.01.15.123456v1.full.pdf",
      "html_url": "https://www.biorxiv.org/content/10.1101/2024.01.15.123456v1",
      "jatsxml": "https://www.biorxiv.org/content/...",
      "published": ""
    }
  ]
}

Common Usage Patterns

Literature Review Workflow

1. Broad keyword search:

python scripts/biorxiv_search.py \
  --keywords "organoids" "tissue engineering" \
  --start-date 2023-01-01 \
  --end-date 2024-12-31 \
  --category bioengineering \
  --output organoid_papers.json

2. Extract and review results:

import json

with open('organoid_papers.json') as f:
    data = json.load(f)

print(f"Found {data['result_count']} papers")

for paper in data['results'][:5]:
    print(f"\nTitle: {paper['title']}")
    print(f"Authors: {paper['authors']}")
    print(f"Date: {paper['date']}")
    print(f"DOI: {paper['doi']}")

3. Download selected papers:

from biorxiv_search import BioRxivSearcher

searcher = BioRxivSearcher()
selected_dois = ["10.1101/2024.01.15.123456", "10.1101/2024.02.20.789012"]

for doi in selected_dois:
    filename = doi.replace("/", "_").replace(".", "_") + ".pdf"
    searcher.download_pdf(doi, f"papers/{filename}")

Trend Analysis

Track research trends by analyzing publication frequencies over time:

python scripts/biorxiv_search.py \
  --keywords "machine learning" \
  --start-date 2020-01-01 \
  --end-date 2024-12-31 \
  --category bioinformatics \
  --output ml_trends.json

Then analyze the temporal distribution in the results.

Author Tracking

Monitor specific researchers' preprints:

# Track multiple authors (each run scans the whole window, so keep it short)
for author in Smith Johnson Williams; do
  python scripts/biorxiv_search.py \
    --author "$author" \
    --days-back 365 \
    --output "${author}_papers.json"
done

Python API Usage

For more complex workflows, import and use the BioRxivSearcher class directly:

from scripts.biorxiv_search import BioRxivSearcher

# Initialize
searcher = BioRxivSearcher(verbose=True)

# Multiple search operations
keywords_papers = searcher.search_by_keywords(
    keywords=["CRISPR", "gene editing"],
    start_date="2024-01-01",
    end_date="2024-12-31",
    category="genomics"
)

author_papers = searcher.search_by_author(
    author_name="Smith",
    start_date="2023-01-01",
    end_date="2024-12-31"
)

# Get specific paper details
paper = searcher.get_paper_details("10.1101/2024.01.15.123456")

# Download PDF
success = searcher.download_pdf(
    doi="10.1101/2024.01.15.123456",
    output_path="paper.pdf

---

*Content truncated.*