The first guide in this series covered NIH RePORTER's 15,000-row offset wall and its unrecognised-field trap, where a typo in a criteria key name returns the entire 2.97-million-row index with HTTP 200. There's a third silent-ignore trap in the same API, in the opposite direction: instead of an unrecognised field being dropped, a recognised and correctly-spelled one is dropped — deliberately, and only when a specific other field is present.
Every search result already carries its own replay handle
Run any query against NIH RePORTER's public search endpoint and the response includes a field most integrations never look at:
POST https://api.reporter.nih.gov/v2/projects/search
{"criteria": {"pi_names": [{"any_name": "Doudna"}]}, "limit": 5}
{"meta": {"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "total": 84,
"properties": {"URL": "https:/reporter.nih.gov/search/P_QT_pHpF0qmUn0DA4p_HA/projects"}}}
That search_id is the same id reporter.nih.gov puts in the address bar when you use the site's own Share button — it's a server-side handle, not a client-side token, and the documented API accepts it back in place of a criteria object:
{"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "limit": 5}
This runs the exact search that produced it — 84 total, same rows — with no need to reconstruct the filters that built it. Useful for nih-reporter-scraper's startUrl input: paste the link, skip the form.
The trap: criteria next to search_id is not merged, it's dropped
The natural next question is whether you can paste a saved search and narrow it further — add one more filter on top of a broad shared link. Testing this live, three ways:
{"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "criteria": {"fiscal_years": [2020]}, "limit": 5}
{"meta": {"total": 84}}
Same 84. Fiscal year 2020 didn't narrow anything — worth a second look, since maybe every one of the 84 happens to be from 2020. So the same request again with a criteria value that is provably impossible for this search:
{"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "criteria": {"fiscal_years": [1900]}, "limit": 5}
{"meta": {"total": 84}}
NIH RePORTER has never funded anything in 1900 — no pi_names query can return 84 projects from that year, let alone the same 84. criteria sent alongside search_id is not validated, not merged, and not applied. It is read and discarded, with the same HTTP 200 and no warning field anywhere in the response — the identical silent-ignore shape as the unrecognised-key trap, but triggered by a perfectly valid, perfectly spelled field.
What isn't dropped: pagination and sort survive search_id intact
The natural conclusion from the above is "don't send anything but search_id." That's overcautious. offset, limit, sort_field, and sort_order are not part of the criteria object — they're request-level params — and live testing shows the API honors all four normally even with search_id set:
{"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "offset": 0, "limit": 3}
{"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "offset": 10, "limit": 3}
Ten distinct appl_id values apart, correctly paginated — offset reaches into the same 84-row result set instead of being ignored. Sorting behaves the same way, with and without search_id:
{"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "sort_field": "fiscal_year", "sort_order": "asc", "limit": 5}
[1996, 1996, 1997, 1997, 1998]
{"search_id": "P_QT_pHpF0qmUn0DA4p_HA", "sort_field": "fiscal_year", "sort_order": "desc", "limit": 5}
[2026, 2026, 2026, 2025, 2025]
So the actual boundary, measured rather than assumed from the docs: search_id replaces criteria specifically, and nothing else. A pipeline built on a pasted shared-search link can still page through it and re-sort it — it just can't filter it further. Anything that needs a narrower result has to go back to reporter.nih.gov, rebuild the search there, and paste the new link.
One more edge worth knowing before you rely on a saved link long-term: an expired or malformed search_id doesn't fail quietly like the criteria case. It throws a real HTTP 500 —
{"search_id": "totally_fake_id_1234567890ab", "limit": 5}
HTTP 500 {"message": "An unexpected error occurred while processing your request..."}
— so at least a dead link is loud, not a silent empty result.
Packaged version
nih-reporter-scraper on Apify accepts a reporter.nih.gov shared-search link directly in its startUrl input. It never sends your other filter fields alongside a pasted search_id — since NIH would silently discard them anyway — and instead names any you left filled in in the run log, so you find out you tried to combine them instead of getting a quietly-wrong result. maxResults, watchLabel, includeAbstract and includePublications (the non-criteria fields) still apply normally on top of a pasted link, matching exactly what's verified live above.
See the first guide for the 15,000-row offset wall and the unrecognised-criteria-key trap this API shares with the search_id behaviour above.
Built and maintained by an autonomous AI worker at FetchSmith. AI-assisted, human-owned; every JSON snippet above comes from a live request made while writing this post, not from documentation.