Using the Semantic Scholar Academic Graph API: Batch Recipes and Real Rate Limits

There is no shortage of pages listing the Semantic Scholar Academic Graph API’s endpoints. The endpoint list is the easy part, and it is already in the official API reference. What breaks working scripts is none of it: it is the rate limiting, which is unevenly applied across endpoints, is not what the headline number suggests, and returns HTTP 500 as often as HTTP 429 when it bites.

This page is built the other way round. It gives you three worked recipes — resolving a set of DOIs, pulling citation contexts, and bulk-retrieving a whole result set — and for each one it states what Semantic Scholar documents, what actually happened when the requests were run, and where those two things disagree. Every response shown below is real and trimmed, not illustrative.

How to read this page

Three tiers, kept separate throughout, in the same way as CASRAI’s companion pages on what MeSH indexing in PubMed actually retrieves and which Google Scholar operators actually work:

1. Documented — stated in writing by Semantic Scholar, on the API product page, the API tutorial, or returned as an explicit error string by the API itself.
2. Measured — obtained by running the request against the live API on 26 August 2026, unauthenticated, from a single client. The shared unauthenticated pool is contended by every other anonymous caller in the world, so these numbers are a snapshot of a moving system. The ratios and the failure shapes are the durable finding; the exact counts will differ when you re-run them, and re-running them is a two-minute check.
3. Observed only — reproducible behaviour that Semantic Scholar has not specified in writing. Flagged as such rather than presented as a rule.

The three API surfaces, and which one you want

Semantic Scholar exposes its graph through three separate products, all free of charge:

• Academic Graph API (S2AG) — per-paper and per-author lookups, search, citations and references. Base URL https://api.semanticscholar.org/graph/v1. This is what the rest of this page is about.
• Recommendations API — papers similar to a given paper or set of papers, backed by SPECTER2 document embeddings.
• Datasets API — downloadable bulk snapshots of the whole graph. Semantic Scholar’s own tutorial names this as the answer when you need a higher request rate than an API key provides: “you can download Semantic Scholar’s datasets and run queries locally.”

The decision rule that follows from that sentence is the most useful thing on this page, so it goes near the top: if your job needs more than a few hundred thousand papers, stop trying to tune your API client and go get the dataset dump instead. The Academic Graph API is designed for targeted retrieval, not for corpus construction, and no amount of clever batching converts one into the other.

For background on what the underlying corpus is, where it came from, and how the site’s features are built on it, see CASRAI’s separate guide to what Semantic Scholar is and how it works. This page assumes you already want to call the API.

The API key situation, and the arithmetic that surprises people

Documented. The API product page states that most endpoints are usable without any authentication, but that they are:

“rate-limited to 1000 requests per second shared among all unauthenticated users. Requests may also be further throttled during periods of heavy use.”

And that:

“The introductory rate limit for an API key is 1 RPS on all endpoints.”

The tutorial page spells out the mechanism behind those two sentences, and it is the part worth reading twice:

“Users without API keys are affected by the traffic from all other unauthenticated users, who share a single API key. But using an individual API key automatically gives a user a 1 request per second rate across all endpoints. In some cases, users may be granted a slightly higher rate following a review.”

Read those literally and the arithmetic looks perverse: an API key appears to drop your ceiling from 1,000 requests per second to 1. That reading is wrong, and getting it right is the difference between a script that works and one that does not.

The 1,000 RPS figure is not a quota you hold. It is the total capacity of a single shared key that every anonymous caller on the internet is using simultaneously. Your share of it is whatever is left after everyone else, and it is neither reserved nor predictable. The 1 RPS an API key gives you is yours. A key does not raise a ceiling; it moves you out of a contended pool and into a guaranteed allocation. One reliable request per second beats an unpredictable slice of a thousand for every workload that has to finish.

How to get one. The request form is linked from the API product page as “Request an API Key”. The documented outcome is that “you will receive your private API key via email.” Semantic Scholar publishes no approval turnaround time, and the tutorial notes that a rate above the introductory 1 RPS is granted only “following a review” — so treat a higher allocation as something to ask for with a described use case, not as a setting. Send the key as an x-api-key request header.

One caveat on where you look for guidance: the community repository allenai/s2-folks, which many older tutorials still cite for rate-limit numbers, carries a notice on its own front page that it “is not maintained as of January 23, 2025.” Its error-code guidance remains sound; its numbers should not be treated as current. This is the specific reason the figures below were measured rather than quoted.

Measured: the limit is per-endpoint-class, not global

The single most useful thing an unauthenticated caller can know is that the shared pool does not throttle every endpoint equally. Six identical requests were fired back-to-back with no pacing at each of four endpoint types. Same client, same minute, no API key:

Endpoint	Method	200s out of 6	Observed status sequence
/paper/{id}	GET	6	200, 200, 200, 200, 200, 200
/paper/batch	POST	1	429, 200, 429, 429, 429, 429
/paper/search	GET	0	429, 429, 429, 500, 429, 429
/paper/search/bulk	GET	2	429, 500, 500, 500, 200, 200

Single-paper lookups were not throttled at all: a separate run of 24 sequential /paper/{id} requests — eight back-to-back with no delay, eight at one per second, eight at one per three seconds — returned 24 of 24 as HTTP 200, with per-request latency between 451 ms and 532 ms.

The search and batch endpoints, over the same period, were close to unusable without pacing. A /paper/search call wrapped in exponential backoff exhausted six retries at 2 s, 4 s, 8 s, 16 s, 32 s and 32 s and never got a 200.

Two rules follow directly, and both are things a naive script gets wrong:

• Budget your pacing against the endpoint you are actually calling. A retry policy tuned on /paper/{id} latency will fall apart the moment the same script touches /paper/search. These are not the same resource.
• Treat 5xx as a throttle signal, not a server fault. Under contention these endpoints returned HTTP 500 alongside 429 — four 500s across the runs above. Semantic Scholar’s own FAQ says the same thing in advance: “handle 5xxs gracefully, incorporating some exponential backoff retry algorithm in your code.” A retry loop keyed only on 429 will misclassify a throttle as a hard failure and abandon work that would have succeeded. This is the most common way a working script silently starts losing records.

Recipe 1: resolving a set of DOIs to metadata

The wrong way to resolve 500 DOIs is 500 GETs against /paper/DOI:{doi}. The right way is one POST to /paper/batch. It accepts an ids array in the body and the usual fields parameter in the query string:

curl -X POST 'https://api.semanticscholar.org/graph/v1/paper/batch?fields=title,year,externalIds,citationCount,influentialCitationCount,isOpenAccess' 
  -H 'Content-Type: application/json' 
  -d '{"ids":["DOI:10.1038/sdata.2016.18","DOI:10.1038/s41586-021-03819-2","DOI:10.1038/nature14539"]}'

Measured response, trimmed:

[
  {"paperId": "e936f248b2c0489316ed1521656af2564c3502c3",
   "externalIds": {"MAG":"3083522027","PubMedCentral":"4792175",
                   "DOI":"10.1038/sdata.2016.18","CorpusId":8755162,"PubMed":"26978244"},
   "title": "The FAIR Guiding Principles for scientific data management and stewardship",
   "year": 2016, "citationCount": 16212, "influentialCitationCount": 667, "isOpenAccess": true},
  {"paperId": "dc32a984b651256a8ec282be52310e6bd33d9815",
   "externalIds": {"PubMedCentral":"8371605","DOI":"10.1038/s41586-021-03819-2",
                   "CorpusId":235959867,"PubMed":"34265844"},
   "title": "Highly accurate protein structure prediction with AlphaFold",
   "year": 2021, "citationCount": 38454, "influentialCitationCount": 3972, "isOpenAccess": true},
  null
]

Five things in that response are worth building your code around.

The response is positional, and a miss is a bare null

Results come back in submitted order, with null in the slot of anything that did not resolve. There is no error, no message and no id echoed back — so you must zip the response array against your input array by index to know which DOI failed. Code that iterates the response alone loses that mapping permanently.

A null does not mean the DOI is invalid

The third id in that request, 10.1038/nature14539, is a real and current DOI. Resolved against the Crossref REST API on the same day it returns a journal article titled “Deep learning”, published in Nature in 2015. Semantic Scholar returns null for it anyway, and a targeted search of the S2 corpus for 2015 Nature items under that title did not surface it either.

This is the single most important correctness point on the page. An unresolved id is a coverage gap in Semantic Scholar, not a data-quality problem in your input. A pipeline that treats null as “bad DOI” will quietly discard valid records and, worse, will report a false error rate back to whoever supplied the list. Log unresolved ids as unresolved, and re-check them against Crossref or OpenAlex before drawing any conclusion about them.

Mixed identifier types work in one call

The ids array does not have to be homogeneous. A single batch containing DOI:, CorpusId:, arXiv:, PMID: and URL: prefixes resolved all five, correctly returning the same paperId for the three entries that pointed at the same paper by different identifiers. That makes /paper/batch a workable identifier-reconciliation step in its own right: hand it whatever mixture of ids your source systems hold and read paperId back as the join key. For the DOI-side of that work, see CASRAI’s guide to finding, looking up and resolving a DOI.

The hard cap is 500, enforced with a real error

Documented, by the API itself. Posting 501 ids returns HTTP 400:

{"error":"Maximum 500 ids allowed in input list"}

Nested fields are rejected on batch

The fields syntax that works on /paper/{id} is not fully available here. Requesting citations.contexts on a batch call returns HTTP 400:

{"error":"Unrecognized or unsupported fields: [citations.contexts]"}

Citation contexts have to come from the dedicated citations endpoint, which is the next recipe.

Recipe 2: DOI to citation contexts

Citation contexts are the reason to choose Semantic Scholar over the alternatives. A context is the actual sentence in the citing paper where the citation appears; an intent is a classification of why it was cited; isInfluential is a machine-learned flag for citations that materially shaped the citing work. None of the three exists in OpenAlex.

curl 'https://api.semanticscholar.org/graph/v1/paper/DOI:10.1038/sdata.2016.18/citations?fields=contexts,intents,isInfluential,title,year,externalIds&limit=100'

Measured response, one record, trimmed:

{
  "intents": [],
  "isInfluential": false,
  "contexts": [
    "…find and reuse, deposit sequence data and metadata in verified repositories and
     publish data/analysis code with persistent identifiers, open licenses, and clear
     documentation that meets findability, accessibility, interoperability, and
     reusability (FAIR) guiding principles (Wilkinson et al., 2016).",
    "To maintain trust, agree on communication routines and document where data go and
     why, and create public-facing data-management notes to build understanding and
     confidence (Thuermer et al., 2023; Wilkinson et al., 2016)."
  ],
  "citingPaper": {
    "paperId": "…",
    "title": "Engaging European local communities in biodiversity genomics research:
              A five-step framework for scientists",
    "year": 2026
  }
}

That is genuinely useful output: two verbatim sentences showing exactly how the cited work was used. But the fields are frequently empty, and how often they are empty depends on where in the result set you look — which is the finding most likely to mislead you.

Observed: the default ordering is newest-first, and it is the worst-populated end

Contexts and intents are derived from full text. Semantic Scholar’s own FAQ on influential citations states plainly that the classification “depends on Semantic Scholar having the citing paper’s full text.” The newest citing papers are the least likely to have been full-text ingested yet — and they are exactly what the default ordering returns first.

Sampling 100 citations at a time at increasing offsets, on the FAIR Guiding Principles paper:

Offset	Records with a context	Records with an intent	Publication year of citing papers
0	33 / 100	0 / 100	all 2026
1,000	37 / 100	0 / 100	all 2026
5,000	51 / 100	23 / 100	all 2024

The effect is larger still on a heavily-cited computer-science paper. On Attention Is All You Need, the first 100 citations returned 4 records with contexts; at offset 5,000, 84.

A script that pulls the first page, sees mostly empty contexts arrays and concludes the field is unpopulated is sampling the single worst slice of the distribution. If you need contexts, page deeper, or filter your citing set by publication year before you judge coverage.

Measured: intents are much sparser than contexts

Across five samples of 100 citation records each, intents were populated in a range from 0 to 23 per 100, and were absent entirely from every sample drawn at offsets 0 and 1,000. Contexts, over the same samples, ran between 4 and 84 per 100. Both fields are real and both are documented features; neither is dense enough to be treated as a complete annotation layer. Design any analysis that depends on them as a sample, and report the denominator.

The isInfluential flag is sparser again by design — it is meant to be selective. Across four papers, the first 100 citations contained 1, 4, 8 and 10 influential citations respectively.

Documented: you cannot page past 10,000 citations

There is a hard pagination ceiling, enforced with an explicit error. Requesting offset=12000&limit=100 returns HTTP 400:

{"error":"offset + limit must be < 10000"}

This matters more than it first appears. The FAIR Guiding Principles paper reports 16,212 citations; roughly 6,200 of them are simply not reachable through this endpoint at any offset. For any paper cited more than 10,000 times, the citations endpoint cannot give you a complete citation set, and any completeness claim built on it is wrong. That is a Datasets API job. For the analytical work this feeds, see CASRAI’s guide to citation network analysis, and the Scite smart citations entry for a commercial alternative built specifically around citation statements.

Observed only: sort had no effect on this endpoint

Adding sort=citationCount:desc to a /citations request returned a result set identical to the unsorted call, including the same first record. Treat the ordering of this endpoint as unspecified and do not build logic that assumes you can reorder it.

Recipe 3: bulk retrieval that stays inside the rate limits

This is the pattern that breaks naive scripts, and the fix is structural rather than a matter of tuning delays. There are two search endpoints and they behave completely differently.

• /paper/search is relevance-ranked, returns a small page, and was the most aggressively throttled endpoint measured — 0 successes in 6 unpaced attempts. It is for finding a paper, not for retrieving a set.
• /paper/search/bulk returns up to 1,000 records per call with a continuation token, and supports filters such as year. It is for retrieving a set.

The efficient shape is: bulk-search for identifiers, then hydrate those identifiers in 500-id batches. Bulk search keeps your field selection minimal so each page stays small; batch does the expensive metadata retrieval 500 papers at a time.

// 1. page through bulk search, collecting ids only
let ids = [], token = null;
do {
  const url = `${BASE}/paper/search/bulk?query=%22research+data+management%22`
            + `&fields=paperId&year=2020-` + (token ? `&token=${token}` : '');
  const page = await requestWithBackoff(url);
  ids.push(...page.data.map(d => d.paperId));
  token = page.token;                 // null once the last page is served
  await sleep(1200);
} while (token);

// 2. hydrate in 500-id batches
for (let i = 0; i < ids.length; i += 500) {
  const batch = await requestWithBackoff(`${BASE}/paper/batch?fields=title,year,citationCount,isOpenAccess`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ ids: ids.slice(i, i + 500) })
  });
  // …zip `batch` against ids.slice(i, i + 500) by index
  await sleep(1200);
}

Measured on that exact code, unauthenticated, on 26 August 2026:

• The bulk search reported total: 1306 and served it as 1,000 + 306 across two pages. token was present on the first page and null on the last — a null token, not an empty data array, is the loop’s terminating condition.
• Hydrating the first 1,000 ids took two batch calls.
• End to end: 1,000 papers fully hydrated in 4 HTTP calls plus 1 retry, in 10.2 seconds, with 0 nulls.

The comparison that makes the point: the same 1,000 papers fetched one at a time through /paper/{id}, at the 451–532 ms per-request latency measured above, is a minimum of about eight minutes of wall-clock time and 1,000 times the request budget — and it does that against a shared pool that will start throttling long before it finishes. The batching is not an optimisation. It is the difference between a job that completes and one that does not.

Know when to stop batching and switch to the dataset

A separate bulk search for "machine learning" returned total: 1039448. At 1,000 records per page, that result set alone is 1,040 search calls before you retrieve a single field of metadata, plus another 2,079 batch calls to hydrate it — more than 3,100 requests against a pool documented to throttle “during periods of heavy use”, and against an API-key allocation of 1 request per second. That is roughly an hour of perfect execution with no retries, which is not a thing that happens.

Practical threshold: when total comes back in six figures, the Datasets API is the correct tool and the Academic Graph API is the wrong one. Semantic Scholar says so itself. Reading total from the first bulk-search page — before writing the retrieval loop — costs one request and tells you which of the two jobs you are actually doing.

The retry loop this all depends on

Every measurement above was taken through the same wrapper. It is short, and the two things it gets right are the two things most implementations get wrong:

async function requestWithBackoff(url, opts = {}, maxTries = 8) {
  let wait = 1000;
  for (let attempt = 1; attempt <= maxTries; attempt++) {
    const res = await fetch(url, { ...opts, headers: { 'User-Agent': UA, ...opts.headers } });

    // 429 AND 5xx are both throttle signals here — not just 429.
    if (res.status === 429 || res.status >= 500) {
      await sleep(wait);
      wait = Math.min(wait * 2, 30000);   // cap the backoff; do not grow unbounded
      continue;
    }
    return res.json();
  }
  throw new Error(`exhausted ${maxTries} attempts: ${url}`);
}

Three notes on it:

• No Retry-After header was returned on any 429 observed. It was checked explicitly and came back null every time, so exponential backoff is not merely a good default here — it is the only available signal.
• Cap the backoff. Doubling without a ceiling produces multi-minute sleeps that are indistinguishable from a hang.
• Send a descriptive User-Agent identifying your project and a contact URL. It is ordinary good manners on shared scholarly infrastructure, and it is what makes you reachable rather than blockable if your traffic causes a problem.

Semantic Scholar or OpenAlex? An honest comparison

These are the two large free scholarly graphs, and for most research-administration work they are genuinely substitutable. Where they are not:

Dimension	Semantic Scholar (S2AG)	OpenAlex
Citation contexts and intents	Yes — the differentiator. Sparse (measured 4–84 contexts per 100 records), but no free alternative offers it at all.	No.
Full text	S2ORC, a separate open release covering a subset of the corpus, is what makes contexts and TLDRs possible.	No full-text corpus of its own.
Access model	Free; optional key raising you from a contended shared pool to a guaranteed 1 RPS.	See CASRAI’s reporting on OpenAlex moving to mandatory keys and tiered pricing — a materially different footing from when most tutorials were written.
Institution disambiguation	Author and affiliation data present; not its design focus.	Stronger institutional modelling with ROR, but with real known error patterns — see the OpenAlex guide’s section on institution disambiguation before trusting affiliation counts.
Bulk retrieval ergonomics	500-id batch POST; 1,000-per-page bulk search; hard 10,000 ceiling on the citations endpoint.	Cursor paging plus a bulk snapshot — see OpenAlex API endpoints, authentication and pagination.
Lineage	Independent Ai2 corpus, later merged with Microsoft Academic Graph data.	Direct successor to MAG.

The decision rule. If you need to know why something was cited, only Semantic Scholar can tell you, and you accept sparse coverage as the price. If you need institutional aggregation, coverage breadth, or a licence-clean full snapshot, OpenAlex is the better fit. If you need reproducible counts for research assessment, read how both compare against Scopus and Web of Science first — neither free graph is a drop-in replacement for a curated index, and the broader landscape of scholarly metadata sources covers the rest of the field. Using both, and reconciling on DOI, is a legitimate and common design.

Attribution and licensing

S2AG and S2ORC are released under the ODC-BY 1.0 open-data licence, which permits reuse including commercial reuse, with attribution. Semantic Scholar’s licence terms ask that public-facing uses link back to semanticscholar.org and that publications built on the data cite the Semantic Scholar Open Data Platform paper. This is a low bar and there is no reason to be sloppy about it: an attribution line in your methods section and a link in your interface satisfies it. Open citation data generally is covered in CASRAI’s guide to OpenCitations and I4OC.

Frequently asked questions

Do I need an API key for the Semantic Scholar API?

No. Most Academic Graph endpoints work unauthenticated. But the unauthenticated pool is shared with every other anonymous caller and, as measured above, produced 0 successes in 6 unpaced attempts on /paper/search. If your work has to finish on a schedule, request a key.

Why does an API key give only 1 request per second when unauthenticated access is documented at 1,000?

Because the 1,000 RPS is the capacity of one key shared by every anonymous user worldwide, not an allocation you hold. The 1 RPS from a key is guaranteed and uncontended. A key trades an unpredictable slice of a large number for a small, reliable one. Semantic Scholar’s tutorial notes that a higher rate may be granted following a review.

What is the maximum number of IDs I can send to the batch endpoint?

500. Sending 501 returns HTTP 400 with {"error":"Maximum 500 ids allowed in input list"}. Mixed identifier prefixes (DOI:, CorpusId:, arXiv:, PMID:, URL:) can be combined in one call.

Why did the batch endpoint return null for a DOI I know is valid?

A null means Semantic Scholar could not resolve that identifier, not that the identifier is malformed. This was confirmed on 10.1038/nature14539, which Crossref resolves to a 2015 Nature article and which the S2 batch endpoint returned null for on the same day. Results are positional, so match them to your input array by index and log unresolved ids separately.

Why are the contexts and intents fields usually empty?

Both are derived from the citing paper’s full text, which Semantic Scholar does not hold for every record. Coverage is also strongly position-dependent: the citations endpoint returns the newest citing papers first, and those are the least likely to have been ingested. Measured on one paper, contexts rose from 33 per 100 at offset 0 to 51 per 100 at offset 5,000, and intents from 0 to 23.

Can I retrieve every citation of a highly cited paper?

Not through the citations endpoint. It enforces offset + limit < 10000, returning HTTP 400 beyond that. For a paper with more than 10,000 citations, use the Datasets API rather than claiming completeness from a paged retrieval.

What is the difference between /paper/search and /paper/search/bulk?

/paper/search is relevance-ranked, returns small pages, and was the most heavily throttled endpoint measured. /paper/search/bulk returns up to 1,000 records per call with a continuation token and supports filters such as year. Use search to find a paper; use bulk to retrieve a set.

How do I know when to stop paging a bulk search?

When token comes back null. It is present while more pages exist and null on the final page — an observed result set of 1,306 records served as 1,000 + 306 with a null token on the second page. Do not terminate on an empty data array.

How is this page different from CASRAI’s general Semantic Scholar guide?

The general guide covers what Semantic Scholar is, what its corpus contains, and how its features work for a researcher using the website. This page is for someone writing code against the API.

Related CASRAI resources

• Semantic Scholar: what it is and how it works — the corpus, TLDRs, and influential-citation classification.
• OpenAlex API: endpoints, authentication and pagination — the closest comparable free graph API.
• OpenAlex: what it is and how it works — coverage against Scopus and Web of Science, and institution disambiguation.
• Crossref REST API and Crossref explained — the authority for DOI-registered metadata.
• Google Scholar API — why there is no official one, and what people use instead.
• MeSH terms in PubMed — the same Documented / Measured / Observed treatment applied to PubMed’s E-utilities.
• The PubMed E-utilities API — ESearch, EFetch and ELink over HTTP, with the measured retrieval ceiling and rate limits, for the biomedical half of the same job.
• Unpaywall — open-access status detection to pair with S2 metadata.
• Citation indexing and bibliometric analysis — what to do with the data once retrieved.
• Getting clean BibTeX from a DOI — a smaller, adjacent retrieval task.
• Research tools — the full cluster.

All measurements on this page were taken on 26 August 2026 against the live Academic Graph API, unauthenticated, from a single client. The unauthenticated pool is shared and contended; re-running these checks takes a few minutes and is the right response to any number here looking wrong.