Research Intelligence Methodology

NCBI E-utilities API (ESearch → EFetch), daily incremental runs. Records deduplicated by PMID.

Publication dates from January 2000 forward. Incremental runs re-query by the date a record was added to PubMed, so records indexed after their publication date are also captured. Papers published before 2025 are still being collected: earlier full-range searches kept only the most recent results per query, and a year-by-year backfill is planned.

A publication is included if it matches either landscape query (AI/ML terms co-occurring with health context terms) or is affiliated with a tracked health AI company. Book chapters indexed in PubMed (e.g., StatPearls) are not included; the data covers journal articles. MeSH terms assigned by the National Library of Medicine are stored with every record and drive clinical-area categorization downstream. A paper is linked to a company only when an author affiliation names the company itself: its name, legal name or a former name. A subsidiary named with the company counts; a distributor or reseller does not. A company's affiliation search may exclude named organizations, places or authors that share its name; each exclusion is listed in our records and checked against the papers it removes. Links our editors add or remove are kept and never overridden by later runs. We checked a random sample of 579 company–paper links by reading every author affiliation; 571 (98.6%, 95% CI 97.3–99.3%) named the company. Companies whose sample fell below 80% correct were given narrower search names and checked again.

api.biorxiv.org, both servers (bioRxiv and medRxiv), daily. The API has no server-side search, so all filtering happens at insert time. Deduplicated by DOI.

All patterns use word boundaries and case-insensitive matching to avoid false positives (e.g., "LLM" must be a standalone token).

A preprint is stored only if it matches at least one pattern. The specific matched keywords are saved on every record for auditability, so any inclusion can be traced back to the exact terms that triggered it. A stored preprint takes its newest version; if that version no longer matches, it is no longer counted. A preprint is linked to a company only when the corresponding author's institution names the company as a whole word, using the same search names as for publications. Links our editors add or remove are kept and never overridden by later runs.

NIH RePORTER Projects API v2 (POST /v2/projects/search), paginated per fiscal year to stay under the API result cap. Deduplicated by application ID.

Fiscal years 2000 to the current fiscal year for the full backfill; nightly incremental runs cover the current and prior fiscal year. A separate organization pass searches each tracked company's reviewed RePORTER organization names across all fiscal years.

A grant counts as health AI research only if a landscape term matches its title or NIH-indexed terms. Grants stored for other reasons are kept but not counted. Examples are grants to a tracked company that match no landscape term, or grants cited by a stored publication. A grant is linked to a company only when its recipient organization's name exactly matches one of that company's reviewed RePORTER organization names, ignoring case and punctuation. Companies with no reviewed name have no automatic grant links. Links our editors add or remove are kept and never overridden by later runs. Project abstracts, funding amounts, PI details, and administering institute are stored in full.

OpenAlex API with an OpenAlex API key and a contact email (OpenAlex meters usage per day). Publications are matched by PubMed ID in batches of 50, with a DOI lookup when the PubMed ID is not found.

Chosen over Google Scholar, which has no API and blocks automation. New publications are looked up nightly. Those not yet in OpenAlex are retried nightly for 60 days, then weekly. All counts are refreshed weekly. A failed lookup is never recorded as zero citations.

ClinicalTrials.gov API v2, daily incremental. Deduplicated by NCT ID; updated trials are refreshed in place.

The landscape query searches all indexed trial text (interventions, conditions, titles, summaries). Incremental runs fetch trials whose registry record was updated in the last 7 days; the daily company check searches each tracked company's reviewed ClinicalTrials.gov sponsor names, as lead sponsor or collaborator.

A trial counts as health AI research only if it matches the AI/ML landscape query. Trials stored for other reasons, such as a tracked company's trials that match no landscape term, are kept but not counted. A trial is linked to a company only when its lead sponsor or a collaborator exactly matches one of that company's reviewed ClinicalTrials.gov sponsor names, ignoring case and punctuation. Contract research organizations working for other sponsors are not linked, and companies with no reviewed name have no automatic trial links. Links our editors add or remove are kept and never overridden by later runs. Phase, enrollment, status, sponsor class, design fields (allocation, masking, age groups), and MeSH condition/intervention terms are stored in full.

Status fields come directly from the registry record. Stop-reason classification runs nightly on new stopped trials.

Many AI trials are observational or device-based and do not use traditional drug phases (marked N/A). The ~21,500 stopped trials carrying a free-text stop reason are classified by an LLM into seven buckets — enrollment, funding, business decision, safety/efficacy, COVID-19, data/results, other — which powers the "Why Trials Stop" analysis on Clinical Validation pages. Classifications are stored per trial and never overwrite registry data.

The official FDA AI-Enabled Medical Device List CSV is downloaded from fda.gov and upserted daily. Each device is then enriched via exact submission-number lookups on openFDA (device/510k by K-number, device/pma, device/classification).

The FDA list itself is the inclusion criterion — we do not decide what counts as an "AI-enabled device"; the FDA does. Clinical areas are assigned in two passes: the FDA review panel deterministically sets a first area, then an LLM reads the device name, generic name, company and review panel and adds up to three areas the panel label alone would miss. The primary area is the review panel's area, unless that is a cross-cutting area and the device has a clinical-specialty tag; then the first clinical specialty in taxonomy order is primary. A device is linked to a tracked company when FDA's applicant name and the company's name are the same after ignoring case, punctuation and legal-form words such as Inc., LLC or GmbH, or when one of the two names begins with the other and either both have at least two words or the shorter has at least eight characters. A name that fits more than one company is not linked. Links our editors add or remove are kept and never overridden by later runs.

openFDA device/recall (recalls), device/event (MAUDE adverse event reports) and device/udi (the FDA's Global Unique Device Identification Database, GUDID). Recalls are checked daily with a 14-day lookback. Adverse events are re-checked daily from a week before the newest stored report, because openFDA publishes them in monthly batches.

A recall is linked to an AI device when it lists the device's 510(k) or PMA number; when it lists neither, it is linked to AI devices with the same product code from the same company. Recall counts are distinct recall events. An adverse event report counts for an AI device in one of two ways. First, by device identifier: the report lists a device identifier (UDI-DI) whose GUDID record names a 510(k), De Novo or PMA number on the FDA AI list; a report whose identifier leads only to other submissions does not count. Second, when a report has no usable identifier: it must carry the device's product code, come from the device's company or one of its subsidiaries, name the device's product line in its brand or model (from a written list), and be received on or after that product line's first AI clearance. Software sold as an option for a larger system counts only reports that name the software itself. Reports rarely give a software version, so a report counts for every AI clearance in its product line decided by the report date; totals count each report once. In a seeded random sample, 200 of 200 identifier matches and 193 of 200 name matches were about the AI device's product line (95% intervals 98.1–100% and 93.0–98.3%). Reports stored earlier by product code alone are kept but not counted.

Hand-curated registry maintained in code with every entry source-verified against primary CMS and AMA documents.

Inclusion requires verification against primary sources (CMS Medicare Physician Fee Schedule, CMS billing articles, AMA CPT releases); draft or rumored codes are excluded. Each entry carries the code, official descriptor, code type (Category I / Category III / HCPCS), AMA Appendix S taxonomy tier (assistive / augmentative / autonomous), clinical area, example devices, effective date, status, and source notes. A code’s presence in this registry does not establish coverage, a payment rate, or product-specific billing eligibility. A small number of non-AI "context codes" (e.g., the 92227–92229 retinal imaging family) are retained and flagged for trend comparison.

data.cms.gov "Medicare Physician & Other Practitioners — by Geography and Service" datasets, one fixed dataset per calendar year (2013–2024), refreshed when CMS publishes a new annual release.

Only rows whose HCPCS code appears in our curated code registry are ingested — utilization is measured for the codes in our registry, including the 2 context codes, which are labeled as context.

Service counts, provider counts, and average payment amounts are aggregated per code per year. Because CMS data lags roughly two years, the most recent year shown is the latest CMS release, not the current calendar year.

USAspending API v2 (spending_by_award), contract award types A–D, award period 2016 to present, paginated and deduplicated by award ID.

Keyword search casts a wide net by design; a second-stage LLM relevance screen then evaluates every contract description individually.

A contract is kept only if it is genuinely about AI/ML applied to healthcare, medicine, clinical care, biomedical research, or health administration. Explicitly rejected: generic IT infrastructure, non-health AI, staffing contracts without AI substance, and non-AI health services. Roughly 21% of fetched contracts pass (346 of 1,650 at last count); passing contracts are tagged to one to three clinical areas with confidence scores, and rejected ones are retained but excluded from dashboards.

SEC XBRL companyconcept API, queried per company CIK on every daily run.

Only companies in our tracked public-company list are pulled. Values come directly from company XBRL filings with no restatement or adjustment on our side.