Skip to content

dhoconno/reporter

Folders and files

NameName
Last commit message
Last commit date

Latest commit

421547c · Mar 24, 2025
Mar 13, 2025
Mar 24, 2025
Mar 24, 2025
Mar 20, 2025
Mar 3, 2025
Mar 9, 2025
Mar 9, 2025
Feb 12, 2025
Mar 20, 2025
Mar 20, 2025
Mar 20, 2025
Mar 24, 2025
Mar 24, 2025
Mar 24, 2025
Mar 24, 2025
Mar 20, 2025
Mar 24, 2025
Mar 24, 2025
Mar 24, 2025
Mar 24, 2025

Repository files navigation

NIH RePORTER reporter

Like the 80s band Mr. Mister, this is the RePORTER reporter. It generates a plot of cumulative grant awards announced in NIH RePORTER by date, comparing the current year to the previous nine. This can identify trends in award disbursement.

It uses the RePORTER API to retrieve all grants by award_notice_date and then creates the output graph. This includes awards that are both newly awarded and non-competing renewals.

Based on a spot check of NIH RePORTER web captures in the Internet Archive Wayback Machine data is refreshed on Sundays. Consequently, current year grant awards could be undercounted by up to one week.

I welcome feedback and corrections if you find any mistakes!

2025-03-09 - when NIH RePORTER is down, the plots from the current and previous month erroneously show no grants from these months. This should self-resolve as soon as RePORTER is online during a daily refresh.

Now with 100% more federal register

Under the Federal Advisory Committee Act study sections that review NIH grants need to be announced in the Federal Register at least 15 days before the study section meets. I parsed the meetings from Federal Register notices and created plots comparing the current year to the previous nine years.

Latest Cumulative Award Count Plot

Cumulative Awards Plot

Interactive Version (Award Counts)

Latest Cumulative Award Amount Plot

Inspired by this analysis in the NYT, I've also added plots of funding amounts to the plots.

Cumulative Award Amounts Plot

Interactive Version (Award Amounts)

Latest Federal Register Study Section Announcements Plot

Cumulative Federal Register Study Section Announcements

Interactive Version (Federal Register Study Section Announcements)

Notes and caveats

  • There may be delays between award notices sent to investigators and their appearance in RePORTER, so that may introduce artifactual lag for the most recent days.
  • I am not an expert on NIH RePORTER and rely on award_notice_date as an indicator. There may be subtleties to how RePORTER works that distort these results.
  • If there are flatlines for any month, please let me know so I can check for an error in the API call results.
  • I'm a virologist, not a proper data scientist. So I welcome any and all constructive feedback from those who find problems or areas of improvement.
  • I'm spot checking the results but there is no guarantee of accuracy.
  • I added a CSV output that shows the underlying data used for the RePORTER analyses.
  • Award amounts are not adjusted for inflation
  • I added Institute-specific plots. If you see any issues with the underlying parsing please let me know.

Institute-Specific Plots

The following NIH Institutes and Centers have individual plots available. Click on any link to view the corresponding interactive visualization.

Institute Available Plots
Center for Scientific Review awards | award amounts | meeting notices
Eunice Kennedy Shriver National Institute of Child Health and Human Development awards | award amounts | meeting notices
Fogarty International Center awards | award amounts | meeting notices
National Cancer Institute awards | award amounts | meeting notices
National Center for Advancing Translational Sciences awards | award amounts | meeting notices
National Eye Institute awards | award amounts | meeting notices
National Heart, Lung, and Blood Institute awards | award amounts | meeting notices
National Human Genome Research Institute awards | award amounts | meeting notices
National Institute of Allergy and Infectious Diseases awards | award amounts | meeting notices
National Institute of Arthritis and Musculoskeletal and Skin Diseases awards | award amounts | meeting notices
National Institute of Biomedical Imaging and Bioengineering awards | award amounts | meeting notices
National Institute of Dental and Craniofacial Research awards | award amounts | meeting notices
National Institute of Diabetes and Digestive and Kidney Diseases awards | award amounts | meeting notices
National Institute of Environmental Health Sciences awards | award amounts | meeting notices
National Institute of General Medical Sciences awards | award amounts | meeting notices
National Institute of Mental Health awards | award amounts | meeting notices
National Institute of Neurological Disorders and Stroke awards | award amounts | meeting notices
National Institute of Nursing Research awards | award amounts | meeting notices
National Institute on Aging awards | award amounts | meeting notices
National Institute on Alcohol Abuse and Alcoholism awards | award amounts | meeting notices
National Institute on Drug Abuse awards | award amounts | meeting notices
National Institute on Minority Health and Health Disparities awards | award amounts | meeting notices
National Library of Medicine awards | award amounts | meeting notices
Office of the Director awards | award amounts | meeting notices

Data last updated: March 20, 2025

Methodology (generated by Claude Sonnet 3.7)

The NIH grant award visualization script retrieves funding data directly from the NIH RePORTER API using a day-by-day accumulation methodology. The script queries the API for all grants with award notices issued between the start of each calendar year and the most recent Monday, going back ten years from the current year. The system implements a seven-day caching mechanism to minimize API calls while ensuring data freshness, with current-month data always being refreshed. The script generates two cumulative plots: one for award counts and another for award amounts, both following year-to-date (YTD) progression. The visualization distinguishes the current year with a solid red line, while previous years appear in pastel colors with dashed lines. All raw data is also exported to a single comprehensive CSV file containing year, date, cumulative metrics, and IC-specific breakdowns, then compressed using zstandard at maximum compression level to conserve space while maintaining analytical accessibility.

The Federal Register meeting analysis script systematically extracts and visualizes "Notice of Closed Meeting" announcements published by NIH institutes. The methodology involves querying the Federal Register's API for NIH documents containing specific title patterns related to closed meetings, processing publications from the start of each calendar year through the current date. The script downloads and parses each document's content using available XML, raw text, or HTML formats, applying regular expression patterns to extract meeting details including committee names, dates, and associated institutes. A comprehensive caching system captures both search results and document content to improve efficiency and reduce server load. The cumulative visualization plots the running total of closed meeting announcements by publication date for multiple years, with the current year highlighted in red and previous years in pastel colors. Unlike the NIH grant award script which uses weekly cutoffs, this analysis includes data through the current day to reflect the Federal Register's daily update schedule. The finished visualization maintains consistent dimensions (1200x800 pixels) and resolution with the grant award plots for visual coherence. All extracted meeting data is compiled into a single CSV file containing publication dates, committee names, meeting dates, and institute information, then compressed using zstandard compression to optimize storage while preserving all details necessary for reproduction or further analysis.

Acknowledgements

  • ChatGPT o3-mini-high and Claude Sonnet assisted with deciphering the RePORTER API format and preparing the plots and GitHub Actions.
  • Claude Sonnet 3.7 wrote the code for plotting insets, which is much fancier than I know how to do myself

About

Plot cumulative awards in NIH RePORTER by date

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages