What's Changed
New & Updated Data
- Add BA subregion level imputations of EIA-930 hourly demand by @zschira in #4149
- Add BA level imputations of EIA-930 demand by @zschira in #4162
- Add EIA-930 regionally aggregated demand output tables by @zschira in #4190
- Add Q1 2025 CEMS data to ETL by @krivard in #4238
- Integrate Q2 bulk electricity EIA API data by @e-belfer in #4246
- Add Q2 2025 EIA 930 data to ETL by @krivard in #4242
- Integrating (most of) 2024 FERC Form 1 by @cmgosnell in #4215
- Update FERC DOIs and XBRL to SQLite settings for 2024 by @zaneselvans in #4250
- Integrate 2014-2018 VCE RARE data by @e-belfer in #4239
- Integrate EIA-860M and EIA-923 for 2025Q2 by @aesharpe in #4240
Other Changes
- fix: up fly.io app concurrency limit to avoid 502s by @jdangerx in #4067
- feat: output a datapackage.json for PUDL parquet files. by @jdangerx in #4070
- Add PUDL viewer log sink by @jdangerx in #4071
- Dbt setup by @zschira in #4011
- New citations! by @krivard in #4084
- Docs update: accumulated typos identified during onboarding by @krivard in #4016
- fix pr number and add thanks to jmelot by @cmgosnell in #4108
- Add survey announcement banner to docs by @zaneselvans in #4110
- chore: add deprecation warning to datasette template by @jdangerx in #4073
- fix: lower concurrency so we don't run into skyrocketing memory issue by @jdangerx in #4118
- chore: add PUDL viewer to docs by @jdangerx in #4115
- make weighted quantile continuous by interpolation by @marianneke in #4090
- chore: remove tasklists since they're being replaced. by @jdangerx in #4087
- Run dbt data tests in CI / integration tests by @zaneselvans in #4122
- feat: add metrics dashboard by @jdangerx in #4135
- Refactor Time series cleaning by @zschira in #4113
- Remove obsolete, failing FERC-714 state demand row count test. by @zaneselvans in #4147
- Remove deprecated minmax rows tests by @zaneselvans in #4146
- Migrate correlation validation test by @aesharpe in #4117
- Add test to compare column values by @aesharpe in #4123
- "Bespoke" category dbt validation migrations (easy/medium only) by @krivard in #4114
- "Bespoke" category dbt validation migrations (hard only) by @krivard in #4151
- fix: up fly app timeout by @jdangerx in #4168
- Remove ZEROLab from PUDL Sustainers. by @zaneselvans in #4164
- Renormalize SEC 10-K company info tables by @zaneselvans in #4134
- Remove obsolete vs_self, vs_historical, & check_unique_rows tests by @zaneselvans in #4183
- Integrate the Census-based FIPS codes to replace
addfips
by @cmgosnell in #4019 - fix: keep db migration job working with rename of parquet-fe-prototype by @jdangerx in #4192
- fix: retry on OS errors by @jdangerx in #4193
- Add devtools script to check for schema drift against dbt by @krivard in #4145
- Refactor dbt helper by @marianneke in #4201
- Remove PUDL user survey banner from documentation. by @zaneselvans in #4208
- Fix aggregated eia930 imputed demand row counts by @zschira in #4213
- Use correct target in dbt_helper integration test by @zaneselvans in #4219
- Update metadata for eia_bulk_elec to eiaapi by @e-belfer in #4212
- Use latest version of ferc xbrl extractor with bug fix by @zschira in #4224
- Make dbt target selection explicit in dbt_helper arguments. by @zaneselvans in #4223
- Migrate vs_bounds validation tests to dbt by @krivard in #4171
- Remove deprecated fbp_ferc1 vs_bounds data validations. by @zaneselvans in #4245
- Migrate existing asset checks into dbt data tests by @zaneselvans in #4198
- Add update table logic for dbt helper by @marianneke in #4222
- Skip MCOE null cols check that is crashing pytest worker. by @zaneselvans in #4253
- Resolve GeoPandas library warnings by @emmanuel-ferdman in #4252
- Disable row counts for etl-fast and remove nightly dbt target by @zaneselvans in #4259
- Improve ergonomics of FERC 1 explosion tables. by @zaneselvans in #4260
- feat: use forkserver when available for speed by @jdangerx in #4258
- fix: rowcounts check now fails on missing/extra partitions by @jdangerx in #4153
- Bloat the Google Batch VM by @zaneselvans in #4269
- Remove memory intensive output tests obviated by dbt row counts. by @zaneselvans in #4271
- add min_count=1 into groupby sum for heat rate by @cmgosnell in #4266
- Fix capitalizations in bibtex files by @krivard in #4209
- Disable Datasette deployment in nightly builds by @zaneselvans in #4274
- Close out release notes for v2025.5.0 by @zaneselvans in #4279
New Contributors
- @marianneke made their first contribution in #4090
- @emmanuel-ferdman made their first contribution in #4252
Full Changelog: v2025.2.0...v2025.5.0