Skip to content

Bundle maintenance fixes (#499-#504) and fix team_game_logs / date parsing on pandas 3 - #514

Open
yasumorishima wants to merge 16 commits into
jldbc:masterfrom
yasumorishima:combine/maintenance-fixes
Open

yasumorishima wants to merge 16 commits into
jldbc:masterfrom
yasumorishima:combine/maintenance-fixes

Conversation

@yasumorishima

Copy link
Copy Markdown

This bundles my six open fixes (#499–#504) into one branch so they can be reviewed and merged together, and adds a few fixes I found while re-testing them against pandas 3. Each fix is in its own commit(s), so any of them can be dropped or reverted on its own. I'll close the six originals with a pointer here.

I've left out my docs typo fix (#498) because #491 makes the same change and was opened first.

Closes #467
Closes #459
Closes #455
Closes #462
Closes #461
Closes #486

From the earlier PRs

New in this PR

  • team_game_logs on pandas 3 – postprocess still called DataFrame.apply(pd.to_numeric, errors="ignore"). pandas 3 removed that option, so team_game_logs() raised ValueError: invalid error value specified. It now applies a small helper that returns the column unchanged when it can't be converted, which keeps the old behaviour (including tables with duplicate column labels).
  • Date columns on pandas 3 – try_parse_dataframe only looked at columns whose dtype name is object or string. pandas 3's default string dtype is named str, so date columns such as statcast's game_date were returned as text. With fix: replace deprecated errors='ignore' with explicit try/except #499 applied, this is what makes test_try_parse_dataframe fail under pandas 3.
  • Athletics from 2025 – the extrapolation in Fix team_ids returning empty data for seasons after 2021 #504 assumed nothing had changed since 2021, but the Athletics are listed as ATH from 2025 on Baseball Reference (/teams/ATH/2025.shtml) and Retrosheet (TEAM2025), and MLB StatsAPI also switches from OAK to ATH in 2025. teamIDBR and teamIDretro now return ATH for 2025 and later. I left teamID / franchID (Lahman) unchanged because I couldn't confirm what Lahman uses for 2025.
  • PyGithub pin – retrosheet now imports github.Auth (Fix deprecated GitHub authentication in retrosheet.py #501), which first appeared in PyGithub 1.59.0, so setup.py now requires pygithub>=1.59 instead of >=1.51.

Testing

I ran the full tests/pybaseball suite on this branch and on master, with pandas 3.0.6 and with pandas 2.2.3 (Python 3.11):

  • pandas 2.2.3: 146 passed on this branch (master: 135 passed). Warnings drop from 423 to 85.
  • pandas 3.0.6: 12 failed / 134 passed on this branch; master has 12 failed and 1 error. The fangraphs, amateur draft, leaders and statcast_batter failures are the same on both and are not addressed here. test_statcast_single_game_request errored during fixture setup on master; on this branch it gets past the dates and fails comparing hc_x (nan vs <NA>), so pandas 3 support still needs more work after this.

The new regression tests fail when run against the source without the fixes and pass with them. mypy reports no new errors compared with master.

Replace deprecated chained assignment with direct assignment
to avoid FutureWarning in pandas 3.0.
Replace deprecated `Github(token)` with `Github(auth=Auth.Token(token))`
to avoid DeprecationWarning from PyGithub.
- Add team.upper() to handle case-insensitive team abbreviations
- Add end_season < start_season validation with clear error message
…erence HTML (jldbc#461)

Baseball Reference changed their table IDs:
- team_batting -> players_standard_batting
- team_pitching -> players_standard_pitching

Also make header extraction dynamic instead of hardcoded slice indices.
The bundled CSV data ends at 2021 because the upstream Lahman/Retrosheet
data hasn't been updated. Since MLB team composition hasn't changed,
extrapolate from the last known year when a more recent season is requested.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant