Skip to content

SPHEREx cleanup: refactor cutouts/psf, preserve MEF headers, drop redundant SDT README - #396

Open
jkrick wants to merge 5 commits into
Caltech-IPAC:mainfrom
jkrick:spherex-cutouts-psf-quality
Open

jkrick wants to merge 5 commits into
Caltech-IPAC:mainfrom
jkrick:spherex-cutouts-psf-quality

Conversation

@jkrick

@jkrick jkrick commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

In preparation for the upcoming DR1 updates. Closes #150, closes #238, closes #275.

Summary

  • Switch TAP access from raw pyvo to astroquery.ipac.irsa.Irsa.query_tap
  • Use SkyCoord instead of separate ra/dec floats
  • writeto(..., overwrite=False) so output isn't silently clobbered
  • Plot wavelength as an astropy Quantity instead of a bare float
  • Drop a duplicated caching tip box in spherex_cutouts.md
  • Preserve the cutout's original primary header (incl. VERSION) in the combined MEF's
    primary HDU instead of writing a blank one, and record each cutout's version in
    CUTOUT_INFO
  • Fold the Source Discovery Tool README's directory-contents list into its notebook
    (the rest of the README was already duplicated there) and remove the README
  • Open spherex_intro.md's spectral image with a context manager, reading all extensions into
    memory so the downstream cells work after the file is closed (spherex_intro.md part of Make all fits.open() calls use a context manager #129)
  • Also treat http.client.RemoteDisconnected as a transient read error in spherex_cutouts.md
    (skip that cutout) and spherex_psf.md (retry), after CI failed on a server disconnect
    mid-read

Note on #129

spherex_cutouts.md, spherex_psf.md, and the SDT notebook already use context managers for
every fits.open() call. spherex_intro.md's bare fits.open() call is fixed here too (purely
mechanical, does not touch the QR3-stale content next to it, which stays for #374). The two
Euclid notebooks and the Roman notebook also listed in #129 are untouched -- out of scope for
this PR.

Test plan

  • Ran spherex_cutouts.md and spherex_psf.md end to end against production
    (jupytext --to notebook --execute); no errors
  • Verified VERSION is present in the output MEF's primary header
  • Verified the SDT notebook still parses correctly after the README merge (full
    execution needs SExtractor/Firefly/bokeh, not run here)
  • Ran spherex_intro.md end to end against production; no errors

jkrick and others added 3 commits October 22, 2026 14:31
…rve MEF headers

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…h-IPAC#129)

Read every extension into memory inside the `with` block so the existing
downstream `hdulist[...]` cells (including the WAVE-TAB spectral WCS,
which needs `fobj=hdulist`) keep working after the file is closed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jkrick jkrick self-assigned this Oct 22, 2026
@jkrick jkrick added the content: spherex Content related issues/PRs for notebooks with SPHEREx relevance label Oct 22, 2026
@jkrick
jkrick requested review from afaisst and troyraen and removed request for afaisst October 22, 2026 22:30
jkrick and others added 2 commits October 22, 2026 16:56
The README was folded into the notebook and deleted in 00c9664, which left
the manifest pointing at a missing file and failed the manifest check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI hit http.client.RemoteDisconnected while reading SPHEREx images;
catch it alongside the other transient errors so cutouts skips the
image and psf retries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jkrick

jkrick commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Added update to fix the failing notebook test. CI hit http.client.RemoteDisconnected while reading a SPHEREx
image, and that error wasn't in the list of transient errors we catch. It's now caught in both places that use
that list: spherex_cutouts.md skips the affected cutout, and spherex_psf.md's retry loop retries it (and still
raises after 3 attempts).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

content: spherex Content related issues/PRs for notebooks with SPHEREx relevance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove source discovery tool README Preserve HDU headers in spherex_cutouts.md notebook Refactor spherex_cutouts and spherex_psf

1 participant