[pavlis]

This branch initiates creating a standard interface to Amazon S3 for mspass for both serial and parallel workflows. There is work to be done before we merge it, but this adds these features:

1. A generic fetch_s3_client that can be used for serial or parallel workflows via the same api. The design parallels the similar use for a database client implemented a few months ago.
2. The prototypes for this came from workflows used for the 2026 Earthscope MsPASS course. This differs significantly from that prototype found in the tutorials repository in several ways. Most notably it defines 3 different classes for setting up an s3 client for GeoLab (requires the earthscope_sdk to function and has a lot of additional complications), Anonymous access for open data, and a Stock version for someone using MsPASS running with an AWS account from an AWS server.
3. The serial version adds a decorator @lru_cache for efficiency when an s3 client needs to be accessed inside a function in a serial workflow. An older version required passing a session object in a serial job which turns out to be a potential issue. When a client is created on each entry from a session object new connections have to be created. That is at least inefficient and at worst may have been subject to leaking resources by not properly closing connections.

I tested the GeoLab and Anonymous versions on GeoLab and both work there for a simple test. Most lacking is a pytest suite to fully exercise this new code. An additional glaring deficiency is no documentation for this new feature in docs. There are some subtle issues we will need to explain for using this we must document before we release the world on this feature.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.08%. Comparing base (c53d735) to head (e028e07).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1039      +/-   ##
==========================================
+ Coverage   74.94%   75.08%   +0.14%     
==========================================
  Files         184      185       +1     
  Lines       37795    37900     +105     
  Branches     2565     2565              
==========================================
+ Hits        28324    28457     +133     
+ Misses       8548     8520      -28     
  Partials      923      923              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pavlis]

Changes I see all look sensible. Grateful to have an AI to create the pytest functions,. I had no idea how to do that. What it creates seems reasonable.

[pavlis]

Thought about merging this but maybe we need a rough documentation page first. On the other hand, putting this in the master will allow us to test it more easily and cleanly. That may be needed to write an effective user manual page anyway. What do you think @wangyinz ?

[pavlis]

@wangyinz maybe we should use a simpler name for this module? We could use just "s3.py" and add other functionality to this module directly, but it would probably be better to keep this organization centered on usage of a client but shorten the name to something like "s3client.py". I'll let you decide as I'm not at all sure which is better or if it even matters.

[wangyinz]

I think we could merge it and add the docs later after testing more on it. For the names, I don't have a strong preference. If had to pick, I guess s3client makes more sense.

(somehow github stop sending me notice on messages like these... sorry for ignoring it earlier)

[pavlis]

I think I will go ahead and change the module name. s3client makes more sense. If I figure out how to do that I'll do it and merge this pull request.