Convert NCBI FTP workflow notebooks to python scripts

[ialarmedalien]

The notebooks notebooks/ncbi_ftp_manifest.ipynb and notebooks/ncbi_ftp_promote.ipynb should be turned into scripts for automated running.

Proposed workflow:

• input: list of IDs of interest or dataset(s) of interest (e.g. GTDB release XXX + RefSeq, etc., just RefSeq, etc.)
• pull assembly_summary files from NCBI, save in refdata tenant under raw_data
• compare with stored assembly_summary files from previous update to generate a list of IDs to update (IDs may be new, retired, updated, etc.)
• run NCBI REST API and NCBI FTP workflows on the CTS using the IDs to update as input
• use converted ncbi_ftp_promote.ipynb script to update the sequence archives in the ref data tenant
• run script (to be written) to update REST API output archives in ref data tenant
• run script to generate/update file index from frictionless manifests in metadata/ directory

May want to consider using PydanticSettings to enable configuration via command line params / env vars / etc.