Release Agilepy 1.7.0

.github/workflows/continuous-integration-workflow.yaml

@@ -30,6 +30,7 @@ jobs:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
+          submodules: recursive
 
       # Cache dependencies to speed up future pipelines. Each job has its own cache
       - name: Cache pip dependencies
@@ -52,6 +53,10 @@ jobs:
           python3 -m pip install -r requirements.lock 
           python3 -m pip install .
 
+      - name: Download test dataset
+        shell: bash -l {0}
+        run: download_test_dataset.sh
+
       - name: Running unit tests and code coverage
         shell: bash -l {0}
         run: start_coverage.sh
@@ -74,6 +79,7 @@ jobs:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
+          submodules: recursive
 
       # Cache dependencies to speed up future pipelines. Each job has its own cache
       - name: Cache pip dependencies
@@ -96,6 +102,10 @@ jobs:
           python3 -m pip install -r requirements.lock 
           python3 -m pip install .
 
+      - name: Download test dataset
+        shell: bash -l {0}
+        run: download_test_dataset.sh
+
       - name: Running REST API tests with SSDC data download
         shell: bash -l {0}
         run: start_test_rest_api.sh
@@ -116,7 +126,8 @@ jobs:
     - name: Checking out code
       uses: actions/checkout@v4
       with:
-          fetch-depth: 0
+        fetch-depth: 0
+        submodules: recursive
 
     - name: Debug
       shell: bash -l {0}
@@ -130,6 +141,10 @@ jobs:
         python3 -m pip install -r requirements.lock
         python3 -m pip install .
 
+    - name: Download test dataset
+      shell: bash -l {0}
+      run: download_test_dataset.sh
+
     - name: Running unit tests and code coverage
       shell: bash -l {0}
       run: start_coverage.sh
@@ -158,6 +173,9 @@ jobs:
     steps:
     - name: Checkout code
       uses: actions/checkout@v4
+      with:
+        fetch-depth: 0
+        submodules: recursives
 
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@v3

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "agilepy/external_packages/BBrta"]
+	path = agilepy/external_packages/BBrta
+	url = https://github.com/ASTRO-EDU/BBrta.git

CHANGELOG.md

@@ -1,6 +1,19 @@
 # Changelog
 
-## Release 1.6.5 (May 20, 2025)
+## Release 1.7.0 (October 2025)
+* [PR!43](https://github.com/AGILESCIENCE/Agilepy/pull/43) solving [#42](https://github.com/AGILESCIENCE/Agilepy/issues/42). New feature: Bayesian Blocks Analysis. Added Classes `AGBayesianBlocks` and `AGBayesianBlocksConfig`, Unit Tests, Test Dataset, Tutorial Notebook and Documentation pages for the Bayesian Blocks Analysis.
+* [PR!55](https://github.com/AGILESCIENCE/Agilepy/pull/55) solving [#14](https://github.com/AGILESCIENCE/Agilepy/issues/14). New feature: Ratemeters Analysis.: Added Classes`AGRatemeters` and `AGRatemetersConfig`, Unit Tests, Test Dataset, Tutorial Notebook and Documentation pages for the AGILE Scientific Ratemeters Analysis.
+* [PR!47](https://github.com/AGILESCIENCE/Agilepy/pull/47) solving:
+  * [#35](https://github.com/AGILESCIENCE/Agilepy/issues/35). Minor Bugfix: remove the deprecated and unused parameter `logfilenameprefix` from the the configuration.
+  * [#61](https://github.com/AGILESCIENCE/Agilepy/issues/61). Minor Testing issue: Replace tests performed with `unittest` with equivalent `pytest` tests.
+* [PR!63](https://github.com/AGILESCIENCE/Agilepy/pull/63) solving [#12](https://github.com/AGILESCIENCE/Agilepy/issues/12), [#26](https://github.com/AGILESCIENCE/Agilepy/issues/26), [#63](https://github.com/AGILESCIENCE/Agilepy/issues/63). Major Class Review: Replace the classes and functions composing the "Engineering API" (`AGEngAgileOffaxisVisibility`,`AGEngAgileOffaxisVisibilityConfig`, `AGEngAgileFermiOffAxisVisibilityComparisonConfig`, `AGEngAgileFermiOffAxisVisibilityComparison`, `AGEngDisplayComparison`) with one class `AGVisibility`, with the goal of computing the pointing direction of the AGILE Spacecraft (and, optionally, Fermi) and the off-axis angle of asource. Add Classes `AGVisibility` and `AGVisibilityConfig`, Unit Tests, Test Dataset, Tutorial Notebook and documentation.
+* [PR!66](https://github.com/AGILESCIENCE/Agilepy/pull/66) solving [#59](https://github.com/AGILESCIENCE/Agilepy/issues/59). Improvements:
+  * Add `AGAnalysis.lightCurveTable` attribute to easily retrieve MLE and AP Light Curves.
+  * Add dedicated Tutorial on Aperture Photometry Light Curve Estimation.
+* Documentation updates and improvements.
+
+
+## Release 1.6.5 (May 20, 2025, d923ff5)
 * [#41](https://github.com/AGILESCIENCE/Agilepy/issues/41): Documentation update on installation procedures.
 * [#48](https://github.com/AGILESCIENCE/Agilepy/issues/48): Fix a bug in the `entrypoint.sh` script for `agilepy` container that caused it to crash.
 * [#46](https://github.com/AGILESCIENCE/Agilepy/issues/46): Implement `agilepy/scripts/start_test_rest_api.sh` to test the REST API with data download from SSDC, and use it in CI pipelines.

MANIFEST.in

@@ -1,5 +1,8 @@
 include README.md
 include agilepy/testing/unittesting/api/data/testcase.maplist4
+include agilepy/testing/unittesting/api/data/test_data.ph
+include agilepy/testing/unittesting/api/data/PKP080686_1_3913_000.lv1.cor.gz
+include agilepy/testing/unittesting/api/data/PKP086465_1_3913_000.lv1.cor.gz
 include agilepy/testing/unittesting/core/test_data/sourceconf.wrongext
 include agilepy/testing/unittesting/core/test_data/AGILE_test_datacoverage
 include agilepy/testing/unittesting/core/test_out/.gitkeep
@@ -11,6 +14,7 @@ recursive-include * *.xml
 recursive-include * *.source
 recursive-include * *.sh
 recursive-include * *.txt
+recursive-include * *.csv
 recursive-include * *.ipynb
 recursive-include * *.cts.gz
 recursive-include * *.coveragerc

README.md

@@ -18,7 +18,10 @@
 
 Agilepy is an open-source Python package developed at INAF/OAS Bologna to analyse AGILE/GRID data built on top of the command-line version of the AGILE/GRID Science Tools.
 
-Agilepy is a user-friendly Python interface designed to simplify AGILE/GRID data analysis by streamlining task configuration and data accessibility. With Agilepy, you can easily generate and display sky maps and light curves, access gamma-ray source catalogues, perform spectral model and position fitting (including background evaluation), analyze aperture photometry, and use wavelet analysis. Additionally, Agilepy offers an engineering interface that enables the analysis of AGILE's off-axis viewing angle time evolution for a specific sky region, comparing it with Fermi/LAT off-axis evolution.
+Agilepy is a user-friendly Python interface designed to simplify AGILE/GRID data analysis by streamlining task configuration and data accessibility.
+With Agilepy, you can easily generate and display sky maps and light curves, access gamma-ray source catalogues, perform spectral model and position fitting (including background evaluation), analyze aperture photometry, use wavelet and bayesian blocks analysis.
+Additionally, Agilepy offers a visibility interface that enables the analysis of AGILE's off-axis viewing angle time evolution for a specific sky region, comparing it with Fermi/LAT off-axis evolution.
+The analysis of the AGILE Scientific Ratemeters is also possible.
 
 
   * Documentation: [https://agilepy.readthedocs.io](https://agilepy.readthedocs.io)

agilepy/api/AGAnalysis.py

@@ -29,7 +29,8 @@
 import os
 import re
 import numpy as np
-# from tqdm import tqdm, trange  
+
+from astropy.table import Table  
 from tqdm.notebook import trange, tqdm
 from time import time
 from pathlib import Path
@@ -101,6 +102,10 @@ def __init__(self, configurationFilePath, sourcesFilePath = None):
             "mle" : None,
             "ap" : None
         }
+        self.lightCurveTable = {
+            "mle" : None,
+            "ap" : None
+        }
 
         self.multiTool = Multi("AG_multi", self.logger)
 
@@ -159,7 +164,6 @@ def getConfiguration(confFilePath, userName, sourceName, tmin, tmax, timetype, g
 output:
   outdir: {outputDir}
   filenameprefix: analysis_product
-  logfilenameprefix: analysis_log
   sourcename: {sourceName}
   username: {userName}
   verboselvl: {verboselvl}
@@ -955,7 +959,6 @@ def lightCurveMLE(self, sourceName, tmin = None, tmax = None, timetype = None, b
         (_, last) = Utils._getFirstAndLastLineInFile(configBKP.getConf("input", "evtfile"))
         idxTmax = float(Utils._extractTimes(last)[1])
 
-        # logFilenamePrefix = configBKP.getConf("output","logfilenameprefix")
         # verboseLvl = configBKP.getConf("output","verboselvl")
 
         self.logger.info( f"[LC] Number of processes: {processes}, Number of bins per process {len(binsForProcesses[0])}")
@@ -994,9 +997,8 @@ def lightCurveMLE(self, sourceName, tmin = None, tmax = None, timetype = None, b
         """
         processes = []
         for pID, pInputs in enumerate(binsForProcesses):
-            logFilenamePrefix += f"_thread_{pID}"
             self.logger.info("generating process...(%d). Chunk size for process: %d"%(pID, len(pInputs)))
-            p = Process(target=self._computeLcBin, args=(pID, pInputs, configBKP, lcAnalysisDataDir, logsOutDir, logFilenamePrefix, verboseLvl))
+            p = Process(target=self._computeLcBin, args=(pID, pInputs, configBKP, lcAnalysisDataDir, logsOutDir, verboseLvl))
             processes.append((pID,p,len(pInputs)))
 
         for pID, p, binsNumber in processes:
@@ -1019,6 +1021,10 @@ def lightCurveMLE(self, sourceName, tmin = None, tmax = None, timetype = None, b
         self.logger.info( "Took %f seconds.", time()-timeStart)
 
         self.lightCurveData["mle"] = str(lcOutputFilePath)
+
+        # Data Table
+        lc_table = Table.read(lcOutputFilePath, format='ascii')
+        self.lightCurveTable["mle"] = lc_table
 
         return str(lcOutputFilePath)
 
@@ -1043,9 +1049,20 @@ def aperturePhotometry(self):
         products = apTool.call()
         self.logger.info( f"Science tool AP produced:\n {products}")
 
-        self.lightCurveData["ap"] = products[0]
+        try:
+            self.lightCurveData["ap"] = products[0]
+            file_name = products[0]
+            file_optional = products[1]
+        except KeyError:
+            # This happens if the file already exists
+            file_name = self.lightCurveData["ap"]
+            file_optional = None
+
+        # Read table
+        lc_table = Table.read(file_name, format='ascii', names=["tmin_TT","tmax_TT","exposure","counts"])
+        self.lightCurveTable["ap"] = lc_table
 
-        return products[0], products[1] 
+        return file_name, file_optional
 
 
     ############################################################################
@@ -1381,11 +1398,11 @@ def _updateFovMinMaxValues(fovbinnumber, fovradmin, fovradmax, stepi):
 
     """
     The light curve data generation could be parallalezed..
-    def _computeLcBin(self, threadID, bins, configBKP, lcAnalysisDataDir, logsOutDir, logFilenamePrefix, verboseLvl):
+    def _computeLcBin(self, threadID, bins, configBKP, lcAnalysisDataDir, logsOutDir, verboseLvl):
 
         logger = AgilepyLogger()
 
-        logger.initialize(logsOutDir, logFilenamePrefix, verboseLvl)
+        logger.initialize(logsOutDir, verboseLvl)
 
         # outputs = []
 

agilepy/api/AGAnalysisWavelet.py

@@ -97,7 +97,6 @@ def getConfiguration(confFilePath, userName, outputDir, verboselvl, ctsmap, scal
 output:
   outdir: %s
   filenameprefix: wavelet_product
-  logfilenameprefix: eng_log
   username: %s
   sourcename: wavelet
   verboselvl: %d