Concatenation Error in NRLMSISE-00 Example

[OrbitalMechanic]

I've installed and attempting to run the NRLMSISE-00 Exalpe to test my installation. I'm running the Annaconda Distribution of Python 3.8 on a Mac Pro Desktop (2019) under Mac OSX Monterey (ver. 12.0.1).

The source code I'm running is:

from` pyatmos import download_sw_nrlmsise00,read_sw_nrlmsise00
# Download or update the space weather file from www.celestrak.com
swfile = download_sw_nrlmsise00() 
# Read the space weather data
swdata = read_sw_nrlmsise00(swfile) 

from pyatmos import nrlmsise00
# Set a specific time and location
t = '2014-07-22 22:18:45' # time(UTC) 
lat,lon,alt = 25,102,600 # latitude, longitude in [degree], and altitude in [km]
nrl00 = nrlmsise00(t,(lat,lon,alt),swdata)
print(nrl00.rho) # [kg/m^3]
print(nrl00.T) # [K]
print(nrl00.nd) # composition in [1/m^3]

The output I get is:

The space weather data in /Users/user/src/sw-data/ is already the latest.
Traceback (most recent call last):

  File "/Users/user/Various_Python_Examples/NRLMSISE-00_Example/NRLMSIME-00_ex.py", line 17, in <module>
    swdata = read_sw_nrlmsise00(swfile)

  File "/Users/user/opt/anaconda3/lib/python3.8/site-packages/pyatmos/msise/spaceweather.py", line 94, in read_sw_nrlmsise00
    SW_OBS_PRE = np.vstack((np.array(SW_OBS),np.array(SW_PRE)))

  File "<__array_function__ internals>", line 5, in vstack

  File "/Users/user/opt/anaconda3/lib/python3.8/site-packages/numpy/core/shape_base.py", line 282, in vstack
    return _nx.concatenate(arrs, 0)

  File "<__array_function__ internals>", line 5, in concatenate

ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 29 and the array at index 1 has size 32


In [17]: runfile('/Users/user/Various_Python_Examples/NRLMSISE-00_Example/NRLMSIME-00_ex.py', wdir='/Users/user/Various_Python_Examples/NRLMSISE-00_Example')
The space weather data in /Users/user/src/sw-data/ is already the latest.
Traceback (most recent call last):

  File "/Users/user/Various_Python_Examples/NRLMSISE-00_Example/NRLMSIME-00_ex.py", line 17, in <module>
    swdata = read_sw_nrlmsise00(swfile)

  File "/Users/user/opt/anaconda3/lib/python3.8/site-packages/pyatmos/msise/spaceweather.py", line 94, in read_sw_nrlmsise00
    SW_OBS_PRE = np.vstack((np.array(SW_OBS),np.array(SW_PRE)))

  File "<__array_function__ internals>", line 5, in vstack

  File "/Users/user/opt/anaconda3/lib/python3.8/site-packages/numpy/core/shape_base.py", line 282, in vstack
    return _nx.concatenate(arrs, 0)

  File "<__array_function__ internals>", line 5, in concatenate

ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 29 and the array at index 1 has size 32

Please advise.

Sam Dupree.

[OrbitalMechanic]

One further observation; in running the JB2008 example:

from pyatmos import download_sw_jb2008,read_sw_jb2008
# Download or update the space weather file from https://sol.spacenvironment.net
swfile = download_sw_jb2008() 
# Read the space weather data
swdata = read_sw_jb2008(swfile)

from pyatmos import jb2008
# Set a specific time and location
t = '2014-07-22 22:18:45' # time(UTC) 
lat,lon,alt = 25,102,600 # latitude, longitude in [degree], and altitude in [km]
jb08 = jb2008(t,(lat,lon,alt),swdata)
print(jb08.rho) # [kg/m^3]
print(jb08.T) # [K] 

I get the result

The space weather data in /Users/user/src/sw-data/ is already the latest.
1.3125893865998707e-14
749.4883243811

instead of

1.2991711750265394e-14
754.2803276187265

Please advise.

Samuel Dupree.

[aprater11]

I'm running into this same issue on my Windows device, with python 3.7.
@OrbitalMechanic : Were you able to reach a resolution here regarding NRLMSISE-00?

EDIT:
Some sleuthing uncovered that this issue is presented during np.vstack, in the read_sw_nrlmsise00 function. The shapes of the SW_OBS (observed data) and the SW_PRE (predicted data) arrays are inconsistent. As a simple workaround, if you only care about either historical or predicted data, you can update line 94 to be...
SW_OBS_PRE = (np.array(SW_OBS)) # -- modified line!

Otherwise, there is probably some clever manipulation that can be done to the spaceweather file to allow these arrays to be combined.

[Cokerush21]

@OrbitalMechanic @aprater11 @lcx366

I've tried the solution that aprater11 suggested. It didn't work until I made the following modification:

SW_OBS_PRE = np.array(SW_OBS) ---> The difference is subtle

To sum up, it finally worked.

If you find another solution to avoid keeping separating observation and prediction data, I'll be around

[aprater11]

I ended up doing something similar -- the issue lies with the predictive spaceweather data (SW_OBS_PRE). My application doesn't need the predictive information, but it still isn't ideal. Best, *Austin Prater* *360.584.7451*

…

On Wed, Mar 9, 2022 at 6:48 PM Cokerush21 ***@***.***> wrote: @OrbitalMechanic <https://github.com/OrbitalMechanic> @aprater11 <https://github.com/aprater11> @lcx366 <https://github.com/lcx366> I've tried the solution that OrbitalMechanic suggested. It didn't work until I made the following modification: SW_OBS_PRE = np.array(SW_OBS) ---> The difference is subtle To sum up, it finally worked. If you find another solution to avoid keeping separating observation and prediction data, I'll be around — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXMS4DMDGWHXUKSQH3BENJTU7FPH5ANCNFSM5I46ON2A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Bownnw]

As you check the spaceweather.py and the SW-ALL.TXT data, you will find the data column are not fitted.

I solve this problem by:

change the function read_sw_nrlmsise00

line 84, sw_p = np.delete(sw_p,[23,25])
line 86, sw_p = np.delete(sw_p,[27])

ROINT

[OrbitalMechanic]

I used the solution posted by Cokerush21, and it worked.

Thanks.

Sam Dupree.

[aaronjswank]

in spaceweather.py replace line:

if flag2 == 1: SW_PRE.append(line.split())

with:

        if flag2 == 1:
            sw_p = line.split()
            if (len(sw_p) == 32):
                sw_p = np.delete(sw_p,[23,24,25])
            else:
                sw_p = np.delete(sw_p,[23,24,25,27])
            SW_PRE.append(sw_p)

[Cokerush21]

Does anybody have a problem now? I mean, it worked a week ago but it seems that after updating the weather file it has another problem similar to the previous one.

After the solution I suggested before, I tried the solution of @aaronjswank. It worked without any other required change in spaceweather.py...but now it doesn't. I went back to my previous solution and it's also failing

This is the output I get with @aaronjswank solution:

C:\Users\coker\AppData\Local\Programs\Python\Python39\lib\site-packages\pyatmos\msise\spaceweather.py:100: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant
to do this, you must specify 'dtype=object' when creating the ndarray.
SW_OBS_PRE = np.vstack((np.array(SW_OBS),np.array(SW_PRE)))
Traceback (most recent call last):
File "c:\Users\coker\Desktop\Projectes VSCode\pyatmos00.py", line 5, in
swdata = read_sw_nrlmsise00(swfile)
File "C:\Users\coker\AppData\Local\Programs\Python\Python39\lib\site-packages\pyatmos\msise\spaceweather.py", line 100, in read_sw_nrlmsise00
SW_OBS_PRE = np.vstack((np.array(SW_OBS),np.array(SW_PRE)))
File "<array_function internals>", line 5, in vstack
File "C:\Users\coker\AppData\Local\Programs\Python\Python39\lib\site-packages\numpy\core\shape_base.py", line 282, in vstack
return _nx.concatenate(arrs, 0)
File "<array_function internals>", line 5, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 23550 and the array at index 1 has size 29

if anyone knows something, please...I'd appreciate your help

ps. If you guys can run this atmosphere model with no problem, could you please paste your spaceweather code on this subject? and numpy (this last one is because I'm suspecting of the vstack thing...)

@lcx366 ???

[lcx366]

The issue is solved in the new version pyatmos 1.2.4 by using the new released space weather file 'SW-ALL.csv'.

The result for the JB2008 example showing

1.3125893865998707e-14
749.4883243811

instead of

1.2991711750265394e-14
754.2803276187265

is caused by the updated weather file, which is slightly different from the previous one.