look up time values from a pandas.DataFrame

[raybellwaves]

I've got some data stored in a dataframe. I would to use the dataframe to select points from a Dataset. It seems to struggle with time values.

import xarray as xr
import pandas as pd

ds = xr.tutorial.open_dataset("air_temperature")
df = pd.DataFrame({"lat": ds["lat"].values[0:2], "lon": ds["lon"].values[0:2], "time": ds["time"].values[0:2]})

ds.sel(time=df["time"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ray/miniconda3/envs/main/lib/python3.9/site-packages/xarray/core/dataset.py", line 2474, in sel
    pos_indexers, new_indexes = remap_label_indexers(
  File "/Users/ray/miniconda3/envs/main/lib/python3.9/site-packages/xarray/core/coordinates.py", line 421, in remap_label_indexers
    pos_indexers, new_indexes = indexing.remap_label_indexers(
  File "/Users/ray/miniconda3/envs/main/lib/python3.9/site-packages/xarray/core/indexing.py", line 117, in remap_label_indexers
    idxr, new_idx = index.query(labels, method=method, tolerance=tolerance)
  File "/Users/ray/miniconda3/envs/main/lib/python3.9/site-packages/xarray/core/indexes.py", line 200, in query
    raise ValueError(
ValueError: cannot use a dict-like object for selection on a dimension that does not have a MultiIndex

ds.sel(lat=df["lat"]) # works

[max-sixty]

Hi @raybellwaves ! Hope all's well.

Is this because the df['lat'] is returning a series? Adding .values seems to return something — is that what you're hoping for?

In [6]: ds.sel(time=df["time"].values)
Out[6]:
<xarray.Dataset>
Dimensions:  (lat: 25, time: 2, lon: 53)
Coordinates:
  * lat      (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 25.0 22.5 20.0 17.5 15.0
  * lon      (lon) float32 200.0 202.5 205.0 207.5 ... 322.5 325.0 327.5 330.0
  * time     (time) datetime64[ns] 2013-01-01 2013-01-01T06:00:00
Data variables:
    air      (time, lat, lon) float32 ...
Attributes:
    Conventions:  COARDS
    title:        4x daily NMC reanalysis (1948)
    description:  Data is from NMC initialized reanalysis\n(4x/day).  These a...
    platform:     Model
    references:   http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanaly...

[raybellwaves]

Thanks yes .values solves this issue. I'm more interested in what it'll take for it to work for a pandas datetime. I believe time in xarray in determined by time (types) in numpy hence you have to do when wanting to pandas datetime operations you have to do .to_index() first. e.g. changing time to a local timezone:

ds = ds.assign_coords(
    {
        "time": ds["time"]
        .to_index()
        .tz_localize("UTC")
        .tz_convert("America/Los_Angeles")
        .tz_localize(None)
        .values
    }
)

If there are known issues on this welcome to link them here. This probably falls under https://github.com/pydata/xarray/issues?q=is%3Aopen+is%3Aissue+label%3Atopic-indexing and possibly #5692

[SayanthRock]

Thanks yes values solves this issue. I'm more interested in what it'll take for it to work for a pandas datetime. I believe time in xarray in determined by time (types) in numpy hence you have to do when wanting to pandas datetime operations you have to do to index() first. e.g. changing time to a local timezone:
):
time_index = ds.indexes["time"]

if time_index.tz is None:
    time_index = time_index.tz_localize(source_tz)

converted_time = (
    time_index
    .tz_convert(target_tz)
    .tz_localize(None)
)

return ds.assign_coords(time=converted_time)

ds = convert_time_coordinate(ds)