---
jupytext:
  formats: content_md///md:myst,content///ipynb
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.13.8
kernelspec:
  display_name: Python 3 (ipykernel)
  language: python
  name: python3
---

(managing-data:pandas:dates-times)=
# Dates and Times

We already met the `datetime` module in [](python:accessing-data:web-access) for handling points of time and time durations. Pandas extends those capabilities by introducing time periods (durations associated with a point) and more advanced calendar arithmetics.

Pandas also provides date and time related index objects to easily index time series data: `DatetimeIndex`, `TimedeltaIndex`, `PeriodIndex`.

```{code-cell} ipython3
import pandas as pd
```

## Time Stamps

The basic data structure for representing points in time are [`Timestamp`](https://pandas.pydata.org/docs/reference/api/pandas.Timestamp.html) objects. They provide lots of useful methods for conversion from and to other date and time formats.

```{code-cell} ipython3
some_day = pd.Timestamp(year=2020, month=2, day=15, hour=12, minute=34)
some_day
```

## Time Deltas

The basic data structure for representing durations are [`Timedelta`](https://pandas.pydata.org/docs/reference/api/pandas.Timedelta.html) objects. They can be used in their own or to shift time stamps.

```{code-cell} ipython3
a_long_time = pd.Timedelta(days=10000, minutes=100)
a_long_time
```

## Periods

A period in Pandas is a time interval paired with a time stamp. Interpretation is as follows:
* The interval is one of several preset intervals, like a calendar month or a week from Monday till Sunday. See [Offset aliases](https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#offset-aliases) and [Anchored aliases](https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#anchored-offsets) for available intervals.
* The time stamp selects a concrete interval, the month or the week containing the time stamp, for instance.

The basic data structure for representing periods are [`Period`](https://pandas.pydata.org/docs/reference/api/pandas.Period.html) objects.

```{code-cell} ipython3
year2010 = pd.Period('1/1/2010', freq='A')
print(year2010.start_time)
print(year2010.end_time)
```

```{code-cell} ipython3
year2010oct = pd.Period('1/1/2010', freq='A-OCT')
print(year2010oct.start_time)
print(year2010oct.end_time)
```

## Time Stamp Indices

Using time stamps for indexing offers lots of nice features in Pandas, because Pandas originally has been developed for handling time series.

### Creating Time Stamp Indices

The constructor for [`DatetimeIndex`](https://pandas.pydata.org/docs/reference/api/pandas.DatetimeIndex.html) objects takes a list of time stamps and an optional frequency. Frequency has to match the passed time stamps. If there is no common frequency in the data, the frequency is `None` (default).

A more convenient method is [`pd.date_range`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html):

```{code-cell} ipython3
index = pd.date_range(start='2018-03-14', freq='2D12H', periods=10)
index
```

```{code-cell} ipython3
index = pd.date_range(start='2018-03-14', end='2018-03-20', freq='D')
index
```

We may also use columns of an existing data frame to create a `DatetimeIndex`:

```{code-cell} ipython3
df = pd.DataFrame({'day': [1, 2, 3, 4], 'month': [4, 6, 9, 9], 'year': [2018, 2018, 2020, 2020]})
display(df)

pd.to_datetime(df)
```

The [`to_datetime`](https://pandas.pydata.org/docs/reference/api/pandas.to_datetime.html) function expects that columns are named `'day'`, `'month'`, `'year'`. It returns a series of type `datetime64` which may be converted to a `DatetimeIndex`.

### Indexing

#### Exact Indexing

Using `Timestamp` objects for label based indexing yields items with the corresponding time stamp, if there are any. Slicing works as usual.

```{code-cell} ipython3
index = pd.date_range(start='2018-03-14', freq='D', periods=10)
s = pd.Series([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], index=index)
print(s, '\n')

print(s.loc[pd.Timestamp('2018-3-16')], '\n')
#print(s.loc[pd.Timestamp('2018-3-16 10:00')], '\n')    # KeyError
print(s.loc[pd.Timestamp('2018-3-16 00:00')], '\n')
print(s.loc[pd.Timestamp('2018-3-16'):pd.Timestamp('2018-3-20')])
```

#### Inexact Indexing

Passing strings containing partial dates/times selects time ranges. This technique is referred to as *partial string indexing*. Slicing is allowed.

```{code-cell} ipython3
index = pd.date_range(start='2018-03-14', freq='D', periods=100)
s = pd.Series(range(1, len(index) + 1), index=index)
print(s, '\n')

print(s.loc['2018-3'], '\n')
print(s.loc['2018-3':'2018-4'])
```

Inexact indexing has some pitfalls, which are described in [Partial string indexing](https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#partial-string-indexing) and [Slice vs. exact match](https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#slice-vs-exact-match) of the Pandas user guide.

### Useful Functions for Time Stamp Indexed Data

Pandas provides lots of functions for working with time stamp indices. Some are:
* [`asfreq`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.asfreq.html) (upsampling with fill values or filling logic)
* [`shift`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.shift.html) (shift index or data by some time period)
* [`resample`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.resample.html) (downsampling with aggregation, see below)

The `resample` method returns a [`Resampler`](https://pandas.pydata.org/pandas-docs/stable/reference/resampling.html) object, which provides several methods for calculating data values at the new time stamps. Examples are `sum`, `mean`, `min`, `max`. All these methods return a series or a data frame.

```{code-cell} ipython3
index = pd.date_range(start='2018-03-14', freq='D', periods=100)
s = pd.Series(range(1, len(index) + 1), index=index)

s2 = s.resample('5D').sum()
s2
```

## Period indices

Period indices work analogously to time stamp indices. Corresponding class is [`PeriodIndex`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.PeriodIndex.html).

```{code-cell} ipython3
index = pd.period_range(start='2018-03-14', freq='D', periods=10)
print(index)

s = pd.Series(range(1, len(index) + 1), index=index)
s
```

Indexing with time stamps selects the appropriate period, like with `IntervalIndex` objects:

```{code-cell} ipython3
s.loc[pd.Timestamp('2018-03-15 12:34')]
```

With time stamp index the above line would lead to a `KeyError`. But for periods it's interpreted as: select the period containing the time stamp.

Similar is possible with slicing:

```{code-cell} ipython3
s.loc[pd.Timestamp('2018-03-15 12:34'):pd.Timestamp('2018-03-18 23:45')]
```

Methods `asfreq`, `shift`, `resample` also work for periods indices.
