Skip to main content
`Ctrl`+`K`

# Dates and Times

We already met the `datetime` module in Web Access for handling points of time and time durations. Pandas extends those capabilities by introducing time periods (durations associated with a point) and more advanced calendar arithmetics.

Pandas also provides date and time related index objects to easily index time series data: `DatetimeIndex`, `TimedeltaIndex`, `PeriodIndex`.

    import pandas as pd

## Time Stamps

The basic data structure for representing points in time are `Timestamp` objects. They provide lots of useful methods for conversion from and to other date and time formats.

    some_day = pd.Timestamp(year=2020, month=2, day=15, hour=12, minute=34)
    some_day

    Timestamp('2020-02-15 12:34:00')

## Time Deltas

The basic data structure for representing durations are `Timedelta` objects. They can be used in their own or to shift time stamps.

    a_long_time = pd.Timedelta(days=10000, minutes=100)
    a_long_time

    Timedelta('10000 days 01:40:00')

## Periods

A period in Pandas is a time interval paired with a time stamp. Interpretation is as follows:

- The interval is one of several preset intervals, like a calendar month or a week from Monday till Sunday. See Offset aliases and Anchored aliases for available intervals.

- The time stamp selects a concrete interval, the month or the week containing the time stamp, for instance.

The basic data structure for representing periods are `Period` objects.

    year2010 = pd.Period('1/1/2010', freq='A')
    print(year2010.start_time)
    print(year2010.end_time)

    2010-01-01 00:00:00
    2010-12-31 23:59:59.999999999

    year2010oct = pd.Period('1/1/2010', freq='A-OCT')
    print(year2010oct.start_time)
    print(year2010oct.end_time)

    2009-11-01 00:00:00
    2010-10-31 23:59:59.999999999

## Time Stamp Indices

Using time stamps for indexing offers lots of nice features in Pandas, because Pandas originally has been developed for handling time series.

### Creating Time Stamp Indices

The constructor for `DatetimeIndex` objects takes a list of time stamps and an optional frequency. Frequency has to match the passed time stamps. If there is no common frequency in the data, the frequency is `None` (default).

A more convenient method is `pd.date_range`:

    index = pd.date_range(start='2018-03-14', freq='2D12H', periods=10)
    index

    DatetimeIndex(['2018-03-14 00:00:00', '2018-03-16 12:00:00',
                   '2018-03-19 00:00:00', '2018-03-21 12:00:00',
                   '2018-03-24 00:00:00', '2018-03-26 12:00:00',
                   '2018-03-29 00:00:00', '2018-03-31 12:00:00',
                   '2018-04-03 00:00:00', '2018-04-05 12:00:00'],
                  dtype='datetime64[ns]', freq='60H')

    index = pd.date_range(start='2018-03-14', end='2018-03-20', freq='D')
    index

    DatetimeIndex(['2018-03-14', '2018-03-15', '2018-03-16', '2018-03-17',
                   '2018-03-18', '2018-03-19', '2018-03-20'],
                  dtype='datetime64[ns]', freq='D')

We may also use columns of an existing data frame to create a `DatetimeIndex`:

    df = pd.DataFrame({'day': [1, 2, 3, 4], 'month': [4, 6, 9, 9], 'year': [2018, 2018, 2020, 2020]})
    display(df)

    pd.to_datetime(df)

| day | month | year
--- | --- | --- | ---
0 | 1 | 4 | 2018
1 | 2 | 6 | 2018
2 | 3 | 9 | 2020
3 | 4 | 9 | 2020

    0   2018-04-01
    1   2018-06-02
    2   2020-09-03
    3   2020-09-04
    dtype: datetime64[ns]

The `to_datetime` function expects that columns are named `'day'`, `'month'`, `'year'`. It returns a series of type `datetime64` which may be converted to a `DatetimeIndex`.

### Indexing

#### Exact Indexing

Using `Timestamp` objects for label based indexing yields items with the corresponding time stamp, if there are any. Slicing works as usual.

    index = pd.date_range(start='2018-03-14', freq='D', periods=10)
    s = pd.Series([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], index=index)
    print(s, '\n')

    print(s.loc[pd.Timestamp('2018-3-16')], '\n')
    #print(s.loc[pd.Timestamp('2018-3-16 10:00')], '\n')    # KeyError
    print(s.loc[pd.Timestamp('2018-3-16 00:00')], '\n')
    print(s.loc[pd.Timestamp('2018-3-16'):pd.Timestamp('2018-3-20')])

    2018-03-14     1
    2018-03-15     2
    2018-03-16     3
    2018-03-17     4
    2018-03-18     5
    2018-03-19     6
    2018-03-20     7
    2018-03-21     8
    2018-03-22     9
    2018-03-23    10
    Freq: D, dtype: int64

    3

    3

    2018-03-16    3
    2018-03-17    4
    2018-03-18    5
    2018-03-19    6
    2018-03-20    7
    Freq: D, dtype: int64

#### Inexact Indexing

Passing strings containing partial dates/times selects time ranges. This technique is referred to as partial string indexing. Slicing is allowed.

    index = pd.date_range(start='2018-03-14', freq='D', periods=100)
    s = pd.Series(range(1, len(index) + 1), index=index)
    print(s, '\n')

    print(s.loc['2018-3'], '\n')
    print(s.loc['2018-3':'2018-4'])

    2018-03-14      1
    2018-03-15      2
    2018-03-16      3
    2018-03-17      4
    2018-03-18      5
                 ...
    2018-06-17     96
    2018-06-18     97
    2018-06-19     98
    2018-06-20     99
    2018-06-21    100
    Freq: D, Length: 100, dtype: int64

    2018-03-14     1
    2018-03-15     2
    2018-03-16     3
    2018-03-17     4
    2018-03-18     5
    2018-03-19     6
    2018-03-20     7
    2018-03-21     8
    2018-03-22     9
    2018-03-23    10
    2018-03-24    11
    2018-03-25    12
    2018-03-26    13
    2018-03-27    14
    2018-03-28    15
    2018-03-29    16
    2018-03-30    17
    2018-03-31    18
    Freq: D, dtype: int64

    2018-03-14     1
    2018-03-15     2
    2018-03-16     3
    2018-03-17     4
    2018-03-18     5
    2018-03-19     6
    2018-03-20     7
    2018-03-21     8
    2018-03-22     9
    2018-03-23    10
    2018-03-24    11
    2018-03-25    12
    2018-03-26    13
    2018-03-27    14
    2018-03-28    15
    2018-03-29    16
    2018-03-30    17
    2018-03-31    18
    2018-04-01    19
    2018-04-02    20
    2018-04-03    21
    2018-04-04    22
    2018-04-05    23
    2018-04-06    24
    2018-04-07    25
    2018-04-08    26
    2018-04-09    27
    2018-04-10    28
    2018-04-11    29
    2018-04-12    30
    2018-04-13    31
    2018-04-14    32
    2018-04-15    33
    2018-04-16    34
    2018-04-17    35
    2018-04-18    36
    2018-04-19    37
    2018-04-20    38
    2018-04-21    39
    2018-04-22    40
    2018-04-23    41
    2018-04-24    42
    2018-04-25    43
    2018-04-26    44
    2018-04-27    45
    2018-04-28    46
    2018-04-29    47
    2018-04-30    48
    Freq: D, dtype: int64

Inexact indexing has some pitfalls, which are described in Partial string indexing and Slice vs. exact match of the Pandas user guide.

### Useful Functions for Time Stamp Indexed Data

Pandas provides lots of functions for working with time stamp indices. Some are:

- `asfreq` (upsampling with fill values or filling logic)

- `shift` (shift index or data by some time period)

- `resample` (downsampling with aggregation, see below)

The `resample` method returns a `Resampler` object, which provides several methods for calculating data values at the new time stamps. Examples are `sum`, `mean`, `min`, `max`. All these methods return a series or a data frame.

    index = pd.date_range(start='2018-03-14', freq='D', periods=100)
    s = pd.Series(range(1, len(index) + 1), index=index)

    s2 = s.resample('5D').sum()
    s2

    2018-03-14     15
    2018-03-19     40
    2018-03-24     65
    2018-03-29     90
    2018-04-03    115
    2018-04-08    140
    2018-04-13    165
    2018-04-18    190
    2018-04-23    215
    2018-04-28    240
    2018-05-03    265
    2018-05-08    290
    2018-05-13    315
    2018-05-18    340
    2018-05-23    365
    2018-05-28    390
    2018-06-02    415
    2018-06-07    440
    2018-06-12    465
    2018-06-17    490
    Freq: 5D, dtype: int64

## Period indices

Period indices work analogously to time stamp indices. Corresponding class is `PeriodIndex`.

    index = pd.period_range(start='2018-03-14', freq='D', periods=10)
    print(index)

    s = pd.Series(range(1, len(index) + 1), index=index)
    s

    PeriodIndex(['2018-03-14', '2018-03-15', '2018-03-16', '2018-03-17',
                 '2018-03-18', '2018-03-19', '2018-03-20', '2018-03-21',
                 '2018-03-22', '2018-03-23'],
                dtype='period[D]')

    2018-03-14     1
    2018-03-15     2
    2018-03-16     3
    2018-03-17     4
    2018-03-18     5
    2018-03-19     6
    2018-03-20     7
    2018-03-21     8
    2018-03-22     9
    2018-03-23    10
    Freq: D, dtype: int64

Indexing with time stamps selects the appropriate period, like with `IntervalIndex` objects:

    s.loc[pd.Timestamp('2018-03-15 12:34')]

    2

With time stamp index the above line would lead to a `KeyError`. But for periods it’s interpreted as: select the period containing the time stamp.

Similar is possible with slicing:

    s.loc[pd.Timestamp('2018-03-15 12:34'):pd.Timestamp('2018-03-18 23:45')]

    2018-03-15    2
    2018-03-16    3
    2018-03-17    4
    2018-03-18    5
    Freq: D, dtype: int64

Methods `asfreq`, `shift`, `resample` also work for periods indices.