10.1 Time Series Concepts: Trend, Seasonality, Stationarity
Alright, let’s cut through the noise. You’ve got a list of dates and values. Your boss wants to know what happens next. Before you can even think about throwing a fancy neural net or an ARIMA model at it, you need to understand the three pillars holding your data up: Trend, Seasonality, and Stationarity. Get these wrong, and your forecast is just a beautifully formatted lie.
Think of it like this: your time series data is a smoothie. Trend is the main fruit, seasonality is the ice that makes it cyclical, and stationarity is you deciding whether you need to blend it again to get a consistent texture. We’re about to become master smoothie critics.
What is Trend, Really?
A trend is the long-term, underlying direction of your data. It’s not about the little ups and downs from day to day; it’s about whether, over years or quarters, the value is generally going up, down, or staying flat. Is your website’s user base growing? That’s an upward trend. Is the use of a legacy technology slowly dying? That’s a downward trend.
The key here is “long-term.” What constitutes “long-term” depends entirely on your data. A trend in five years of daily stock data is different from a trend in one hour of millisecond-scale sensor readings. Humans are terrible at visually separating trend from noise, so we use math. Let’s use a classic: the rolling mean.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# Let's create some dummy data with a clear upward trend and a lot of noise
np.random.seed(42) # For reproducibility, which is nice for books, less so for real life
dates = pd.date_range(start='2010-01-01', end='2020-01-01', freq='M')
trend = np.linspace(0, 100, len(dates)) # A straight line from 0 to 100
noise = np.random.normal(0, 15, len(dates)) # Mean 0, Std Dev 15
values = trend + noise
series = pd.Series(values, index=dates)
# Calculate a 12-month (one year) rolling mean to smooth out the seasonal-ish noise
rolling_mean = series.rolling(window=12).mean()
# Plot the raw data and the trend we extracted
fig, ax = plt.subplots(figsize=(10, 6))
series.plot(ax=ax, label='Raw Data', alpha=0.5)
rolling_mean.plot(ax=ax, label='12-Month Rolling Mean (Trend)', color='red', linewidth=3)
ax.set_title('Separating Trend from Noise')
ax.legend()
plt.show()
That rolling mean line? That’s your trend. It’s blissfully ignorant of the chaos happening around it, just plodding along its predetermined path. Choosing the window size (window=12 here) is a critical decision—too small, and your “trend” is still noisy; too large, and you might smooth out meaningful long-term changes. It’s an art.
The Rhythms of Your Data: Seasonality
Seasonality is the predictable, periodic fluctuation in your data. It repeats at a fixed frequency. The name gives away the most common example: retail sales spike every December (holiday season), drop in January, and maybe have another smaller bump around Black Friday. But “season” can mean any regular interval: daily (website traffic peaks at 2 PM), weekly (low usage on weekends), hourly (power grid demand).
Seasonality is what makes time series forecasting interesting. If there was only a trend, you could just draw a line. Seasonality adds the repeating patterns you can exploit.
# Let's add a strong annual seasonality to our trending data
seasonal_component = 10 * np.sin(2 * np.pi * np.arange(len(dates)) / 12) # Annual sine wave
series_with_seasonality = series + seasonal_component
# Let's plot a few years to see the pattern
ax = series_with_seasonality['2015':'2018'].plot(figsize=(10, 6))
ax.set_title('A Couple Years of Data Showing Clear Seasonality')
plt.show()
See those regular waves? That’s your seasonality. A model that understands this pattern can confidently predict that the next December will be high, even if the raw value for last month was low. The big mistake here is confusing seasonality for cycles. Cycles are rises and falls that don’t have a fixed frequency (e.g., economic booms and busts). Seasonality is clockwork.
The Bedrock Concept: Stationarity
This is the big one, the concept that newbies stumble over and that makes statisticians nod sagely. Stationarity is the statistical properties of your series—specifically its mean and variance—being constant over time.
Why do we care? Because most classical time series forecasting models (like the whole ARIMA family) are built on the mathematical assumption that your data is stationary. Feeding non-stationary data into one of these models is like putting diesel in a gasoline engine; it might run for a bit, but it’s going to seize up catastrophically.
A series with a clear trend is non-stationary because its mean changes over time (the mean of the first year is much lower than the mean of the last year). A series with changing volatility (like periods of calm followed by periods of wild swings) is non-stationary because its variance changes.
The formal statistical test for this is the Augmented Dickey-Fuller (ADF) test. The null hypothesis of the ADF test is that the series is non-stationary. So you’re rooting for a p-value below a threshold (typically 0.05) to reject the null and conclude your data is stationary.
from statsmodels.tsa.stattools import adfuller
# Test our original trending series (should be NON-stationary)
adf_result = adfuller(series)
print(f'ADF Statistic for Trending Series: {adf_result[0]:.2f}')
print(f'p-value: {adf_result[1]:.4f}')
if adf_result[1] > 0.05:
print("-> Fail to reject null hypothesis. Series is NON-stationary. (Duh)")
# Now, let's make it stationary by taking the first difference: Δy_t = y_t - y_{t-1}
diffed_series = series.diff().dropna() # .diff() creates a NaN for the first element, so we drop it
adf_result_diff = adfuller(diffed_series)
print(f'\nADF Statistic for Differenced Series: {adf_result_diff[0]:.2f}')
print(f'p-value: {adf_result_diff[1]:.4f}')
if adf_result_diff[1] <= 0.05:
print("-> Reject null hypothesis. Series is STATIONARY.")
Differencing is the most common way to annihilate a trend and achieve stationarity. You’re no longer modeling the actual value, but the change from one time point to the next. It’s a classic case of “if the mountain won’t come to Muhammad…”; if the data isn’t stationary, we transform the data until it is. For data with both trend and seasonality, you might need both regular differencing and seasonal differencing. It’s a whole thing.
The pitfall? Over-differencing. You can difference a series that’s already stationary and accidentally introduce a structure that models will waste effort trying to learn. Always test. Always plot. The ADF test isn’t perfect, but it’s your first line of defense against building a model on a foundation of sand.