import numpy as np
import pandas as pd
import plotly.graph_objects as go
from plotly.subplots import make_subplots
from scipy import stats
import fe1tools as fe
fe.setup() # plot theme, display options, random seed
BLUE, ORANGE, AQUA = fe.PALETTE["blue"], fe.PALETTE["orange"], fe.PALETTE["aqua"]
prices = fe.load_prices(["SPY"])
spy = fe.returns(prices)["SPY"]1 Week 0
Pre-course material. Work through this before Week 1. It assumes no programming and no econometrics beyond school statistics, and it establishes the vocabulary — estimand, estimator, sampling distribution, standard error — that every later week takes for granted. The five questions at the end are a self-check: if you can answer them, you are ready to start.
Learning objectives. After this lesson you will be able to
- distinguish an estimand from an estimator and an estimate, and explain why every number on a performance report is a draw from a distribution;
- derive and simulate the sampling distribution of the sample mean and the sample variance, and state the standard error of each;
- explain why the sample mean becomes normally distributed even when returns are violently non-normal, and why the sample variance does not;
- tell a standard deviation from a standard error, and read a confidence interval correctly;
- obtain the same two estimators three ways — least squares, method of moments and maximum likelihood — and name where each principle returns later in the course;
- recognise the sample mean and variance as the simplest special cases of a linear regression, a time-series model and a volatility model.
1.1 Introduction of modern tools of estimation and inference
A fund reports two numbers for its equity allocation over the past twenty years: an average return of 11.7% a year and a volatility of 19.2%. Both are used as inputs to decisions — the first to a discount rate or a funding assumption, the second to a risk limit.
Neither is a fact about the world. Each is a number computed from one finite stretch of history by applying a rule to data. Change the stretch of history and the number changes. The subject of this lesson is how much such a number is worth: what it is an estimate of, how far from the truth it is likely to be, and why the answer is completely different for the first number than for the second.
Every chart in this lesson is interactive: hover to read values, drag to zoom, double-click to reset.
1.1.1 Which average? A first estimation problem
Before asking how precise the 11.7% is, we should ask what it measures. “Average return” is ambiguous, and the ambiguity is worth real money.
years = len(spy) / 252
arith = spy.mean() * 252 # arithmetic mean
logmean = np.log1p(spy).mean() * 252 # mean log return
px = prices["SPY"]
cagr = (px.iloc[-1] / px.iloc[0]) ** (1/years) - 1 # realised compound rate
vol = spy.std(ddof=0) * np.sqrt(252)
pd.DataFrame({
"annualised %": [arith*100, logmean*100, cagr*100, vol*100],
}, index=["arithmetic mean", "mean log return", "compound growth rate (CAGR)",
"volatility"]).round(2)| annualised % | |
|---|---|
| arithmetic mean | 11.70 |
| mean log return | 9.85 |
| compound growth rate (CAGR) | 10.35 |
| volatility | 19.21 |
The arithmetic mean is 11.70% a year, but £1 invested at the start did not grow at 11.70% a year. It grew at the compound rate of 10.35%. The two differ because returns compound multiplicatively while the arithmetic mean adds: a \(-50\%\) year followed by a \(+50\%\) year averages zero arithmetically and loses a quarter of your capital.
The gap is not arbitrary. For small returns,
\[ \mathbb{E}\!\left[\ln(1+R)\right] \;\approx\; \mathbb{E}[R] - \tfrac{1}{2}\sigma^2 , \tag{1.1}\]
so the compounding penalty is governed by the variance. Here \(\tfrac12\sigma^2 = \tfrac12(0.192)^2 = 1.85\%\), and indeed \(11.70\% - 1.85\% = 9.85\%\), which is the mean log return in the table to two decimal places. The compound rate follows as \(e^{0.0985}-1 = 10.35\%\).
Two lessons are already visible, and both run through the whole course. The first is that the first and second moments are not independent concerns: volatility does not merely add uncertainty around a central path, it lowers the central path. The second is that using the wrong average compounds the error. Projecting £100 forward thirty years at 11.70% gives about £2,760; at the realised compound rate it gives about £1,920. The arithmetic figure is the right answer to a different question — the expected terminal wealth of a highly skewed distribution — and the wrong answer to “what will this typically grow to”.
Estimate, then assumption. A forward-looking figure such as 7% is not an estimate of this sample at all; it is an assumption about the future, usually built as a risk-free rate plus an equity risk premium, and defended on grounds this data cannot settle — current valuations, or a view that the sample period was unusually kind. This lesson is about the estimate. Distinguishing the two, and refusing to let a historical average pose as a forecast, is the first discipline of the subject.
1.1.2 What makes financial econometrics its own subject
Financial data are not simply economic data measured more often. Four features shape every method in this course, and they explain why we begin where we do.
The signal in the mean is minute; the signal in the variance is large. A typical daily return is a fraction of a percent against a daily standard deviation many times larger. Expected returns are buried in noise, while volatility is comparatively easy to see. This asymmetry, quantified below and revisited in Section 1.9, is why the risk-management half of finance is far more empirically successful than the expected-return half.
Levels are close to unpredictable; variances are strongly predictable. If returns were easily forecastable the forecast would be traded away. No such force disciplines volatility, which clusters persistently — the fact that makes Section 1.7’ GARCH model work.
The distributions are not normal. Daily equity returns are fat-tailed and mildly skewed. Any procedure that quietly assumes normality is making a claim the data reject, so we must know which of our tools depend on it and which do not.
The sample cannot be enlarged at will. History arrives at one year per year. Worse, lengthening the sample eventually reaches back to a different economic regime, so more data is not unambiguously better — a tension with no clean resolution, only judgement.
1.1.3 The statistical model behind every estimate
Standard statistics imagines drawing a sample from a population: many possible draws, one taken. Financial data do not work that way. There is one history, it happened once, and we cannot resample it. So what exactly is being estimated?
The modelling answer is to treat the observed returns as one realisation of a stochastic process \(\{R_t\}\) — a sequence of random variables, only one draw of which we ever observe. The estimand \(\mu\) is a property of that process, not of our data.
For this to be workable, two assumptions do the heavy lifting.
Stationarity requires the process’s statistical properties to be stable over time. In the weak (covariance-stationary) form used throughout this course, \(\mathbb{E}[R_t] = \mu\) and \(\operatorname{Var}(R_t) = \sigma^2\) for all \(t\), and \(\operatorname{Cov}(R_t, R_{t-k})\) depends on \(k\) alone. Without it, “the mean return” is not a well-defined target: there would be a different \(\mu\) each period and nothing for a single number to estimate.
Ergodicity requires that averaging over time recovers what averaging over repeated draws would have given. This is the assumption that lets one history substitute for the many we cannot observe. It is what licenses computing a sample mean at all.
These two assumptions are the price of admission, and both are visibly imperfect here. Section 1.2 will show the volatility estimate shifting in a way a constant \(\sigma\) struggles to explain, and Section 1.10 shows returns are not independent across time. Much of the rest of the course consists of relaxing these assumptions in a controlled way: allowing the variance to move (volatility models), allowing dependence (time-series models), allowing the mean to depend on covariates (regression). Naming them now makes those extensions legible rather than arbitrary.
1.1.4 Estimand, estimator, estimate
Three ideas are routinely collapsed into one, and prising them apart is the single most useful habit in econometrics.
| Term | Symbol | What it is |
|---|---|---|
| Estimand | \(\mu\), \(\sigma^2\) | The unknown property of the process we want to know |
| Estimator | \(\bar{X}\), \(s^2\) | A rule mapping data to a guess. A random variable |
| Estimate | 11.7%, 19.2% | The number that rule returned on this realisation |
The estimand is fixed and unknown. The estimate is known and, in a specific sense, wrong. The estimator sits between them, and because it is a function of random data it carries a distribution of its own — the sampling distribution. That distribution is the central object of this lesson, because it is what converts a number into a number-with-a-margin.
The distinction also separates the two activities named in the title of this section. Estimation produces the point guess. Inference quantifies how far that guess might be from the estimand, and it is inference, not estimation, that requires the sampling distribution.
The structure never changes. When we later write \(\hat{\beta}\) for a regression slope, \(\hat{\phi}\) for an autoregressive coefficient, or \(\hat{\alpha}\) for a GARCH parameter, each means exactly what \(\bar{X}\) means here: a rule applied to one realisation of a random process, carrying a distribution with it.
1.1.5 How an estimator is judged
Any number of rules could serve as an estimator of \(\mu\): the sample mean, the median, the first observation, the constant 7%. Four criteria decide between them, and the rest of this lesson establishes each in turn for \(\bar{X}\) and \(s^2\).
| Property | Question it answers | Where it appears |
|---|---|---|
| Unbiasedness | Is it right on average across realisations? \(\mathbb{E}[\hat\theta] = \theta\) | eq. 1.2, eq. 1.7 |
| Consistency | Does it converge to the truth as \(n\) grows? | Section 1.2, the law of large numbers |
| Efficiency | Among sensible rules, does it have the smallest variance? | eq. 1.3 and its \(1/\sqrt{n}\) rate |
| Asymptotic distribution | What shape does its error take, so we can build intervals? | eq. 1.4, the central limit theorem |
They are not interchangeable, and the order matters. Unbiasedness is the weakest of the four: an unbiased estimator with enormous variance is useless, as the average return will shortly demonstrate. Consistency without a known asymptotic distribution gives a point but no interval. It is the fourth property that makes inference possible at all.
1.1.6 Why the mean and the variance first
It is fair to ask why a graduate course opens with two statistics from school mathematics. The reason is not that they are easy but that they are general. Every model in this course estimates a mean or a variance; what changes is only what the moment is allowed to depend on.
| Model | What it estimates | Lesson |
|---|---|---|
| Sample mean \(\bar{X}\) | an unconditional mean | this one |
| Linear regression | a mean conditional on covariates | factor models |
| ARMA | a mean conditional on the past | time-series models |
| Sample variance \(s^2\) | an unconditional variance | this one |
| GARCH | a variance conditional on the past | volatility models |
Because the mean and variance carry no conditioning structure to distract us, their sampling distributions can be derived on paper, simulated in a few lines, and seen in a picture. The concepts established that way — bias, consistency, standard errors, asymptotic normality — transfer unchanged to every row of that table. Section 1.7 makes the transfer explicit by showing that the sample mean is a regression and the sample variance is a fitted volatility model, in the exact sense of producing identical numbers.
1.1.7 The experiment we cannot run
To see a sampling distribution we would need to rerun the last twenty years many times and record the estimate each time. History declines to cooperate.
The workaround is to build an artificial population whose parameters we know exactly, by treating the realised sample as if it were the whole process, and then draw repeated samples from it. Because we set the parameters, we can check the estimates against the truth — something never possible with real data. This is the logic of the bootstrap, and it turns the theorems that follow into pictures.
The device is not free: resampling observations independently discards the time dependence that Section 1.10 shows is present, so it understates some uncertainties. We return to that there, and repair it.
# Treat the realised sample as the "population" for the experiment.
mu_pop = spy.mean() # daily mean
sig_pop = spy.std(ddof=0) # daily volatility
kappa = stats.kurtosis(spy, fisher=False) # kurtosis (normal = 3)
print(f"observations {len(spy):>10,d} ({len(spy)/252:.1f} years)")
print(f"mean, annualised {mu_pop*252:>10.2%}")
print(f"volatility, ann. {sig_pop*np.sqrt(252):>10.2%}")
print(f"kurtosis {kappa:>10.2f} (normal = 3)")
print(f"skewness {stats.skew(spy):>10.2f}")observations 5,153 (20.4 years)
mean, annualised 11.70%
volatility, ann. 19.21%
kurtosis 18.18 (normal = 3)
skewness 0.00Note the kurtosis of 18 against a normal-distribution value of 3. Daily equity returns are nothing like normal, and the skewness is near zero. Both numbers matter later, and in ways that are easy to get backwards.
1.2 Watching an estimate settle down
Before any theory, look at what actually happens to these two statistics as data accumulate. At the end of each day we recompute the average return and the volatility using everything observed so far, annualise both, and plot them. Around each running estimate we draw the band that our eventual formulas will justify. The picture makes the central message of the lesson visible before we have proved anything.
t = np.arange(1, len(spy) + 1) # observations so far
run_vol = spy.expanding().std(ddof=1) * np.sqrt(252)
run_mean = spy.expanding().mean() * 252
se_mean = run_vol / np.sqrt(t / 252) # sigma / sqrt(years)
se_vol = run_vol * np.sqrt((kappa - 1) / (4 * t))
burn = 60 # ignore the first weeks
idx = spy.index[burn:].to_pydatetime() # native datetimes for plotly
xband = list(idx) + list(idx[::-1])
fig = make_subplots(rows=1, cols=2, horizontal_spacing=0.12,
subplot_titles=("Average return, annualised",
"Volatility, annualised"))
for col, (est, se, hue, key, label) in enumerate([
(run_mean, se_mean, BLUE, "blue", "Average return"),
(run_vol, se_vol, ORANGE, "orange", "Volatility")], start=1):
lo = ((est - 1.96*se).iloc[burn:] * 100).to_numpy()
hi = ((est + 1.96*se).iloc[burn:] * 100).to_numpy()
fig.add_trace(go.Scatter( # the shaded band
x=xband, y=list(hi) + list(lo[::-1]), fill="toself",
fillcolor=fe.band_colour(key, 0.18), line=dict(width=0),
hoverinfo="skip", showlegend=False), row=1, col=col)
fig.add_trace(go.Scatter( # the estimate itself
x=idx, y=(est.iloc[burn:] * 100).to_numpy(),
line=dict(color=hue, width=2), name=label,
hovertemplate="%{x|%b %Y}<br>%{y:.1f}%<extra></extra>"),
row=1, col=col)
fig.update_yaxes(title_text="%", ticksuffix="%", row=1, col=1, range=[-40, 60])
fig.update_yaxes(title_text="%", ticksuffix="%", row=1, col=2)
fig.add_hline(y=0, line=dict(color=fe.INK["muted"], width=1), row=1, col=1)
# Name the shading on the chart itself, not only in the caption.
fig.add_annotation(x=0.30, y=0.90, xref="x domain", yref="y domain",
row=1, col=1, text="shaded = 95% confidence band",
showarrow=False, xanchor="left",
font=dict(size=10.5, color=fe.INK["secondary"]))
fe.figure(fig, height=420)
fig.update_layout(margin=dict(t=86), legend=dict(y=1.16))
figWhat the shaded area means. The solid line is the estimate itself: the average return, or the volatility, computed from every observation available up to that date. The shading around it is the 95% confidence band — the range of values that are statistically indistinguishable from the estimate given how much data has accumulated so far.
Read it as the width of our ignorance. A narrow band means the data pin the number down; a wide band means the line could honestly have been drawn almost anywhere inside the shading. The band is \(\pm 1.96\) standard errors, and since the standard error shrinks like \(1/\sqrt{n}\), the shading narrows as the sample grows. Watching how fast it narrows, in each panel, is the point of the figure.
The right-hand panel behaves the way intuition says it should. The volatility estimate is erratic for the first year or two, then converges and stays put, and its band closes to a ribbon around it. By the end we know this number well.
The left-hand panel is the uncomfortable one. Five years in, the running average return stood below 4%; five years after that it was above 9%; by the twenty-year mark it had reached 11.6%. These are not small revisions. And the band never closes: twenty years in it is still some sixteen percentage points wide, which is wide enough to accommodate almost any view a reasonable person might hold about equity returns.
rows = []
for yr in [1, 5, 10, 20]:
i = min(int(yr * 252) - 1, len(spy) - 1)
rows.append({
"after": f"{yr} year" + ("s" if yr > 1 else ""),
"mean %": run_mean.iloc[i] * 100,
"mean +/- %": 1.96 * se_mean.iloc[i] * 100,
"vol %": run_vol.iloc[i] * 100,
"vol +/- %": 1.96 * se_vol.iloc[i] * 100,
})
pd.DataFrame(rows).set_index("after").round(2)| mean % | mean +/- % | vol % | vol +/- % | |
|---|---|---|---|---|
| after | ||||
| 1 year | 7.48 | 20.49 | 10.45 | 2.68 |
| 5 years | 3.75 | 21.05 | 24.01 | 2.75 |
| 10 years | 9.29 | 12.62 | 20.36 | 1.65 |
| 20 years | 11.58 | 8.34 | 19.03 | 1.09 |
Two features of that table are worth naming now, because they are the two properties every estimator in this course is judged on. The estimates drift toward something and their bands shrink: that is consistency, the sampling distribution collapsing onto the truth as data accumulate. And the rate at which the bands shrink is painfully slow. The rest of this section explains where that rate comes from and why nothing can be done about it.
One detail is worth flagging, since we return to it at the end of the lesson. The volatility estimate settles, but it settles on different values at different times. In 2010 it stood near 24% with a band of roughly \(\pm 2.8\) points; today it is near 19% with a band of about \(\pm 1.1\). Those two intervals do not overlap, and they were computed from heavily overlapping samples, which should have made them agree rather than disagree. If \(\sigma\) were genuinely a single fixed constant, that is not what we would expect to see. It is a first hint that \(\sigma\) is not fixed at all, which is precisely what volatility models exist to handle.
1.3 The sampling distribution of the mean
Now the theory. Draw \(X_1,\dots,X_n\) independently from a population with mean \(\mu\) and variance \(\sigma^2\) and form the sample mean \(\bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i\). Two results follow with no further assumptions:
\[ \mathbb{E}[\bar{X}] = \mu, \qquad \operatorname{Var}(\bar{X}) = \frac{\sigma^2}{n} . \tag{1.2}\]
The first says the estimator is unbiased. Averaged over all the histories that might have occurred, it lands on the truth. Note how weak that guarantee is: it says nothing whatever about the sample you actually have. An unbiased estimator can be badly wrong on any given occasion, and if its variance is large it usually will be.
Which makes the second result the more useful of the two. Its square root is the standard error,
\[ \operatorname{SE}(\bar{X}) = \frac{\sigma}{\sqrt{n}}, \tag{1.3}\]
the typical distance between the estimate and the truth. The \(\sqrt{n}\) in the denominator is where the pain lives. To halve your error you need four times the data; to cut it by a factor of ten you need a hundred times as much. Every standard error you meet later in this course — on a regression slope, on an autoregressive coefficient, on a GARCH parameter — has this same shape.
1.3.1 Where the square root comes from
The \(\sqrt{n}\) is worth understanding rather than memorising, because it explains why more data helps so little.
Suppose each observation is the truth plus an error, \(X_i = \mu + \varepsilon_i\), with the errors independent and each having variance \(\sigma^2\). When you add \(n\) of them, the errors partly cancel: some are positive, some negative, and they do not line up. Variances of independent quantities add, so the sum of the errors has variance \(n\sigma^2\) and standard deviation \(\sigma\sqrt{n}\). Dividing by \(n\) to form the average divides that standard deviation by \(n\), leaving \(\sigma\sqrt{n}/n = \sigma/\sqrt{n}\).
The cancellation is what buys us precision, and it is only partial. If the errors were instead perfectly correlated they would not cancel at all and averaging would achieve nothing. That is not an idle case: it is roughly what happens when you average the returns of thirty stocks in the same sector during a crash, and it is why Section 1.10 takes dependence seriously.
Standard deviation or standard error?
These get confused constantly, including by people who should know better, and the confusion is expensive.
The standard deviation \(\sigma \approx 19\%\) describes the data: how much the return varies from year to year. It does not shrink when you collect more data, because it is a fact about the world rather than about your knowledge of it. A twenty-year sample and a hundred-year sample both put equity volatility near 19%.
The standard error \(\sigma/\sqrt{n} \approx 4.2\%\) describes your estimate: how far the average return you computed is likely to sit from the true average return. This one does shrink with more data.
“Equity returns are 11.7% with a volatility of 19.2%” and “our estimate of the average equity return is 11.7%, give or take 4.2%” are entirely different statements. The first is about risk. The second is about ignorance.
1.3.2 The shape: the central limit theorem
Neither result in eq. 1.2 assumed normality, and neither needs it. But knowing the centre and the spread of the sampling distribution is not yet enough to build a confidence interval. For that we need its shape — and the shape is what the central limit theorem hands us, for nothing.
1.3.2.1 Two different promises: the law of large numbers and the CLT
These are constantly conflated, and separating them is the key to understanding what the CLT actually claims.
The law of large numbers says that as \(n\) grows, \(\bar{X}\) converges to \(\mu\). It tells you where the estimate ends up. It is the mathematical content of the right-hand panel of Figure 1.1, and on its own it is almost useless for inference: knowing that you will eventually arrive somewhere says nothing about how far away you are today.
The central limit theorem describes the error on the way there. It says that the gap \(\bar{X} - \mu\), once rescaled, has a definite and knowable shape. That shape is what turns an estimate into an interval.
The rescaling is the subtle part. As \(n\) grows, \(\bar{X} - \mu\) collapses to zero, so the distribution of the raw error degenerates to a spike and there is nothing left to describe. The fix is to magnify the error at exactly the rate it is shrinking. Since eq. 1.3 says it shrinks like \(1/\sqrt{n}\), multiplying by \(\sqrt{n}\) holds it still, and what remains in the limit is a fixed distribution:
\[ \sqrt{n}\,\frac{\bar{X} - \mu}{\sigma} \;\xrightarrow{d}\; N(0, 1). \tag{1.4}\]
The left-hand side is just the estimation error measured in units of its own standard error, \((\bar{X}-\mu)/\operatorname{SE}(\bar{X})\). The theorem says that quantity is standard normal in large samples — which is precisely why a 95% interval is the estimate plus or minus 1.96 standard errors, and why a \(t\)-statistic can be compared against 1.96 at all.
1.3.2.2 Why the result is remarkable
Read the statement again and notice what is absent from the right-hand side. There is no reference to the population the data came from. Returns, insurance claims, dice rolls, queue lengths, measurement errors in a laboratory: average enough of any of them and the standardised average has the same bell-shaped distribution. The parent distribution’s own shape washes out entirely, leaving only its mean and variance behind.
This universality is what makes statistics practical. We almost never know the distribution generating financial returns — and as the kurtosis of 18 above already shows, it is certainly not normal. The CLT means we do not need to know. It is the reason a single table of normal critical values serves for essentially every inference problem in the course.
1.3.2.3 What has to be true for it to work
The theorem is not unconditional. Two requirements matter in finance.
The first is finite variance. The proof needs \(\sigma^2 < \infty\); if the population has such heavy tails that its variance is infinite, sample means do not converge to a normal distribution at all. This is not a purely theoretical worry — some empirical studies of financial returns have argued for tail indices close to the boundary where variance ceases to exist. In practice, equity index returns are usually taken to have finite variance, but the assumption is doing real work.
The second is independence, or something close to it. Our data are not independent: volatility clusters. Versions of the CLT survive weak dependence, but the variance term must be replaced by a long-run variance that accounts for autocorrelation — which is exactly what the HAC standard errors of Section 1.7 compute, and exactly what the block bootstrap of Section 1.10 estimates by resampling.
1.3.2.4 How large is “large enough”?
Textbooks often quote \(n = 30\). That number is folklore, and the honest answer is that it depends entirely on the shape of the parent distribution — in particular on its skewness.
The leading correction term in the expansion of the sampling distribution decays like
\[ \operatorname{skew}(\bar{X}) \;\approx\; \frac{\operatorname{skew}(X)}{\sqrt{n}}, \tag{1.5}\]
so the asymmetry of the parent is inherited by the sample mean and fades only at rate \(1/\sqrt{n}\). A symmetric population converges almost immediately; a badly skewed one can resist for thousands of observations.
The following experiment makes the point with two populations drawn from the very same data. The first is daily returns, which are wildly fat-tailed but almost perfectly symmetric. The second is squared daily returns — the raw material of every variance calculation in this lesson — which are ferociously skewed, because squaring sends one crash to the far right and pins everything else near zero.
rng = np.random.default_rng(fe.SEED) # re-seeded again below, so later
# results are unaffected by this section
r = spy.to_numpy()
clt_pops = {"Daily returns": r, "Squared returns": r**2}
pd.DataFrame({
"skewness": [stats.skew(v) for v in clt_pops.values()],
"kurtosis": [stats.kurtosis(v, fisher=False) for v in clt_pops.values()],
}, index=list(clt_pops)).round(2)| skewness | kurtosis | |
|---|---|---|
| Daily returns | 0.00 | 18.18 |
| Squared returns | 16.81 | 416.18 |
Daily returns have a skewness of essentially zero and a kurtosis of 18. Squared returns have a skewness of about 17. Now standardise the sample mean of each, exactly as eq. 1.4 prescribes, and see which one becomes normal.
NS = [5, 30, 250]
fig = make_subplots(rows=2, cols=3, horizontal_spacing=0.05, vertical_spacing=0.20,
subplot_titles=[f"{p} — n={n}" for p in clt_pops for n in NS])
for row, (pname, pop_v) in enumerate(clt_pops.items(), start=1):
mu_v, sd_v = pop_v.mean(), pop_v.std(ddof=0)
for col, n in enumerate(NS, start=1):
s = rng.choice(pop_v, size=(40_000, n), replace=True)
z = (s.mean(axis=1) - mu_v) / (sd_v / np.sqrt(n))
edges = np.linspace(-4, 4, 41)
counts, _ = np.histogram(z, bins=edges, density=True)
fig.add_trace(go.Bar(
x=(edges[:-1] + edges[1:]) / 2, y=counts,
width=(edges[1] - edges[0]) * 0.86,
marker=dict(color=fe.band_colour("blue", 0.55),
line=dict(color=BLUE, width=0.6)),
name="Standardised sample mean",
showlegend=(row == 1 and col == 1),
hovertemplate="z = %{x:.2f}<extra></extra>"), row=row, col=col)
grid = np.linspace(-4, 4, 200)
fig.add_trace(go.Scatter(
x=grid, y=stats.norm.pdf(grid),
line=dict(color=ORANGE, width=2, dash="dash"),
name="Standard normal N(0,1)",
showlegend=(row == 1 and col == 1), hoverinfo="skip"),
row=row, col=col)
fig.update_xaxes(range=[-4, 4], nticks=5, tickangle=0)
fig.update_yaxes(showticklabels=False, showgrid=False)
fe.figure(fig, height=560)
fig.update_layout(margin=dict(t=90), legend=dict(y=1.10))
figThe top row is the CLT behaving as advertised. At \(n = 5\) the histogram is already symmetric and unimodal, though still too peaked in the middle and a little too heavy in the tails; by \(n = 30\) it sits under the normal curve almost perfectly, and by \(n = 250\) the two are indistinguishable — all of this from a parent with a kurtosis of 18. Fat tails, on their own, are not an obstacle.
The bottom row is the warning. At \(n = 5\) the distribution is a spike against the left wall with a tail running off to the right. At \(n = 30\) — the folklore threshold — it is still hopelessly asymmetric. Even at \(n = 250\), a full year of daily data, the skew is plainly visible.
rows = []
for pname, pop_v in clt_pops.items():
mu_v, sd_v = pop_v.mean(), pop_v.std(ddof=0)
for n in NS:
s = rng.choice(pop_v, size=(40_000, n), replace=True)
z = (s.mean(axis=1) - mu_v) / (sd_v / np.sqrt(n))
rows.append({
"population": pname, "n": n,
"skew of z": stats.skew(z),
"predicted skew/sqrt(n)": stats.skew(pop_v) / np.sqrt(n),
"excess kurtosis of z": stats.kurtosis(z),
"P(z < -1.96)": np.mean(z < -1.96),
})
pd.DataFrame(rows).set_index(["population", "n"]).round(3)| skew of z | predicted skew/sqrt(n) | excess kurtosis of z | P(z < -1.96) | ||
|---|---|---|---|---|---|
| population | n | ||||
| Daily returns | 5 | 0.053 | 0.002 | 2.970 | 0.029 |
| 30 | -0.032 | 0.001 | 0.459 | 0.027 | |
| 250 | -0.004 | 0.000 | 0.086 | 0.027 | |
| Squared returns | 5 | 7.821 | 7.517 | 89.555 | 0.000 |
| 30 | 3.103 | 3.069 | 14.379 | 0.000 | |
| 250 | 1.043 | 1.063 | 1.524 | 0.000 |
Two things in that table deserve attention.
The predicted and realised skewness columns track each other closely, which confirms eq. 1.5 as more than hand-waving: the parent’s asymmetry really does decay at rate \(1/\sqrt{n}\), and you can forecast in advance how large a sample you will need.
The final column shows what the failure costs. For daily returns the true left-tail probability moves steadily toward the nominal 0.025 and is within a fraction of a percentage point of it by \(n = 250\). For squared returns it is essentially zero at every sample size shown — meaning a 95% confidence interval built on the normal approximation would not have 95% coverage; it would be wrong in a systematic, one-sided way that no amount of care elsewhere could detect.
What the central limit theorem does not say
- It does not say returns are normal. It is a statement about averages, not about the raw data. Daily returns keep their kurtosis of 18 no matter how many of them you collect, which is why a value-at-risk model built on a normal assumption still fails in a crash.
- It does not rescue the far tail. The approximation is at its best near the centre of the distribution and at its worst several standard errors out — exactly where risk management operates.
- It does not apply to every statistic. It is the sample mean that goes normal. The sample variance is an average too, but of a violently skewed quantity, as the bottom row of Figure 1.2 shows. That single fact explains most of what happens in the next section.
- It does not repair dependence. With autocorrelated data the limit still holds under conditions, but \(\sigma\) must be replaced by a long-run variance. Using the ordinary \(\sigma/\sqrt{n}\) then gives the wrong width.
This is the result that licenses every \(t\)-statistic and confidence interval in the course without our ever claiming that returns are normally distributed. Let us now watch it work on the actual estimator we care about, using data with a kurtosis of 18.
rng = np.random.default_rng(fe.SEED)
pop = spy.to_numpy()
B = 20_000 # number of parallel histories
SIZES = {"1 month (n=21)": 21, "1 year (n=252)": 252, "5 years (n=1260)": 1260}
draws = {}
for name, n in SIZES.items():
sample = rng.choice(pop, size=(B, n), replace=True)
draws[name] = {
"n": n,
"mean": sample.mean(axis=1),
"var": sample.var(axis=1, ddof=1),
}Each row of sample is one alternative history of length n, and we now have twenty thousand of them at each sample size.
fig = make_subplots(rows=1, cols=3, horizontal_spacing=0.06,
subplot_titles=tuple(draws))
for col, (name, d) in enumerate(draws.items(), start=1):
m_ann = d["mean"] * 252 * 100 # annualise, in per cent
lo, hi = np.percentile(m_ann, [0.2, 99.8]) # trim the extreme tail
edges = np.linspace(lo, hi, 33) # 32 bins -- readable, not dense
counts, _ = np.histogram(m_ann, bins=edges, density=True)
fig.add_trace(go.Bar(
x=(edges[:-1] + edges[1:]) / 2, y=counts,
width=(edges[1] - edges[0]) * 0.86, # gap between bars
marker=dict(color=fe.band_colour("blue", 0.55),
line=dict(color=BLUE, width=0.8)),
name="Simulated estimates", showlegend=(col == 1),
hovertemplate="%{x:.1f}%<extra></extra>"), row=1, col=col)
se = sig_pop / np.sqrt(d["n"]) * 252 * 100 # density implied by the CLT
grid = np.linspace(lo, hi, 240)
fig.add_trace(go.Scatter(
x=grid, y=stats.norm.pdf(grid, mu_pop * 252 * 100, se),
line=dict(color=ORANGE, width=2.2, dash="dash"),
name="Normal curve predicted by the CLT", showlegend=(col == 1),
hoverinfo="skip"), row=1, col=col)
fig.add_vline(x=mu_pop * 252 * 100, line=dict(color=fe.INK["secondary"], width=1.5),
row=1, col=col)
fig.update_xaxes(title_text="annualised mean, %", ticksuffix="%",
nticks=5, tickangle=0)
fig.update_yaxes(showticklabels=False, showgrid=False)
fig.update_yaxes(title_text="density", row=1, col=1)
fig.add_annotation(x=mu_pop * 252 * 100, y=0.97, yref="y domain", row=1, col=1,
text=" true mean", showarrow=False, xanchor="left",
yanchor="top", font=dict(size=10.5, color=fe.INK["secondary"]))
fe.figure(fig, height=400)
fig.update_layout(margin=dict(t=86), legend=dict(y=1.16))
figThree things in that figure repay a second look. The histogram sits squarely over the vertical line in every panel, which is unbiasedness made visible. The spread narrows, but grudgingly: going from one month to five years multiplies the data by sixty and narrows the distribution by a factor of only \(\sqrt{60} \approx 7.7\). And the dashed normal curve, which has no business fitting a sample drawn from a distribution with kurtosis 18, fits almost perfectly by \(n = 252\).
rows = []
for name, d in draws.items():
m_ann = d["mean"] * 252
rows.append({
"sample": name,
"E[mean] (sim)": m_ann.mean(),
"population mean": mu_pop * 252,
"+/- MC error": 1.96 * m_ann.std(ddof=1) / np.sqrt(B),
"SE (sim)": m_ann.std(ddof=1),
"SE = sigma/sqrt(n)": sig_pop / np.sqrt(d["n"]) * 252,
"skew": stats.skew(m_ann),
"excess kurt": stats.kurtosis(m_ann),
})
pd.DataFrame(rows).set_index("sample").round(4)| E[mean] (sim) | population mean | +/- MC error | SE (sim) | SE = sigma/sqrt(n) | skew | excess kurt | |
|---|---|---|---|---|---|---|---|
| sample | |||||||
| 1 month (n=21) | 0.1235 | 0.117 | 0.0092 | 0.6657 | 0.6656 | 0.0484 | 0.8668 |
| 1 year (n=252) | 0.1191 | 0.117 | 0.0027 | 0.1916 | 0.1921 | -0.0228 | 0.0172 |
| 5 years (n=1260) | 0.1176 | 0.117 | 0.0012 | 0.0862 | 0.0859 | -0.0045 | -0.0258 |
The simulated standard error and the formula \(\sigma/\sqrt{n}\) agree closely at every sample size. The simulated mean does not match the population mean quite so exactly, and the third column explains why: a simulation is itself an estimate, built from twenty thousand replications rather than infinitely many, so it carries its own standard error of \(\text{SE}/\sqrt{B}\). Every gap in the first two columns falls inside that margin. It is a useful reminder that the computer has not escaped the problem this lesson is about — it has only made it small enough to ignore.
The last two columns are the central limit theorem doing its work. The skewness and excess kurtosis of the sampling distribution fall away toward zero as \(n\) grows, even though the underlying data are exactly as fat-tailed at \(n = 1260\) as they were at \(n = 21\). Averaging washes the non-normality out.
1.4 The sampling distribution of the variance
Now the second number on the report. The sample variance is
\[ s^2 = \frac{1}{n-1}\sum_{i=1}^{n}\left(X_i - \bar{X}\right)^2 . \tag{1.6}\]
1.4.1 Why we divide by \(n-1\)
The divisor is \(n-1\) rather than \(n\), and the reason is the sampling-distribution logic of this lesson rather than an arbitrary convention.
We would like to measure deviations from the true mean \(\mu\), but we do not know it, so we use \(\bar{X}\) instead. And \(\bar{X}\) is by construction the number that makes \(\sum_i (X_i - m)^2\) as small as it can possibly be — that is what the sample mean is, as Section 1.6 shows formally. Measuring spread around the point specially chosen to minimise it must understate the true spread. Dividing by \(n-1\) instead of \(n\) inflates the answer by just enough to compensate, and restores
\[ \mathbb{E}[s^2] = \sigma^2 . \tag{1.7}\]
The intuition generalises: we spent one observation’s worth of information estimating \(\bar{X}\), so only \(n-1\) independent pieces of information about spread remain. That phrase, degrees of freedom, returns verbatim in regression, where the residual variance is divided by \(n-k\) after estimating \(k\) coefficients.
n = 21
sample = rng.choice(pop, size=(200_000, n), replace=True)
v1 = sample.var(axis=1, ddof=1).mean()
v0 = sample.var(axis=1, ddof=0).mean()
print(pd.DataFrame({
"annualised variance": [v1 * 252, v0 * 252, sig_pop**2 * 252],
"ratio to sigma^2": [v1/sig_pop**2, v0/sig_pop**2, 1.0],
}, index=["divisor n-1 (unbiased)", "divisor n (biased)",
"population sigma^2"]).round(5))
print(f"\nratio of the two estimators : {v0/v1:.6f}")
print(f"(n-1)/n = 20/21 : {20/21:.6f}") annualised variance ratio to sigma^2
divisor n-1 (unbiased) 0.0370 1.0014
divisor n (biased) 0.0352 0.9537
population sigma^2 0.0369 1.0000
ratio of the two estimators : 0.952381
(n-1)/n = 20/21 : 0.952381The unbiased version lands on \(\sigma^2\), missing it only by the Monte Carlo error of the simulation itself. The biased version sits below it, and the last two lines show the relationship exactly: the two estimators differ by the constant factor \((n-1)/n = 20/21 \approx 0.952\), so their ratio reproduces that number to every digit printed. The shortfall is not noise. It is systematic, and it is what “biased” means.
1.4.2 Why the distribution leans to the right
The variance behaves quite differently from the mean, and the reason is that squaring destroys the cancellation that made averaging work so well.
When we average returns, a bad day is offset by a good one. When we average squared deviations there is nothing to offset: every term is positive, and a large deviation contributes its square, so a single extreme day can dominate the whole calculation. The effect is easy to demonstrate.
calm = spy.loc["2005"].to_numpy()
worst_day, worst_date = spy.min(), spy.idxmin().date()
swapped = np.append(calm[:-1], worst_day)
pd.DataFrame({
"annualised volatility %": [calm.std(ddof=1) * np.sqrt(252) * 100,
swapped.std(ddof=1) * np.sqrt(252) * 100],
}, index=["2005 as it happened", f"2005 with one day replaced by {worst_date}"]).round(1)| annualised volatility % | |
|---|---|
| 2005 as it happened | 10.3 |
| 2005 with one day replaced by 2020-03-16 | 15.1 |
Swapping one day in 251 — a single observation, less than half a percent of the sample — raises the estimated volatility by about half. That asymmetry is the whole story. A sample that happens to contain a crash produces a variance estimate far above the truth, while a sample that happens to miss one can only be modestly below it, since variance is bounded below by zero. Most samples therefore understate the variance slightly and a minority overstate it dramatically, which is exactly what a right-skewed distribution looks like.
1.4.3 Unbiased is not the same as precise
So \(s^2\) is centred correctly. That is where the good news stops. Under normality the sample variance follows a scaled chi-squared distribution,
\[ \frac{(n-1)\,s^2}{\sigma^2} \sim \chi^2_{n-1}, \qquad\text{so}\qquad \operatorname{Var}(s^2) = \frac{2\sigma^4}{n-1}, \tag{1.8}\]
which is not symmetric, for the reason just given. But eq. 1.8 is the normal-theory answer, and we have established that returns are not normal. In general, for large \(n\),
\[ \operatorname{Var}(s^2) \;\approx\; \frac{\sigma^4\,(\kappa - 1)}{n}, \qquad \kappa = \frac{\mathbb{E}[(X-\mu)^4]}{\sigma^4}, \tag{1.9}\]
where \(\kappa\) is the kurtosis. Setting \(\kappa = 3\) recovers eq. 1.8, as it must. Our data have a kurtosis of 18, which inflates the variance of \(s^2\) by a factor of \((\kappa-1)/2\), about 8.6, and so inflates the standard error by a factor of about 2.9.
That is not a technicality. Anyone building a confidence interval for volatility from the textbook chi-squared formula will report an interval roughly three times too narrow, and will be surprised three times as often as they expect.
Notice also the connection back to the previous subsection. Kurtosis measures how much of the variance comes from rare large moves, and it is exactly those moves that make \(s^2\) unreliable. Fat tails and an imprecise variance estimate are the same phenomenon seen from two angles.
fig = make_subplots(rows=1, cols=3, horizontal_spacing=0.06,
subplot_titles=tuple(draws))
for col, (name, d) in enumerate(draws.items(), start=1):
n_i = d["n"]
v_ann = d["var"] * 252
lo, hi = np.percentile(v_ann, [0.2, 99.5]) # long right tail -- trim to see shape
edges = np.linspace(lo, hi, 33)
counts, _ = np.histogram(v_ann, bins=edges, density=True)
fig.add_trace(go.Bar(
x=(edges[:-1] + edges[1:]) / 2, y=counts,
width=(edges[1] - edges[0]) * 0.86,
marker=dict(color=fe.band_colour("blue", 0.55),
line=dict(color=BLUE, width=0.8)),
name="Simulated estimates", showlegend=(col == 1),
hovertemplate="%{x:.3f}<extra></extra>"), row=1, col=col)
# normal-theory density: (n-1)s^2/sigma^2 ~ chi2(n-1), rescaled
grid = np.linspace(lo, hi, 320)
scale = sig_pop**2 * 252 / (n_i - 1)
fig.add_trace(go.Scatter(
x=grid, y=stats.chi2.pdf(grid / scale, n_i - 1) / scale,
line=dict(color=ORANGE, width=2.2, dash="dash"),
name="Normal-theory (chi-squared) prediction", showlegend=(col == 1),
hoverinfo="skip"), row=1, col=col)
fig.add_vline(x=sig_pop**2 * 252, line=dict(color=fe.INK["secondary"], width=1.5),
row=1, col=col)
fig.update_xaxes(title_text="annualised variance", nticks=5, tickangle=0)
fig.update_yaxes(showticklabels=False, showgrid=False)
fig.update_yaxes(title_text="density", row=1, col=1)
fig.add_annotation(x=sig_pop**2 * 252, y=0.97, yref="y domain", row=1, col=1,
text=" true variance", showarrow=False, xanchor="left",
yanchor="top", font=dict(size=10.5, color=fe.INK["secondary"]))
fe.figure(fig, height=400)
fig.update_layout(margin=dict(t=86), legend=dict(y=1.16))
figSet that beside Figure 1.3. The mean’s distribution was symmetric and sat neatly under its theoretical curve. This one has a long right tail, and the normal-theory curve is far too tall and narrow to describe it. The skew does fade as \(n\) grows, but a good deal more slowly than the non-normality of the mean did.
rows = []
for name, d in draws.items():
n_i, v_ann = d["n"], d["var"] * 252
sd_normal = np.sqrt(2 * (sig_pop**2 * 252)**2 / (n_i - 1))
sd_kurt = np.sqrt((sig_pop**2 * 252)**2 * (kappa - 1) / n_i)
rows.append({
"sample": name,
"SE (sim)": v_ann.std(ddof=1),
"SE normal theory": sd_normal,
"SE kurtosis-adj.": sd_kurt,
"sim / normal": v_ann.std(ddof=1) / sd_normal,
"skew": stats.skew(v_ann),
})
pd.DataFrame(rows).set_index("sample").round(4)| SE (sim) | SE normal theory | SE kurtosis-adj. | sim / normal | skew | |
|---|---|---|---|---|---|
| sample | |||||
| 1 month (n=21) | 0.0342 | 0.0117 | 0.0334 | 2.9318 | 3.6756 |
| 1 year (n=252) | 0.0095 | 0.0033 | 0.0096 | 2.8944 | 1.0520 |
| 5 years (n=1260) | 0.0043 | 0.0015 | 0.0043 | 2.9342 | 0.4852 |
The kurtosis-adjusted formula tracks the simulation closely at every sample size. The normal-theory formula is out by a factor of about three throughout, and being wrong by the same factor at \(n = 1260\) as at \(n = 21\) tells you this is a bias in the formula rather than a small-sample problem that more data will cure.
1.5 When \(\sigma\) is unknown: Student’s \(t\)
There is a gap in the argument so far, and it is the gap that made a brewer famous.
Every standard error we have written down contains \(\sigma\), the population standard deviation, which is exactly as unknown as \(\mu\). In practice we substitute \(s\), and the substitution has a cost: we are now dividing by a quantity that is itself an estimate, wobbling from sample to sample. Because \(s\) is sometimes too small, the ratio \((\bar{X}-\mu)/(s/\sqrt{n})\) takes extreme values more often than a normal distribution would.
William Sealy Gosset ran into this at the Guinness brewery in Dublin in the 1900s, where samples of barley and yeast were small and the normal approximation was visibly failing him. He worked out the correct distribution and published it in 1908 under the pen name “Student”, because Guinness forbade its staff from publishing. Hence Student’s \(t\)-distribution, with \(n-1\) degrees of freedom: the same \(n-1\) we met when dividing the sum of squares, and for the same reason.
rows = []
for n_i in [10, 21, 60, 252, len(spy)]:
t_crit = stats.t.ppf(0.975, n_i - 1)
rows.append({
"sample size": n_i,
"t critical value": t_crit,
"normal": stats.norm.ppf(0.975),
"interval wider by": f"{t_crit/stats.norm.ppf(0.975) - 1:.1%}",
})
pd.DataFrame(rows).set_index("sample size").round(3)| t critical value | normal | interval wider by | |
|---|---|---|---|
| sample size | |||
| 10 | 2.262 | 1.96 | 15.4% |
| 21 | 2.086 | 1.96 | 6.4% |
| 60 | 2.001 | 1.96 | 2.1% |
| 252 | 1.969 | 1.96 | 0.5% |
| 5153 | 1.960 | 1.96 | 0.0% |
The practical guidance falls straight out of that table. With ten observations the correction matters a great deal, widening the interval by 15%. By a year of daily data it is half a percent, and with the full sample it has vanished. In this lesson we use 1.96 throughout without apology, because \(n\) is over five thousand. In a lesson on monthly data over a short window, or on a hedge fund with a three-year track record, we will not have that luxury.
1.6 Three routes to the same estimator
We have treated \(\bar{X}\) and \(s^2\) as given. But where does an estimator come from in the first place? There are three classical answers, and this course uses all three. On the mean, reassuringly, they agree.
Least squares chooses the single number \(m\) sitting closest to the data in squared-error terms, minimising \(S(m) = \sum_i (X_i - m)^2\). Differentiating gives \(S'(m) = -2\sum_i (X_i - m) = 0\), so
\[ \hat{m} = \frac{1}{n}\sum_i X_i = \bar{X}. \tag{1.10}\]
This also settles the promise made in eq. 1.7: the sample mean really is the minimiser of the sum of squared deviations, which is why measuring spread around it understates \(\sigma^2\).
The method of moments starts from a population relationship, here \(\mathbb{E}[X] = \mu\), replaces the population expectation by its sample analogue and solves. That gives \(\frac{1}{n}\sum_i X_i = \hat{\mu}\), which is \(\bar{X}\) once again. Applying the same recipe to the second moment, \(\mathbb{E}[(X-\mu)^2] = \sigma^2\), yields \(\hat{\sigma}^2 = \frac{1}{n}\sum_i (X_i - \bar{X})^2\).
Maximum likelihood assumes a distribution — take \(X_i \sim N(\mu,\sigma^2)\) — and picks the parameter values that make the observed sample most probable. The log-likelihood is
\[ \ell(\mu, \sigma^2) = -\frac{n}{2}\ln(2\pi\sigma^2) - \frac{1}{2\sigma^2}\sum_i (X_i - \mu)^2 , \tag{1.11}\]
and setting its derivatives to zero returns \(\hat{\mu} = \bar{X}\) together with \(\hat{\sigma}^2 = \frac{1}{n}\sum_i (X_i - \bar{X})^2\).
Two lessons follow, and both matter later. The first is that least squares and maximum likelihood coincide here, and they do so for a reason visible in eq. 1.11: the only place \(\mu\) appears is inside a sum of squares, so maximising the likelihood is minimising that sum. This is not a fact about means. It is why OLS and Gaussian maximum likelihood give identical regression coefficients, and it is worth remembering when someone insists the two are different techniques.
The second is that all three principles deliver the \(\div n\) variance, not the \(\div (n-1)\) one. The unbiased \(s^2\) is not what any principle produces; it is a correction applied afterwards, by hand, because we happen to value unbiasedness. Maximum likelihood estimators are consistent but not generally unbiased, and that trade-off reappears every time we fit a volatility model.
| Principle | The rule | On the mean | Where it returns |
|---|---|---|---|
| Least squares | Minimise the sum of squared deviations | \(\bar{X}\) | OLS regression, the market model, factor models |
| Method of moments | Match sample moments to population moments | \(\bar{X}\) | GMM, instrumental variables, cross-sectional asset pricing tests |
| Maximum likelihood | Maximise the probability of the observed sample | \(\bar{X}\), \(\tfrac{1}{n}\sum(X_i-\bar{X})^2\) | ARMA models, GARCH and every volatility model |
1.7 The same machinery in every model that follows
This section is the point of the lesson. We now show, not by analogy but by identity, that the sample mean and the sample variance are the simplest cases of the three model families the rest of the course is built on.
1.7.1 a. Estimating the parameters of a linear regression
A regression with no explanatory variables, only a constant, is \(X_i = \beta_0 + \varepsilon_i\). Least squares picks \(\beta_0\) to minimise \(\sum_i (X_i - \beta_0)^2\), which is eq. 1.10 exactly. The OLS coefficient must therefore be the sample mean, and the OLS standard error must be \(s/\sqrt{n}\).
import statsmodels.api as sm
y = spy.to_numpy()
X = np.ones((len(y), 1)) # a single column of ones: the constant
ols = sm.OLS(y, X).fit()
print(f"OLS coefficient {ols.params[0]:.10f}")
print(f"sample mean {spy.mean():.10f}")
print(f" identical? {np.allclose(ols.params[0], spy.mean())}")
print()
print(f"OLS standard error {ols.bse[0]:.10f}")
print(f"s/sqrt(n) {spy.std(ddof=1)/np.sqrt(len(y)):.10f}")
print(f" identical? {np.allclose(ols.bse[0], spy.std(ddof=1)/np.sqrt(len(y)))}")
print()
print(f"OLS residual variance {ols.resid.var(ddof=1):.10f}")
print(f"sample variance {spy.var(ddof=1):.10f}")OLS coefficient 0.0004641998
sample mean 0.0004641998
identical? True
OLS standard error 0.0001686337
s/sqrt(n) 0.0001686337
identical? True
OLS residual variance 0.0001465375
sample variance 0.0001465375Not approximately equal, but identical to every digit printed. Everything this lesson has established about \(\bar{X}\) is therefore already a statement about regression.
Now add a second column to X, say the market return, and you have the market model \(R_i = \alpha + \beta R_{m,i} + \varepsilon_i\). The estimator changes from an average into a projection, and the algebra becomes matrix algebra, but not one concept in this lesson has to be replaced. Both \(\hat{\alpha}\) and \(\hat{\beta}\) are random variables. Both have sampling distributions. Both have standard errors that shrink like \(1/\sqrt{n}\). Their approximate normality comes from a central limit theorem rather than from any assumption about returns, and their residual variance carries a degrees-of-freedom correction for the same reason \(s^2\) does.
1.7.2 b. Estimating a time-series model
The mean assumed observations were independent. Financial data are not: today’s return is mildly related to yesterday’s, and today’s volatility strongly so. The natural generalisation is an autoregression,
\[ X_t = c + \phi X_{t-1} + \varepsilon_t , \tag{1.12}\]
and setting \(\phi = 0\) collapses it back to “\(X_t\) varies around a constant mean”, which is the model we have been fitting implicitly all lesson.
Dependence has a specific consequence, and it is not the one most people expect. It changes the standard error, not the estimate.
# The same regression on a constant, with standard errors that allow for dependence
hac = sm.OLS(y, X).fit(cov_type="HAC", cov_kwds={"maxlags": 21})
ar1 = sm.tsa.ARIMA(y, order=(1, 0, 0)).fit()
print("the estimate is unchanged:")
print(f" OLS / sample mean, annualised {ols.params[0]*252:.4%}")
print(f" AR(1) mean parameter, annualised {ar1.params[0]*252:.4%}")
print()
print("the standard error is not:")
print(f" assuming independence (annualised) {ols.bse[0]*252:.4%}")
print(f" allowing dependence, HAC {hac.bse[0]*252:.4%}")
print()
print(f"AR(1) coefficient phi = {ar1.params[1]:.4f}")the estimate is unchanged:
OLS / sample mean, annualised 11.6978%
AR(1) mean parameter, annualised 11.6978%
the standard error is not:
assuming independence (annualised) 4.2496%
allowing dependence, HAC 3.5350%
AR(1) coefficient phi = -0.1052The estimated autoregressive coefficient \(\hat{\phi}\) is -0.105, small and negative, so daily returns mildly reverse: an up day is slightly more likely to be followed by a down day than by another up day. Because the dependence is negative, successive returns cancel a little better than independent ones would, and the honest standard error comes out smaller than the independence-based one rather than larger. That runs against the usual intuition that dependence always costs you precision, and it is worth knowing that the sign matters. Section 1.10 reaches the same conclusion by a completely different route.
The general lesson is the one to carry forward: the estimator was fine, the inference was not. Most of the complications in the time-series half of this course — autocorrelation, overlapping observations, HAC standard errors — are corrections to a sampling distribution rather than to an estimate.
1.7.3 c. Fitting a volatility model
Finally the variance. Maximum likelihood applied to a constant-variance normal model should return exactly the \(\div n\) sample variance of Section 1.6, and we can check that against a purpose-built volatility package rather than take it on trust.
from arch import arch_model
# arch works in percentage units for numerical stability
const_vol = arch_model(y * 100, mean="Constant", vol="Constant").fit(disp="off")
print(f"MLE sigma^2 (constant-variance model) {const_vol.params['sigma2']/1e4:.10f}")
print(f"sample variance, divisor n {spy.var(ddof=0):.10f}")
print(f"sample variance, divisor n-1 {spy.var(ddof=1):.10f}")MLE sigma^2 (constant-variance model) 0.0001465091
sample variance, divisor n 0.0001465091
sample variance, divisor n-1 0.0001465375Identical to the \(\div n\) version, precisely as eq. 1.11 predicts. Now make one change and let the variance depend on the past. That is GARCH(1,1),
\[ \sigma_t^2 = \omega + \alpha\,\varepsilon_{t-1}^2 + \beta\,\sigma_{t-1}^2 , \tag{1.13}\]
fitted by maximising the same log-likelihood eq. 1.11, with the constant \(\sigma^2\) replaced by \(\sigma_t^2\). The estimation principle does not change at all. Only the thing being estimated has been allowed to move.
garch = arch_model(y * 100, mean="Constant", vol="GARCH", p=1, q=1).fit(disp="off")
om, al, be = garch.params["omega"], garch.params["alpha[1]"], garch.params["beta[1]"]
long_run = om / (1 - al - be) / 1e4 # implied unconditional variance
cond_vol = garch.conditional_volatility / 100 * np.sqrt(252)
print(f"persistence alpha + beta {al + be:.4f}")
print(f"long-run volatility (annualised) {np.sqrt(long_run*252):.2%}")
print(f"sample volatility (annualised) {sig_pop*np.sqrt(252):.2%}")
print()
print(f"conditional volatility ranges from {cond_vol.min():.1%} to {cond_vol.max():.1%}")persistence alpha + beta 0.9776
long-run volatility (annualised) 17.98%
sample volatility (annualised) 19.21%
conditional volatility ranges from 7.1% to 105.9%The long-run volatility implied by the model is close to the single number the fund reported. But the conditional volatility, the model’s estimate of risk on a given day, ranges from about 7% to 106% annualised. The sample variance is an average of something that moves by more than an order of magnitude, and averages of wildly varying quantities are exactly what Figure 1.1 warned us about.
gidx = spy.index.to_pydatetime()
fig = go.Figure()
fig.add_trace(go.Scatter(
x=gidx, y=np.asarray(cond_vol) * 100,
line=dict(color=BLUE, width=1.1), name="GARCH(1,1) conditional volatility",
hovertemplate="%{x|%d %b %Y}<br>%{y:.1f}%<extra></extra>"))
fig.add_trace(go.Scatter(
x=[gidx[0], gidx[-1]], y=[sig_pop*np.sqrt(252)*100]*2, mode="lines",
line=dict(color=ORANGE, width=2, dash="dash"),
name="Sample volatility (the single constant estimate)",
hoverinfo="skip"))
fig.update_yaxes(title_text="annualised volatility, %", ticksuffix="%")
fe.figure(fig, height=420)
fig.update_layout(margin=dict(t=70), legend=dict(y=1.10))
fig1.7.4 The map
| This lesson | Generalises to | Estimated by | Later lesson |
|---|---|---|---|
| \(\bar{X}\), the unconditional mean | \(\alpha + \beta R_m\), a conditional mean | Least squares | Linear regression, factor models |
| \(\bar{X}\) with independent data | \(c + \phi X_{t-1}\), a mean depending on the past | Least squares / MLE | Time-series models |
| \(s^2\), the unconditional variance | \(\sigma_t^2\), a variance depending on the past | Maximum likelihood | Volatility models |
1.8 In Python: what the two reported numbers are worth
We can now put an honest interval around each of the two numbers on their page. For the mean we use eq. 1.3. For the volatility we need the standard error of \(s\) rather than of \(s^2\), which follows from eq. 1.9 by the delta method:
\[ \operatorname{SE}(s) \;\approx\; \sigma\sqrt{\frac{\kappa-1}{4n}} . \tag{1.14}\]
n_obs = len(spy)
years = n_obs / 252
mean_ann = mu_pop * 252
vol_ann = sig_pop * np.sqrt(252)
se_mean_full = vol_ann / np.sqrt(years) # eq-se-mean
se_vol_full = vol_ann * np.sqrt((kappa - 1) / (4 * n_obs)) # eq-se-vol
report = pd.DataFrame({
"estimate": [mean_ann, vol_ann],
"std. error": [se_mean_full, se_vol_full],
"95% lower": [mean_ann - 1.96*se_mean_full, vol_ann - 1.96*se_vol_full],
"95% upper": [mean_ann + 1.96*se_mean_full, vol_ann + 1.96*se_vol_full],
}, index=["average return", "volatility"])
(report * 100).round(2)| estimate | std. error | 95% lower | 95% upper | |
|---|---|---|---|---|
| average return | 11.70 | 4.25 | 3.37 | 20.03 |
| volatility | 19.21 | 0.55 | 18.13 | 20.30 |
What a 95% confidence interval does not mean
It does not mean there is a 95% probability that the true average return lies between the two numbers in that row. The true value is a fixed constant; it is either in the interval or it is not, and no probability attaches to it.
What the 95% describes is the procedure. If we could rerun history many times and construct this interval each time, about 95% of those intervals would contain the truth. We have one of them and no way of knowing whether it is one of the lucky ones.
In practice the distinction rarely changes a decision, but it does change how you should speak. “We are 95% sure the return is between 3% and 20%” is a claim about your beliefs. “This interval was built by a method that works 95% of the time” is a claim about your method, and it is the one the arithmetic supports.
1.9 Reading the output
That table answers the question the lesson opened with, and not in the way the report implies.
The average return is not known. Twenty years of daily data, more than five thousand observations, and the interval runs from roughly 3.4% to 20.0%. The data cannot distinguish a mediocre equity market from a spectacular one. A choice between 11.7% and 7% as a forward assumption therefore cannot be settled by this sample, however long it looks; it has to be argued on other grounds.
The volatility, by contrast, is known rather well: about a percentage point either side. A comparison against a 20% limit is a real comparison, because the margin of error is small relative to the distance being judged.
This asymmetry is the practical heart of the lesson. Second moments are estimable from market data; first moments are not. It is why risk models are built on volatility rather than on expected returns, why mean-variance optimisers that take expected returns as inputs produce such unstable portfolios, and why “we have twenty years of data” is far less reassuring than it sounds. It is also why the volatility half of this course can be so much more ambitious than the expected-return half.
1.9.1 How long would it take to prove skill?
The same arithmetic answers a question that comes up whenever a manager’s track record is discussed. Suppose a strategy has a genuine annualised Sharpe ratio \(SR = \mu_{\text{ann}}/\sigma_{\text{ann}}\). Its \(t\)-statistic after \(T\) years is
\[ t = \frac{\hat{\mu}_{\text{ann}}}{\operatorname{SE}(\hat{\mu}_{\text{ann}})} = \frac{\mu_{\text{ann}}}{\sigma_{\text{ann}}/\sqrt{T}} = SR\sqrt{T}, \tag{1.15}\]
so demonstrating skill at the 5% level requires \(SR\sqrt{T} \geq 1.96\), or \(T \geq (1.96/SR)^2\) years.
sr = np.array([0.2, 0.3, 0.5, 0.75, 1.0, 1.5])
pd.DataFrame({
"annualised Sharpe ratio": sr,
"years required": (1.96 / sr) ** 2,
}).set_index("annualised Sharpe ratio").round(1)| years required | |
|---|---|
| annualised Sharpe ratio | |
| 0.20 | 96.0 |
| 0.30 | 42.7 |
| 0.50 | 15.4 |
| 0.75 | 6.8 |
| 1.00 | 3.8 |
| 1.50 | 1.7 |
A manager with a true Sharpe ratio of 0.5, which would be a genuinely good long-only equity manager, needs about fifteen years before the record alone can establish that the skill is real. Most mandates are reviewed on three. The implication is not that manager selection is hopeless, but that it cannot rest on the track record by itself, and that a three-year number is far weaker evidence than the confidence with which it is usually presented.
1.9.2 Why more frequent data does not help the mean
A natural response to all this is to sample more often. Move from monthly to daily and the sample grows from 245 observations to over five thousand. Surely that must help?
For the mean it does not, and eq. 1.3 shows why. Sample \(T\) years at \(m\) observations per year, so \(n = mT\). Per-period volatility is \(\sigma_{\text{ann}}/\sqrt{m}\), and annualising the mean multiplies by \(m\), so the standard error of the annualised mean is
\[ m \times \frac{\sigma_{\text{ann}}/\sqrt{m}}{\sqrt{mT}} = \frac{\sigma_{\text{ann}}}{\sqrt{T}} . \tag{1.16}\]
Every \(m\) has cancelled. What remains depends on the calendar span \(T\) alone. Slicing the same twenty years more finely adds observations but no information about the average. The same calculation for the variance gives \(\sigma^2_{\text{ann}}\sqrt{(\kappa-1)/(mT)}\), which does fall with \(m\).
rows = []
for label, rule, m in [("daily", None, 252), ("weekly", "W", 52), ("monthly", "ME", 12)]:
r = spy if rule is None else prices["SPY"].resample(rule).last().pct_change().dropna()
n_i, k_i = len(r), stats.kurtosis(r, fisher=False)
vol_i = r.std(ddof=0) * np.sqrt(m)
rows.append({
"frequency": label,
"observations": n_i,
"years": n_i / m,
"SE(mean) %": vol_i / np.sqrt(n_i / m) * 100,
"SE(vol) %": vol_i * np.sqrt((k_i - 1) / (4 * n_i)) * 100,
})
pd.DataFrame(rows).set_index("frequency").round(2)| observations | years | SE(mean) % | SE(vol) % | |
|---|---|---|---|---|
| frequency | ||||
| daily | 5153 | 20.45 | 4.25 | 0.55 |
| weekly | 1068 | 20.54 | 3.92 | 0.86 |
| monthly | 245 | 20.42 | 3.31 | 0.84 |
The standard error of the mean refuses to fall. It is 3.3% computed from 245 monthly observations and 4.3% from 5,153 daily ones — slightly higher, in fact, where a naive count of observations would have promised a reduction by a factor of \(\sqrt{21} \approx 4.6\). What little movement there is across the three rows comes from the annualised volatility itself differing by frequency, not from the extra observations buying any information about the average.
The volatility column behaves quite differently, falling by a third from 0.84% at monthly frequency to 0.55% at daily. It is not perfectly monotone — weekly comes out marginally worse than monthly — because kurtosis also falls as the horizon lengthens, and eq. 1.9 depends on both \(n\) and \(\kappa\). The direction is nonetheless clear, and it is the opposite of what happened to the mean.
If you want a better estimate of expected return, then, the only lever is a longer history. And history arrives at the rate of one year per year.
1.10 What can go wrong
Everything above rests on an assumption that financial data violate. Observations were taken to be independent, and our bootstrap sampled days at random, which destroys the volatility clustering Figure 1.5 has already put on display. Calm days follow calm days and crises arrive in clusters, so a random scramble of the calendar is not a plausible alternative history.
The damage can be measured with a block bootstrap, which resamples contiguous 21-day blocks and so preserves short-run dependence while still reshuffling the sample.
def block_bootstrap(x, n, B, block, rng):
"""Resample n observations in contiguous blocks of length `block`."""
n_blocks = int(np.ceil(n / block))
starts = rng.integers(0, len(x) - block, size=(B, n_blocks))
idx = (starts[:, :, None] + np.arange(block)).reshape(B, -1)[:, :n]
return x[idx]
B2, n2 = 5_000, 252
iid_s = rng.choice(pop, size=(B2, n2), replace=True)
block_s = block_bootstrap(pop, n2, B2, block=21, rng=rng)
pd.DataFrame({
"SE(annualised mean) %": [iid_s.mean(axis=1).std(ddof=1) * 252 * 100,
block_s.mean(axis=1).std(ddof=1) * 252 * 100],
"SE(annualised variance)": [iid_s.var(axis=1, ddof=1).std(ddof=1) * 252,
block_s.var(axis=1, ddof=1).std(ddof=1) * 252],
}, index=["independent bootstrap", "block bootstrap"]).round(4)| SE(annualised mean) % | SE(annualised variance) | |
|---|---|---|
| independent bootstrap | 19.3453 | 0.0096 |
| block bootstrap | 16.1748 | 0.0242 |
The standard error of the mean falls once dependence is respected, which is the same conclusion the HAC standard error reached in Section 1.7 by a completely different route and for the same reason: \(\hat{\phi}\) is negative. Two methods that share no machinery agreeing on a counterintuitive result is about as much reassurance as empirical work offers.
The variance is a different story. Its standard error is roughly two and a half times larger under the block bootstrap, so the honest conclusion is worse than the one we drew from Table 1.9: even that interval, already three times wider than normal theory would suggest, is too narrow.
Three further cautions are worth stating plainly.
The population is not the truth. We treated one realised sample as though it were the population, which is a device rather than a fact. If the data-generating process changed over twenty years, and Figure 1.1 and Figure 1.5 both suggest it did, then no amount of resampling recovers a parameter that was never constant in the first place.
Non-normality does not disappear, it relocates. The central limit theorem rescues the mean, and we leaned on it heavily. It does nothing at all for the returns themselves, which is why a value-at-risk model built on a normal assumption still fails on exactly the days it matters.
Annualised volatility differs across frequencies in Table 1.11, 19.2% from daily data against 14.9% from monthly. That gap is not sampling error and it will not shrink with more data. It is a consequence of the autocorrelation we found in Section 1.7, and it deserves a lesson of its own.
Key takeaways
- Every reported statistic is one draw from a sampling distribution. The estimate is not the estimand, and this is as true of a GARCH parameter as of an average.
- \(\operatorname{SE}(\bar{X}) = \sigma/\sqrt{n}\), because independent errors only partly cancel. Quartering your error costs sixteen times the data, and every standard error in this course has the same shape.
- A standard deviation describes the data and does not shrink with more of it. A standard error describes your estimate and does.
- The central limit theorem makes the sample mean normal even when returns have a kurtosis of 18. That is what licenses \(t\)-statistics later without ever assuming returns are normal.
- The sample variance is unbiased but right-skewed, because squaring removes the cancellation that helps the mean. Its standard error depends on kurtosis and, for real returns, is about three times the normal-theory value.
- Least squares, method of moments and maximum likelihood all return \(\bar{X}\). They are the principles behind OLS, GMM and GARCH respectively, and the first two coincide whenever the likelihood is Gaussian.
- The sample mean is a regression on a constant and the sample variance is a constant-variance model fitted by maximum likelihood — exactly, not approximately. Regression, ARMA and GARCH generalise them by letting the mean or the variance depend on something.
- Dependence changes the standard error, not the estimate, and the direction depends on the sign of the dependence.
- Volatility is estimable; expected return is not. Twenty years of S&P 500 data place the average return somewhere between 3% and 20% a year, and a manager with a Sharpe ratio of 0.5 needs fifteen years to prove it.
Check your understanding
Five questions. Commit to an answer before opening the solution — the hints are written for the specific misunderstanding behind each wrong option, so they are only useful once you have picked one.
Question 1
A risk report states that the annualised volatility of an equity fund is 19.2%, and that the standard error of that estimate is 0.6%. An analyst concludes: “So the fund’s return will be within about 1.2% of its average in 95% of years.”
What is wrong with the conclusion?
- Nothing — that is the correct interpretation of a standard error.
- It confuses the standard error with the standard deviation.
- It should have used 1.96 standard errors rather than 2.
- The interval should be one-sided, not two-sided.
Show the answer
B is correct. The two numbers describe entirely different things. The standard deviation of 19.2% describes the data: annual returns really do vary by roughly that much, and that variation is a fact about markets. The standard error of 0.6% describes the estimate: it says our measurement of the volatility is accurate to within about a percentage point. The analyst has used the precision of the measurement as if it were the spread of the outcomes, understating the fund’s actual variability by a factor of about thirty.
The give-away is that a standard error shrinks as you collect more data, while a standard deviation does not. If the analyst’s reading were right, gathering more history would make equities less risky.
If you chose A — reread the callout above on standard deviation versus standard error. Ask yourself what happens to each number as \(n\) grows: a quantity that shrinks with sample size cannot be describing the riskiness of the asset.
If you chose C — 1.96 against 2 is a rounding detail worth about 2% of the interval width. The error here is roughly thirty-fold, so look for something structural rather than arithmetic.
If you chose D — a symmetric two-sided interval is exactly right for a question about being “within” some distance of the average. The problem is which quantity is being spread, not the shape of the interval.
Question 2
Daily equity returns have a kurtosis of about 18, far from the normal value of 3. Which statement about the central limit theorem is correct?
- The CLT does not apply, because the data are not normally distributed.
- The CLT will make the distribution of daily returns approach normality as more days are collected.
- The CLT makes the distribution of the sample mean approach normality, but says nothing about the returns themselves.
- The CLT applies only once the sample exceeds 30 observations.
Show the answer
C is correct. The theorem is a statement about averages, not about raw data. Figure 1.3 shows the sampling distribution of the mean becoming indistinguishable from a normal curve by \(n = 252\), even though every one of those samples was drawn from the same fat-tailed population. The returns keep their kurtosis of 18 forever; only their average becomes normal.
This distinction has teeth. It is why we may use \(t\)-statistics and confidence intervals on estimated parameters throughout the course, and simultaneously why a value-at-risk model that assumes normal returns fails in a crash.
If you chose A — this reverses the theorem’s purpose. The CLT exists precisely so that we do not need a normal population; that is what makes it useful rather than circular. It requires finite variance and near-independence, not normality.
If you chose B — collecting more days gives you more observations from the same distribution; it does not change that distribution. Plot a histogram of 5,000 daily returns and one of 500 and they have the same shape. Nothing about the raw data is converging to anything.
If you chose D — \(n = 30\) is folklore. Figure 1.2 shows a symmetric population that is essentially normal by \(n = 5\), and a skewed one still visibly non-normal at \(n = 250\). The requirement depends on the parent’s skewness, per eq. 1.5, not on a universal threshold.
Question 3
You have five years of daily returns and your estimate of the average return has a standard error of 8%. Your colleague suggests two options for improving it: (i) switch from daily to hourly data over the same five years, multiplying the number of observations by roughly seven; or (ii) extend the sample back to twenty years of daily data.
Which will reduce the standard error of the estimated average return, and by how much?
- Both, and (i) helps more because it adds more observations.
- Only (ii), roughly halving the standard error.
- Both, each roughly halving the standard error.
- Only (i), reducing it by a factor of about \(\sqrt{7}\).
Show the answer
B is correct. From eq. 1.16, the standard error of an annualised mean is \(\sigma_{\text{ann}}/\sqrt{T}\), where \(T\) is the span in years. The sampling frequency cancels out completely. Slicing the same five years more finely gives you more numbers but no additional information about the average: finer sampling tells you more about how the price moved within the period, while the average return over a period depends only on where it started and where it ended.
Extending to twenty years quadruples \(T\), so the standard error falls by \(\sqrt{4} = 2\) — from 8% to about 4%. Table 1.11 confirms the first half of this empirically: the standard error of the mean barely moves across daily, weekly and monthly sampling of the same twenty years.
If you chose A or D — you are counting observations rather than calendar time. This is the single most common misconception in the lesson. Work through eq. 1.16 and watch every \(m\) cancel; then note that the variance estimate genuinely does improve with frequency, which is why the intuition feels right but applies to the wrong moment.
If you chose C — the direction is right for (ii) but wrong for (i). Only option (ii) lengthens the calendar span.
Question 4
Using a year of daily returns, you compute a 95% confidence interval for volatility from the textbook chi-squared formula, \(\operatorname{Var}(s^2) = 2\sigma^4/(n-1)\). Compared with an interval that accounts for the fat tails of returns, your interval will be:
- Too wide, by a factor of about three.
- About right; the chi-squared formula is robust to non-normality.
- Too narrow, by a factor of about three.
- Too narrow, but only for small samples; the error vanishes as \(n\) grows.
Show the answer
C is correct. The chi-squared result is a normal-theory formula. The general expression is \(\operatorname{Var}(s^2) \approx \sigma^4(\kappa-1)/n\) from eq. 1.9, which reduces to the chi-squared answer only when \(\kappa = 3\). With \(\kappa \approx 18\) the variance of \(s^2\) is inflated by \((\kappa-1)/2 \approx 8.6\), so the standard error is inflated by \(\sqrt{8.6} \approx 2.9\). Table 1.7 confirms this by simulation.
Reporting an interval three times too narrow is not a harmless conservatism error — it runs the wrong way. You would be roughly three times more confident in your volatility estimate than the data warrant.
If you chose A — check the direction. Fat tails mean occasional enormous squared deviations, which make the variance estimate less stable across samples, not more. Extra uncertainty widens the honest interval.
If you chose B — the sample mean is robust to non-normality thanks to the CLT, and it is tempting to extend that comfort to the variance. It does not transfer: the variance estimate depends on the fourth moment of the data, so a fat-tailed population hits it directly.
If you chose D — look at the “sim / normal” column of Table 1.7. The ratio is about 2.9 at \(n = 21\) and still about 2.9 at \(n = 1260\). A constant discrepancy across sample sizes is the signature of a wrong formula, not a small-sample effect.
Question 5
You run an ordinary least squares regression of a stock’s returns on nothing but a constant — a single column of ones. Which is true of the results?
- The coefficient equals the sample mean, and its standard error equals \(s/\sqrt{n}\).
- The coefficient equals the sample mean, but the standard error requires a separate calculation unrelated to this lesson.
- The regression is not identified, because there are no explanatory variables.
- The coefficient equals the sample median, since OLS minimises absolute deviations.
Show the answer
A is correct, and exactly so — not approximately. Least squares chooses the constant that minimises \(\sum_i (X_i - \beta_0)^2\), which is the defining property of the sample mean, per eq. 1.10. The code in Section 1.7 prints both to ten decimal places and they agree, as do the standard error and \(s/\sqrt{n}\), and the residual variance and \(s^2\).
This is the most important idea in the lesson for what follows. The sample mean is not merely analogous to a regression; it is one, in the special case where the only regressor is a constant. Everything established here about sampling distributions, standard errors and the central limit theorem is therefore already a set of statements about regression. Adding a real regressor generalises the estimator without disturbing any of the concepts.
If you chose B — the standard error comes from the same \(\operatorname{Var}(\bar{X}) = \sigma^2/n\) derived in eq. 1.2. OLS reports \(\sqrt{s^2/n}\), which is \(s/\sqrt{n}\) with the estimate \(s\) in place of the unknown \(\sigma\) — precisely the substitution that motivates Student’s \(t\).
If you chose C — a constant is itself a regressor, so the model is perfectly well identified with one parameter. It fits every observation with the same number and asks which single number fits best.
If you chose D — OLS minimises squared deviations, which gives the mean. Minimising absolute deviations gives the median, and that is a different estimator (quantile regression) with quite different properties.