What is a Sufficient Statistic 🌱

Definition

If $Y_1,\dots ,Y_n$ are iid samples drawn from a distribution with parameter $\theta$ then a statistic $U=U(Y_1,\dots ,Y_n)$ is said to be sufficient for $\theta$ iff the conditional distribution of the data $Y_1,\dots ,Y_n\mid U$ does not depend on the parameter $\theta$.

Method to Find Them

If the likelihood can be expressed as the following, then $U(Y_1,\dots ,Y_n)$ is a sufficient statistic

Minimum Sufficient Statistic

A minimal sufficient statistic $T(Y_1,\dots ,Y_n)$ is a sufficient statistic that can be expressed as some function of any other sufficient statistic $\tilde{T}(Y_1,\dots ,Y_n)$:

Finding a minimal sufficient statistic

Let $x_1,\dots ,x_n\sim f(x;\theta )$ and $y_1,\dots ,y_n\sim f(x;\theta )$ be two samples from the same distribution and $T$ be a statistic. Suppose:

If for all $\theta$ where $R$ is defined we have:

then $T$ is a minimal sufficient statistic.

Rao - Blackwell Theorem

Let $\hat{\theta }$ be an unbiased estimator for $\theta$ such that $V\left(\hat{\theta }\right)<\infty$. If $U$ is a sufficient statistic for $\theta$, then for all $\theta$: