Nonparametric regression is a category of regression analysis in which the predictor does not take a predetermined form but is constructed according to information derived from the data. That is, no parametric form is assumed for the relationship between predictors and dependent variable. Nonparametric regression requires larger sample sizes than regression based on parametric models because the data must supply the model structure as well as the model estimates.

Definition

In nonparametric regression, we have random variables ${\displaystyle X}$ and ${\displaystyle Y}$ and assume the following relationship:

${\displaystyle \mathbb {E} [Y\mid X=x]=m(x),}$

where ${\displaystyle m(x)}$ is some deterministic function. Linear regression is a restricted case of nonparametric regression where ${\displaystyle m(x)}$ is assumed to be affine. Some authors use a slightly stronger assumption of additive noise:

${\displaystyle Y=m(X)+U,}$

where the random variable ${\displaystyle U}$ is the `noise term', with mean 0. Without the assumption that ${\displaystyle m}$ belongs to a specific parametric family of functions it is impossible to get an unbiased estimate for ${\displaystyle m}$, however most estimators are consistent under suitable conditions.

List of general-purpose nonparametric regression algorithms

This is a non-exhaustive list of non-parametric models for regression.

• nearest neighbors, see nearest-neighbor interpolation and k-nearest neighbors algorithm
• regression trees
• kernel regression
• local regression
• multivariate adaptive regression splines
• smoothing splines
• neural networks^[1]

Examples

Kernel regression

Main article: Kernel regression

Example of a curve (red line) fit to a small data set (black points) with nonparametric regression using a Gaussian kernel smoother. The pink shaded area illustrates the kernel function applied to obtain an estimate of y for a given value of x. The kernel function defines the weight given to each data point in producing the estimate for a target point.

Kernel regression estimates the continuous dependent variable from a limited set of data points by convolving the data points' locations with a kernel function—approximately speaking, the kernel function specifies how to "blur" the influence of the data points so that their values can be used to predict the value for nearby locations.

See also

• Lasso (statistics)
• Local regression
• Non-parametric statistics
• Semiparametric regression
• Isotonic regression
• Multivariate adaptive regression splines

References

Further reading

• Bowman, A. W.; Azzalini, A. (1997). Applied Smoothing Techniques for Data Analysis. Oxford: Clarendon Press. ISBN 0-19-852396-3.
• Fan, J.; Gijbels, I. (1996). Local Polynomial Modelling and its Applications. Boca Raton: Chapman and Hall. ISBN 0-412-98321-4.
• Henderson, D. J.; Parmeter, C. F. (2015). Applied Nonparametric Econometrics. New York: Cambridge University Press. ISBN 978-1-107-01025-3.
• Li, Q.; Racine, J. (2007). Nonparametric Econometrics: Theory and Practice. Princeton: Princeton University Press. ISBN 978-0-691-12161-1.
• Pagan, A.; Ullah, A. (1999). Nonparametric Econometrics. New York: Cambridge University Press. ISBN 0-521-35564-8.

External links

Wikimedia Commons has media related to Nonparametric regression.

• HyperNiche, software for nonparametric multiplicative regression.
• Scale-adaptive nonparametric regression (with Matlab software).