Talk:Bayesian linear regression

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search
WikiProject Statistics (Rated C-class, High-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

C-Class article C  This article has been rated as C-Class on the quality scale.
 High  This article has been rated as High-importance on the importance scale.
WikiProject Mathematics (Rated C-class, Low-importance)
WikiProject Mathematics
This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of Mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Mathematics rating:
C Class
Low Importance
 Field:  Probability and statistics

Bayes or Empirical Bayes?[edit]

This isn't a description of Bayesian linear regression. It's a description of the Empirical Bayes approach to linear regression rather than a full Bayesian approach. (Empirical Bayes methods peek at the data to ease the computational burden.) Not only that, it assumes a natural conjugate prior, which is a reasonable approach in many cases, but it is a serious constraint. The fully Bayesian approach with non-conjugate priors is nowadays tractable in all but the largest models through the use of Markov chain Monte Carlo techniques. In my view, this article represents a particular and rather dated approach. Blaise 21:53, 10 April 2007 (UTC)

Can you fix it and check my rating? Thanks - Geometry guy 14:19, 13 May 2007 (UTC)
I don't agree with the first comment, in that it is a full Bayesian approach as opposed to a max likelihood approach. It's true that a conjugate prior is limiting but it is also an enabler for application to very high dimensional input spaces. Also, this same approach can be extended into a kind of adaptive-variance Kalman filter in cases where the model parameters are expected to change through time. i.e. I think the page is very valuable, but would also like to see how to derive matrix A.
Malc S, January 2009.
— Preceding unsigned comment added by (talkcontribs) 12:02, 30 January 2009‎

Some questions[edit]

I've got some questions about this article. What this method gives you is a weighted combination of some prior slope, and a new slope estimate from new data.

Q1) the weights are determined by the A. In this one-dimensional case I presume this is just a variance. There are no details as to how this A would be calculated or estimated. Does anyone know?

Q2) I could set this problem up using classical statistics, I think. I'd say "let's make a prediction based on a weighted combination of the prior slope and the new slope". Then I'd do some algebra to derive an expression for the weight. Does anyone have any idea whether the final answer would be much, if any, different?

thanks 20:09, 14 October 2007 (UTC)


Maybe a section on computation would be helpful. When the amount of data is large, direct computation can be difficult. — Preceding unsigned comment added by (talkcontribs) 14:00, 7 August 2011‎

I would suggest that the computations be kept separate from the analytic results using the conjugate prior. I'm doing research, and I come to this page, and all I want is the posterior. It is there, but it is buried in computations. I don't care about the computations; I already know how to do them. I'm not saying get rid of the computations, but not having the posterior, posterior mean, etc, all available quickly makes this article less useful. -- (talk) 19:08, 9 September 2012 (UTC)

Suggested merge[edit]

I've added a suggested merge tag, suggesting the page be merged into ridge regression, since this is essentially a Bayesian interpretation of ridge regression, which has exactly the same mathematics. Jheald (talk) 17:39, 11 November 2012 (UTC)

Proposed merge with Bayesian multivariate linear regression[edit]

Single-variable linear regression is just a special case of multivariable. I see no reason to have two articles; if anything, it's confusing. InverseHypercube (talk) 07:11, 27 March 2016 (UTC)

Oppose merge; while it is true that the single-variable case is a special case of the multivariable one, it is helpful to keep them separate because those without a mathematical specialism may more easily understand the single-variable case. Klbrain (talk) 21:09, 8 January 2018 (UTC)
Closing, given lack of support. Klbrain (talk) 09:37, 15 February 2018 (UTC)

Assessment comment[edit]

The comment(s) below were originally left at Talk:Bayesian linear regression/Comments, and are posted here for posterity. Following several discussions in past years, these subpages are now deprecated. The comments may be irrelevant or outdated; if so, please feel free to remove this section.

Geometry guy 00:02, 22 May 2007 (UTC) There is a variable A introduced with no explanation. Later it is describes as coming from the Cholesky decomposistion (U). The comments do not seem to be in synch with the expresssions. —Preceding unsigned comment added by (talk) 15:16, 28 March 2009 (UTC)

Last edited at 15:17, 28 March 2009 (UTC). Substituted at 19:49, 1 May 2016 (UTC)

Suggestion: Intro paragraph[edit]

A generic problem with science and math articles is approachability. Stats articles like this one would really benefit from a plain/simple english paragraph explaining what the topic means to a general audience. Keep the first more technical introduction paragraph, but have a second paragraph that explains it to your neighbor who wants to make a genuine attempt at understanding what the topic is about but isn't a stats professor.

However, the technical definition needs work too: In statistics , Bayesian linear regression is an approach to linear regression in which the statistical analysis is undertaken within the context of Bayesian inference.

When you say 'In statistics' in the sentence do you really have to say "the statistical analysis"? I mean, in statistics can't we infer that analysis is statistical? Precision and painfully descriptive language is necessary in academia, but isn't that just a tad excessive? Additionally, while maybe technically correct, that definition sounds very circular and not informative. Like surprise surprise Bayesian linear regression refers to is linear regression using Bayesian' methods. Okay... Does that actually tell me anything that I didn't already know? It sounds like you defined the term by repeating the term. Can this please be hashed out more?

What about Inference?[edit]

In linear regression, one of the most important results is a confidence interval on the parameter estimates. The bayesian analog is, I believe, the credibility interval. To most practitioners (of merit), a parameter estimate without an accompanying confidence interval is more or less useless. So why no section on inference/credibility intervals in this article? That is a big part of looking at results of models but there is no mention at all here, so I think the article is missing a crucial component. Chafe66 (talk) 20:45, 19 April 2018 (UTC)