Analysis of Variance, Regression, and the “General Linear Model”

Analysis of variance (ANOVA) and linear regression are two of the most popular statistical techniques used in behavioral research.  I’ve often come across statements to the effect that ANOVA and linear regression are really “the same thing” – that in some sense they are special cases of something called “the general linear model”.  And yet, in many statistics texts there are few details given of exactly how this works – exactly how, say, ANOVA is part of a general linear model.

I recently came across a classic article that addresses this point directly, Jacob Cohen’s “Multiple Regression as a General Data-Analytic System” (1968).

Cohen has written many articles on the proper use of statistics in psychological research (and in research in general).  Statistics clearly plays an important role in correctly designing and analyzing experiments, but many researchers quite naturally are not statisticians, and there is often a gap between statistical knowledge (of statisticians) and the everyday application of statistics by researchers in other fields.  Cohen has made many contributions to address this divide, by bringing a knowledge of mathematical statistics to bear on research practices.

In this article Cohen shows that analysis of variance, analysis of covariance, and other statistical techniques can be converted into equivalent cases of multiple regression.  Once he has given details about how this can be done (including how to carry out the requisite hypothesis tests, etc.), he goes on to discuss why multiple regression is ultimately a more flexible and powerful technique, and thus can be viewed as a “general linear model”.

One of the main advantages of multiple regression is that it can accommodate a wide range of situations easily – interval/ratio variables, categorical variables, interactions, and so forth – all in a single model.  ANOVA and other specific techniques (such as t tests) are specialized in that they can only handle a fairly limited range of situations.  Multiple regression, on the other hand, can handle many different cases, all at the same time in a single model.

Reference

Cohen, J. (1968). Multiple regression as a general data-analytic system. Psychological Bulletin, 70, 426-443.