| Factor Analysis |
Factor analysis is most commonly used to group variables, cluster analysis is most commonly used to group customers, and MDS is most commonly used to group products. However, each procedure can be used for other purposes as well. The procedures are differentiated not so much by their purposes as by the methods and/or data used to accomplish these purposes.
Factor analysis is a procedure that groups items -usually variables- on the basis of correlations. A factor analysis will form groups of variables that have strong correlations (either positive or negative) with one another. Because the technique relies upon correlations, all variables should be quantitative in nature, though dummy (0-1) variables can be used.
More specifically, a factor analysis calculates a series of factors, each of which is a weighted combination of the variables being analyzed. These combinations take the form:
F = w1x1 + w2x2 + ... +wkxk
where F is the factor, x1 through xk are the variables being analyzed, and w1 through wk are weights applied to these variables. The weights for each factor are determined so as to maximize the sum of squared correlations between this factor and the various contributing variables, subject to a constraint that each factor be uncorrelated with all preceding factors.
| Overview of Descriptive Results |
The key descriptive results obtained from a factor analysis are factor loadings, eigenvalues, and possibly factor scores, as follows:
| Factor loadings are the
correlations between a factor and the individual variables being analyzed.
For example, if a factor analysis shows that V6 has a loading of -.12 on
Factor 2, this means that the correlation between V6 and the second factor
is -.12.
Each factor will have loadings for all of the variables being analyzed. Variables that have loadings with absolute values larger than .50 are said to "load highly" on the factor and are considered to be members of a group of variables identified by the factor. Variables that have loadings with absolute values of less than .50 are usually ignored in interpreting the factor. For example, in the SPORTS data, if V7 (importance of everyday prices) and V8 (importance of sale prices) are the only two variables to load higher than .50 on a particular factor, this factor identifies a group formed by V7 and V8. Since both of the variables relate to price, this factor would be referred to as a "price" factor. Note that the names of the factors do not come directly from the data but depend on the researcher's interpretation. The eigenvalue for a factor equals the sum of the squared loadings for all variables on that factor -in other words, the sum of the squared correlations between that factor and all of the variables in the analysis. Since the first factor is chosen so as to maximize the sum of the squared correlations without any constraints, the first factor naturally has the largest eigenvalue. The second factor has the second largest eigenvalue, the third factor has the third largest eigenvalue, and so on. The eigenvalues provide a measure of the percentage of variance in the contributing variables that is "explained" by the factor. The sum of the eigenvalues represents the total amount of variance to be explained in the analysis, and the ratio of each individual eigenvalue to that sum indicates the percentage of variance explained by the relevant factor. For example, if the sum of the eigenvalues in a factor analysis is 11.00 and the eigenvalue for the first factor is 2.17, the first factor accounts for 2.17/11.00 = .197, or 19.7%, of the total variance. ! The third descriptive measure available from a factor analysis is factor scores. When a factor analysis is used to group variables, the resulting factors can be treated as new variables that represent combinations of the original variables. Appropriate values for each observation on these new variables (the factors) can be calculated. These values are called "factor scores." As a general rule, statistical software packages that have factor analysis routines will calculate and save factor scores if desired. |
| Overview of Inferential Results |
Three types of inferential analyses are appropriate in a factor analysis:
| Testing
whether the overall analysis is "significant" in the sense of
effectively accomplishing its grouping objectives.
Testing whether a particular factor makes a significant contribution to the overall analysis, and consequently should be retained for purposes of interpretation and/or further analysis. Testing whether a particular variable is significantly associated with a particular factor and consequently should be considered part of the group of variables defined by that factor. |
In factor analysis, these inferential goals are primarily achieved by using "rules of thumb" rather than formal statistical tests (Acito, Anderson, & Engledow, 1980). These rules include the following:
| The
overall factor analysis generally can be considered effective if the total
percentage of variance explained by the retained factors exceeds 70%. If
the retained factors account for less than 70% of the variance, this fact
should be noted in reports.
This is not to say there is anything wrong with the retained factors when the explained variance is less than 70%. It simply means that the original variables contained a substantial amount of information that the factors were not able to capture, so the factors should not be viewed as a good summary of the original variables. An alternative, nonquantitative criterion for determining the effectiveness of a factor analysis is simply to judge whether the factors can be meaningfully interpreted and are useful for some purpose. If the results are meaningful and useful, the analysis should be deemed a success. The number of "significant" or meaningful factors can be determined in various ways. The most common approach is to retain all factors with eigenvalues larger than 1.0. In most applications of factor analysis, an eigenvalue of 1.0 is regarded as the amount of variance attributable to a single variable. Therefore, factors with eigenvalues of less than 1.0 are viewed as "explaining" less than one variable's worth of variance. Since the purpose of the analysis is to form groups with two or more variables, these factors are considered to be nonsignificant and are dropped from further consideration. A second approach is to use a scree test, a procedure that uses decreases in eigenvalues to determine the factors to be retained. A scree test can be done either graphically or numerically; the eigenvalues are arranged in descending order and a dramatic drop between two eigenvalues is looked for. If a dramatic drop is seen, it might be best to retain only the factors whose eigenvalues come before that drop. For example, if the first eigenvalue is 5.44, the second eigenvalue is 4.32, and the third eigenvalue is 1.40, only the first two factors might be considered significant. If a dramatic drop is not seen, it is time to fall back on the rule of thumb that significant factors must have eigenvalues greater than 1.0. A third approach is to retain factors according to theoretical and/or interpretive judgments. For example, a factor analysis of V6 to V16 in the Sportdat data might produce four factors with eigenvalues larger than 1.0, and V6 (location) might not load highly on any of these factors because this variable is unique. Rather than proceed with an analysis that ignores location as a store choice factor, one can simply assert a fifth factor consisting of V6 by itself. In measuring the contribution of individual variables, each variable is usually assigned to the factor for which it has the highest loading (measured in terms of absolute value). If this loading is larger than .50, the variable is considered to be part of the group of variables defined by the factor. Variables with loadings of less than .50 usually are ignored in interpreting factors. In some situations, this quantitative rule will be overridden by judgmental considerations. For example, a variable with a loading in the .35 to .50 range might be assigned to a particular factor if the variable has a theoretical relationship with other variables that load highly on that factor, especially if the variable does not load highly on any other factor. Similarly, a variable that loads highly on a factor but bears no explainable relationship to the other high-loading variables might be ignored in interpreting the factor. |
On to an Example and Additional Commentary!
Where applicable, © 2009, David
M. Compton