Introduction

Two factors are crossed if all combinations of the two factors are used in the experiment. A two-way model can investigate the interaction effects.

Notations:

Example (Battery Experiment) Two treatment factors: Duty (two levels: alkaline and heavy duty, coded as 1 and 2, respectively ) and Brand (two levels: name brand and store brand, coded as 1 and 2, respectively). The two factors are crossed. The four treatment combinations were coded as 1, 2,3, and 4, and one-way model was used in previous chapters to analyze the effects of the treatments.

If we code the treatment combinations as \(11,12,21\) and 22 , the same one-way model can be equivalently written as

\[\begin{equation} Y_{i j t}=\mu_{i j}+\varepsilon_{i j t}, i=1,2, j=1,2, t=1, \ldots, r_{i j} \tag{1} \end{equation}\]

where \(\varepsilon_{i j}\) are i.i.d \(N\left(0, \sigma^2\right)\). This is also called a cell-mean model.

In this chapter, we investigate the contributions that each of the factors individually makes to the response. We could either have a 2-way complete model or a 2-way main effects model (if it is assured that this is no interaction).

Two-Way Complete Models

Given any numbers \(\mu_{ij},i=1, \cdots, a, j=1, \cdots, b\), we can always write

\[\begin{equation} \mu_{i j}=\mu+\alpha_i+\beta_j+(\alpha \beta)_{i j}, i=1, \cdots, a, j=1, \cdots, b \tag{2} \end{equation}\]

where \(\sum_i^a \alpha_i=0\), \(\sum_j^b \beta_j=0\), and \(\sum_{i=1}^a (\alpha\beta)_{ij}=0\) for any \(j\), \(\sum_{j=1}^b (\alpha\beta)_{ij}=0\) for any \(i\).

Then the model becomes \[ Y_{i j t}=\mu+\alpha_i+\beta_j+(\alpha \beta)_{i j}+\varepsilon_{i j t}, i=1,2, j=1,2, t=1, \ldots, r_{i j}. \]

This is called the two-way complete model or two-way ANOVA model. It is just a rewriting of model (1). Note

\[\begin{equation} \begin{split} &\alpha_i=\bar{\mu}_{i \cdot}-\bar{\mu}_{\cdot\cdot}, ~ \beta_j=\bar{\mu}_{\cdot j}-\bar{\mu}_{\cdot\cdot},\\ & (\alpha \beta)_{i j} =\mu_{i j}+\bar{\mu}_{\cdot\cdot}-\bar{\mu}_{i \cdot}-\bar{\mu}_{\cdot j} . \end{split} \end{equation}\]

The Meaning of Interaction

An example: Factor A with 3 levels and Factor B with 2 levels. Consider two cases for the 6 treatment means as shown below.

The left-side indicates no interaction effects, i.e., \((\alpha \beta)_{i j}=0\) for all \(i, j\). Then \[ \mu_{ij}=\mu+\alpha_i+\beta_j. \]

Hence the differences between the two levels of B, say, 1 and 2, are the same regardless the level of \(A\). Specifically, \[ \mu_{i2}-\mu_{i1}=\beta_2-\beta_1=4, \text{ for all } i. \]

However, the right-side shows that the differences between the two levels of Factor B depends on the levels of A. Specifically \[ \mu_{12}-\mu_{11}=4, \mu_{22}-\mu_{21}=2, \mu_{32}-\mu_{31}=7. \]

We will say there are interaction effects in the right-side case, and no interaction effects in the left-side case.

In reality, the means \(\mu_{ij}\) are unknown and we shall plot the sample means instead. This results in an interaction plot.

Two-Way Main Effects Model

When no interaction effects exist, we have a two-way main effects model \[ Y_{i j t}=\mu+\alpha_i+\beta_j+\varepsilon_{i j t}, i=1,2, j=1,2, t=1, \ldots, r_{i j}. \]

Contrast in the Two-Way Complete Model

As in the one-way model, a contrast is of the form \(\sum_{i}\sum_jc_{ij}\mu_{ij}\) where \(\sum_{i}\sum_jc_{ij}=0\).

Interaction Contrasts

If the coefficients satisfy the additional conditions \(\sum_i c_{i j}=0\) for each \(j\) and \(\sum_j c_{i j}=0\) for each \(i\), \[ \sum_i \sum_j c_{i j} \mu_{i j}=\sum_i \sum_j c_{i j}(\alpha \beta)_{i j} \] is called an interaction contrast.

Example of interaction contract:

\[ (\mu_{12}-\mu_{11})-(\mu_{22}-\mu_{21}). \]

Q1: Is \(\frac{\mu_{11}+\mu_{21}}{2}-\frac{\mu_{12}+\mu_{22}}{2}-(\mu_{31}-\mu_{32})\) a contrast? Is it an interaction contrast?

Q2: Is \(\frac{\mu_{11}+\mu_{21}+\mu_{31}}{2}-\frac{\mu_{12}+\mu_{22}+\mu_{32}}{2}\) a contrast? Is it an interaction contrast?

Main Effects Contrasts

How to measure the difference between two levels of Factor B in the presence of interaction effects? A reasonable choice is

\[ (\mu_{12}+\mu_{22}+\mu_{32})/3-(\mu_{11}+\mu_{21}+\mu_{31})/3 \\ =\bar \mu_{\cdot 2}-\bar \mu_{\cdot 1}= (\beta_2+(\alpha\beta)_{\cdot 2})-(\beta_1+(\alpha\beta)_{\cdot 1}). \] In general, a contrast in the main effects of Factor B takes the form \[ \sum_j k_j \bar{\mu}_{. j}=\sum_j k_j\left(\beta_j+(\overline{\alpha \beta})_{. j}\right), \] where \(\Sigma k_j=0\) and \((\overline{\alpha \beta})_{. j}=\frac{1}{a} \sum_i(\alpha \beta)_{i j}\).

Note the contrast here is NOT \(\sum_j k_j \beta_j\).

A contrast in the main effects if Factor A takes the form \[ \sum_i c_i \bar{\mu}_{i\cdot}=\sum_i c_i\left(\alpha_i+(\overline{\alpha \beta})_{i .}\right) \] where \(\sum_i c_i=0\) and \((\overline{\alpha \beta})_{i\cdot}=\frac{1}{b} \sum_{j=1}^b(\alpha \beta)_{i j}\).

Note the main effect contrast here is NOT \(\sum_j k_j \beta_j\) or \(\sum_i c_i \alpha_i\).

The Essence of Contrast

Any contrast is a linear combination of the treatment means with the contrast coefficients adding up to be 0. The interaction contrasts and main effects contrasts are particular contrasts with additional requirements on the coefficients.

Contrast for Two Particular Treatments

Unlike in the one-way model, the contrast for two treatment means, say, \(\mu_{22}-\mu_{11}\), involves both the main effects and the interaction effects.

\[ \mu_{22}-\mu_{11}=(\mu+\alpha_2+\beta_2+(\alpha\beta)_{22})-(\mu+\alpha_1+\beta_1+(\alpha\beta)_{11}) \\ =(\alpha_2-\alpha_1)+(\beta_2-\beta_1)+(\alpha\beta)_{22}-(\alpha\beta)_{11}. \]

The first two terms are main effects contrasts but the third term is not an interaction contrast!

Analysis of the Two-Way Complete Model

Least Squares Estimators

The model parameters are \(\sigma^2\) and \(\mu_{ij}\), \(i=1, 2, \ldots, a, j=1, 2, \ldots, b\). These are the same parameters as in the one-way model. Therefore, the least squares estimators are the sample means:

\[ \hat \mu_{ij}=\bar Y_{ij}. \] \(\bar Y_{ij}\) has a normal distribution with mean \(\mu_{ij}\) and variance \(\sigma^2/r_{ij}\), where \(r_{ij}\) denotes the sample size of treatment \(ij\).

Estimator for \(\sigma^2\) is the same as in the one-way model: MSE.

The confidence upper limit is given by

\[ \sigma^2 \leq \frac{s s E}{\chi_{n-a b, 1-\alpha}^2} \]

Contrasts and Inferences for Contrasts

A 100(1- \(\alpha)\) )% confidence interval for a single contrast is of the form

\[ \text{estimate}\pm t_{n-ab,\alpha/2}\text{ (standard error of estimate).} \]

Multiple Comparisons for the Complete Model

Tukey’s method for all pairwise comparisons and Dunnett’s method for treatment-versus-control are applicable to main effects contrasts or interaction contrasts. However, these methods do not provide a simultaneous error rate for both main effects and interaction contrasts are considered. One can apply Bonferroni method to achieve a simultaneous error rate. For example, one intends to have 95% simultaneous for all pairwise comparisons of the main effects of Factor A, and all pairwise comparisons for all the interaction interaction effects. The Bonferroni method implies the simultaneous error rate will be 95% if one obtains the 97.5% simultaneous confidence intervals for the all pairwise comparisons of Factor A and 97.55% simultaneous confidence intervals for all pairwise comparisons of the interaction contrasts. Then all the intervals together have a simultaneous error rate 95%.

For multiple contrasts, the 100(1- \(\alpha) \%\) simultaneous confidence intervals are of the form \(\text{estimate} \pm w \times \text{(std error of estimate)}\), where \(w\) varies with each method.

For example, for comparing main effects of factor \(A\), use the \(w\) values provided in the following table:

Method Bonferroni Scheffe Tukey Dunnett
\(w\) \(t_{n-a b, \alpha /(2 m)}\) \(\sqrt{(a-1) F_{a-1, n-a b, \alpha}}\) \(\frac{q_{a, n-a b, \alpha}}{\sqrt{2}}\) needs the multivariate t-distribution
Table A.4, p802 F value in A.6,804 q value in A.8, p814 A.10, 818

The \(w\) values can be given similarly for the multiple contrasts concerning the interaction contracts or the main effects contracts for Factor \(B\).

\(w\) values for interaction contrasts:

Method Bonferroni Scheffe Tukey Dunnett
\(w\) \(t_{n-a b, \alpha /(2 m)}\) \(\sqrt{(b-1) F_{b-1, n-ab, \alpha}}\) \(\frac{q_{b, n-a b, \alpha}}{\sqrt{2}}\) needs the multivariate t-distribution
Table A.4, p802 F value in A.6,804 q value in A.8, p814 A.10, 818

\(w\) values for interaction contrasts:

Method Bonferroni Scheffe Tukey Dunnett
\(w\) \(t_{n-a b, \alpha /(2 m)}\) \(\sqrt{(ab-1) F_{ab-1, n-a b, \alpha}}\) \(\frac{q_{ab, n-a b, \alpha}}{\sqrt{2}}\) needs the multivariate t-distribution
Table A.4, p802 F value in A.6,804 q value in A.8, p814 A.10, 818 
data react;
input Order Trtmt A  B    y;
lines;
  1     6   2  3  0.256
  2     6   2  3  0.281
  3     2   1  2  0.167
  4     6   2  3  0.258
  5     2   1  2  0.182
  6     5   2  2  0.283
  7     4   2  1  0.257
  8     5   2  2  0.235
  9     1   1  1  0.204
 10     1   1  1  0.170
 11     5   2  2  0.260
 12     2   1  2  0.187
 13     3   1  3  0.202
 14     4   2  1  0.279
 15     4   2  1  0.269
 16     3   1  3  0.198
 17     3   1  3  0.236
 18     1   1  1  0.181
;
run;
PROC GLM data=react;
    CLASS A B;
    MODEL Y = A B A*B;
    LSMEANS A/PDIFF CL ALPHA=0.01;
    LSMEANS B/PDIFF = ALL CL ADJUST = TUKEY ALPHA = 0.01;
    LSMEANS A*B/pdiff=ALL ajust=Tukey;
run;

proc glm data=react;
class Trtmt;
model Y=trtmt;
lsmeans trtmt/pdiff=all adjust=Tukey;
run;

Note the order of factors in the CLASS statement determines how the treatments are coded. The order of factors in the MODEL statement makes no difference. Run the following code and compare with the LSMEANS output from the previous results.

PROC GLM data=react;
    CLASS B A;
    MODEL Y = A B A*B;
    LSMEANS A/PDIFF CL ALPHA=0.01;
    LSMEANS B/PDIFF = ALL CL ADJUST = TUKEY ALPHA = 0.01;
    LSMEANS A*B/pdiff=ALL ajust=Tukey;
run;

ANOVA for the Two-Way Complete Model

There are three standard hypotheses that are usually examined when the two-way complete model is used. The first hypothesis is that the interaction between treatment factors \(A\) and \(B\) is negligible; that is,

\[\begin{equation} H_0^{A B}:(\alpha \beta)_{i j}=0, \text{ for all } i, j. \tag{3} \end{equation}\]

Note the textbook states the same hypothesis in an alternative but equivalent way:

\[\begin{equation} H_0^{A B}:\left\{(\alpha \beta)_{i j}-(\alpha \beta)_{i q}-(\alpha \beta)_{s j}+(\alpha \beta)_{s q}=0 \text { for all } i \neq s, j \neq q\right\}. \tag{4} \end{equation}\]

One can verify (4) can be expressed as

\[\begin{equation} H_0^{A B}:\left\{\mu_{i j}-\mu_{i q}-\mu_{s j}+\mu_{s q}=0 \text { for all } i \neq s, j \neq q\right\}. \tag{5} \end{equation}\]

The interpretation of (5) is clear: the pairwise difference for any two levels of a factor does not depend on the level of another factor. That is what no-interaction means.

The other two standard hypotheses are the main-effect hypotheses:

\[ H_0^A: \bar \mu_{1\cdot}=\ldots=\bar \mu_{a\cdot} \\ H_0^B: \bar \mu_{\cdot 1}=\ldots=\bar \mu_{\cdot b} \] where \(\bar\mu_{i\cdot}=(1/b)\sum_{j=1}^b \mu_{ij}\) and \(\bar\mu_{\cdot j}=(1/a)\sum_{i=1}^a \mu_{ij}\).

Again, note the textbook expresses these two hypothesis in a different but equivalent way by introducing more notations.

Testing Interactions

We test \(H_0^{AB}\) in (3) against the alternative hypothesis \(H_A^{A B}:\{\) the interaction is not negligible \(\}\). The idea is to compare the sum of squares for error \(ssE\) under the two-way complete model with the sum of squares for error \(s s E_0^{A B}\) under the reduced model obtained when \(H_0^{A B}\) is true. The difference \[ s s A B=s s E_0^{A B}-s s E \] is called the sum of squares for the interaction \(A B\), and the test rejects \(H_0^{A B}\) in favor of \(H_A^{A B}\) if \(s s A B\) is large relative to \(s s E\). Under \(H_0^{AB}\), \(ssAB/\sigma^2\) has an \(F\) distribution with degrees of freedom equal to the number of parameters reduced.

Note \(H_0^{AB}\) has \((a-1)(b-1)\) constraints and each constraint reduces the number of parameter by 1. Hence the degree of freedom of the \(F\) distribution is \((a-1)(b-1)\). Hence \[\begin{equation} F=\frac{ssAB/(a-1)(b-1)}{ssE/(n-ab)}\sim F_{(a-1)(b-1), n-ab} \end{equation}\]

We reject \(H_0^{AB}\) at significance level \(\alpha\) if the test statistic \(F>F_{(a-1)(b-1), n-ab, \alpha}\) or the p-value\(<\alpha\).

Note here I use \(ssAB\) to denote a random variable (when talking about distribution) as well as a particular value obtained from the experimental data. The book distinguishes these two by using different notations \(SS(AB)\) and \(ssAB\).

Let us see how \(ssAB\) is calculated when the sample sizes all equal to \(r\).

Write \(\mu_{ij}=\mu+\alpha_i+\beta_j+(\alpha \beta)_{i j}\) where \[\begin{equation} \begin{split} &\alpha_i=\bar{\mu}_{i \cdot}-\bar{\mu}_{\cdot\cdot}, ~ \beta_j=\bar{\mu}_{\cdot j}-\bar{\mu}_{\cdot\cdot},\\ & (\alpha \beta)_{i j} =\mu_{i j}+\bar{\mu}_{\cdot\cdot}-\bar{\mu}_{i \cdot}-\bar{\mu}_{\cdot j} . \end{split} \end{equation}\]

Under \(H_0^{AB}\), \((\alpha\beta)_{ij}=0\). We therefore have \[ \mu_{ij}=\bar\mu_{i\cdot}+\bar\mu_{\cdot j}-\bar\mu_{\cdot \cdot}. \] whose least squares estimator when the sample sizes are equal is \[ \hat\mu_{ij}=\bar{Y}_{i \cdot\cdot}+\bar{Y}_{\cdot j \cdot}-\bar{Y}_{\cdot\cdot \cdot}. \] Then \[ \begin{aligned}ssE_0^{AB} &=\sum_i \sum_j \sum_t\left(y_{i j t}-\bar{y}_{i . .}-\bar{y}_{j .}+\bar{y}_{\ldots . .}\right)^2 \\ &=\sum_i \sum_j \sum_t\left(y_{i j t}-\bar{y}_{i j .}\right)^2+r \sum_i \sum_j\left(\bar{y}_{i j .}-\bar{y}_{i .}-\bar{y}_{j .}+\bar{y}_{\cdot\cdot\cdot}\right)^2\\ &=ssE+r \sum_i \sum_j\left(\bar{y}_{i j .}-\bar{y}_{i .}-\bar{y}_{j .}+\bar{y}_{\cdot\cdot\cdot}\right)^2. \end{aligned} \] Therefore,

\[\begin{aligned} s s A B &=s s E_0^{A B}-s s E \\ &=r \sum_i \sum_j\left(\bar{y}_{i j .}-\bar{y}_{i .}-\bar{y}_{. j .}+\bar{y}_{\ldots .}\right)^2 \\ &=r \sum_i \sum_j \bar{y}_{i j .}^2-b r \sum_i \bar{y}_{i .}^2-a r \sum_j \bar{y}_{. j .}^2+a b r \bar{y}_{\ldots}^2 \end{aligned} \] This formula holds only when the sample sizes are all equal to \(r\). When sample sizes are unequal, the formula becomes more complex but can be expressed using matrix and vector notations.

Testing Main Effects

Consider testing \(H_0^A: \bar \mu_{1\cdot}=\ldots=\bar \mu_{a\cdot}.\) This reduced model reduces the number of parameters by \((a-1)\). To fined the SSE under the reduced model, first note the least squares estimator of \(\mu_{ij}\) is \[ \bar{y}_{i j \cdot}-\bar{y}_{i\cdot\cdot}+\bar{y}_{\cdot\cdot\cdot}. \] Hence \[\begin{align} s s E_0^A&=\sum_i \sum_j \sum_t\left(y_{i j t}-\bar{y}_{i j .}+\bar{y}_{i . .}-\bar{y}_{\ldots . .}\right)^2 \\ &=\sum_{i=1}^a \sum_{j=1}^b \sum_{t=1}^r\left(y_{i j t}-\bar{y}_{i j .}\right)^2+b r \sum_{i=1}^a\left(\bar{y}_{i . .}-\bar{y}_{\ldots}\right)^2\\ &=ssE+b r\sum_{i=1}^a\left(\bar{y}_{i . .}-\bar{y}_{\ldots}\right)^2 \end{align}\]

and \[ s s A=s s E_0^A-s s E=b r \sum_{i=1}^a\left(\bar{y}_{i . .}-\bar{y}_{\ldots}\right)^2 \] Reject \(H_0^A\) if \[\frac{m s A}{m s E}>F_{a-1, n-a b, \alpha}, \] where \(m s A=s s A /(a-1)\) and \(m s E=s s E /(n-a b)\).

Testing the main effects of Factor \(B\) is similar. The following is called the ANOVA table for the two way complete model:

Source of Variation Degrees of Freedom Sum of Squares Mean Squares Ratio
A \(a-1\) ssA ssA/(a-1) msA/msE
B \(b-1\) ssB msB/(b-1) msB/msE
AB \((a-1)(b-1)\) ssAB msAB/(a-1)(b-1) msAB/msE
Error n-ab ssE ssE/(n-ab)
Total n-1 ssTotal