Chapter 16. Analysing experiments with multiple factors

In an earlier chapter, we discussed the design and analysis of 1-Factor experiments.  We described a ‘factor’ as series of groups (‘treatments’) that differ in a systematic way.  We considered an example 1-Factor experiment with three levels:  ‘drug applied’, ‘placebo applied’ and ‘nothing applied’.  It is very often useful to design experiments with more than one factor, however, which we address in this Chapter. 

For example, imagine we wanted to test the effects of a drug (one Factor) in both females and males.  One approach would be to conduct a 1-Factor experiment with 6 levels that comprised all combinations of Drug and Sex; i.e., ‘drug applied to females’, ‘drug applied to males’, ‘placebo applied to females’, ‘placebo applied to males’, ‘Nothing applied to females’, and ‘Nothing applied to males’.  Alternatively, one could conduct one experiment that included two factors:  Drug and Sex.  This chapter focuses on the latter approach, designing and analysing multi-factor experiments.

We focus on experiments with two factors, but it is possible and sometimes desirable to conduct experiments with more factors. For example, I conducted a three-factor experiment during my PhD.  The principles presented in this Chapter for two-factor experiments also apply to experiments with more factors. 

A word of warning, however: adding more factors to an experiment can make results more difficult to interpret – imagine your frustration if you design and complete an experiment with many factors, but cannot make sense of the results!

Multi-factor experiments allow simultaneous tests of multiple hypotheses.  For example, if designed appropriately, an experiment with the two factors, Drug and Sex, would allow simultaneous tests of three hypotheses:

1. Does the first factor (say, arbitrarily, Drug) affect the dependent- (i.e., y-) variable after accounting for effects of the second factor (Sex)?
2. Does the second factor (Sex) affect the dependent- (i.e., y-) variable after accounting for effects of the first factor (Drug)?
3. Does the effect of one factor (say, Drug) depend on the level of the other factor (say, Sex)?

These biological questions differ qualitatively from questions that a 1-Factor experiment would typically address.  Specifically, a 1-factor experiment allows comparison among levels of a single factor, without accounting for additional information.  By contrast, hypotheses that address the ‘main effects’ of a multi-Factor analysis (hypotheses I & ii) allows a researcher compare levels of one main effect (say, Drug) while averaging over the effects of the other main(s) effect (say, Sex).  This allows a researcher to make comparisons among levels of one factor while controlling for effects of another factor(s) in the model.  Further, the third hypothesis, above, introduces another very different and useful perspective / hypothesis, which we discuss more in a moment.

The preceding paragraph highlights that different experimental designs allow researchers to address qualitatively different biological questions.  Hence, understanding experimental design and data analysis, generally, increases the diversity of biological questions a researcher may ask.  (On a personal note, this insight was what turned me on to understanding experimental design and data analysis.  As I learned more about experimental design I became aware of more types of biological questions I could address, which was exciting and empowering.)

The third hypothesis (iii), above, is called an ‘interaction’.  Generally speaking, ‘interactions’ are not more biologically important than the first two hypotheses (i & ii) regarding ‘main effects’.  However we will spend more time thinking about interactions than main effects in this Chapter because interpreting interactions can be subtler.  In the context of the 2-Factor, Drug and Sex experiment, above, evidence for an interaction between Drug and Sex would imply that the effect of Drug treatments differ between females and males. 

Similarly, the reverse would also be true:  evidence for an interaction would imply that the differences between Sexes would vary among the levels of the factor, Drug.  Clearly, evidence for an interaction between Sex and Drug in this experiment would be biologically interesting:  it would shed light on differences between females and males relevant to drug development.  Indeed, evidence indicates that drugs do affect females and males differently, with important consequences for medicine and society.

As an example, this Nature article discusses several areas of basic neuroscience in which careful consideration of SABV has led to critical discoveries of the ways in which fundamental neurobiological processes differ in males and females:

 Studies of ecology and evolution often think in terms of ‘interactions’.  For example, these fields recognize that the effect of a gene (or allele) will often depend on the environment.  This phenomenon can lead to ‘local adaptation’, where members of a population perform better in their ‘home’ environment than in an ‘away’ environment because natural selection favours different alleles in different environments.  Taken to an extreme, this process of local adaptation can help generate new species. 

Alternatively, we may view ‘females’ and ‘males’ as different ‘environments’.  For example, an allele might increase fitness in one environment (e.g., when the allele occurs in females) but decrease fitness when the allele occurs in the other environment (males).   This interaction between Genotype and Sex is thought to underlie the evolution of sexual dimorphism.  Consider an ecological example: the amount of pollen removed from or delivered to a flower will likely depend on the ‘fit’ between pollinator’s morphology and a flower’s shape.  Hence, we expect that the quantity of pollen delivered to and removed from a flower will involve an interaction between floral morphology and pollinator morphology. 

Please note:  Here, and throughout this website, we define ‘females’ and ‘males’ by an individual’s complement of sex chromosomes.  For example, in humans, we define males and females as individuals that do or do not possess a Y chromosome, respectively (the Y chromosome contains the SRY gene which triggers different developmental events).  By contrast, females and males butterflies typically have ZW and ZZ sex chromosomes, respectively. 

We define females and males in this way because, usually, the biological hypotheses that address differences between females and males (define here as ‘Sex’) aim to understand consequences of genetics differences at their ‘sex determining regions’ (often sex chromosomes) for a biological phenomenon of interest.  In other words, this definition of Sex matches the biological hypotheses under investigation.  We note that this definition of females and males (Sex) need not correspond to concepts of ‘Gender’.  If we use the term ‘Gender’ in this website, we have done so in error and will correct the mistake.

These examples help explain why statistical analyses of ecological and evolutionary data commonly consider ‘interactions’ between variables.  On the other hand, my experience suggests that the field of Biomedical Sciences less frequently addresses hypotheses involving interactions.  If true, it would be interesting to understand why this is, because, arguably, understanding interactions would be equally important in Biomedical Sciences (e.g., a mutation might have different effects among mouse strains or between Sexes). 

I hope that this chapter, in some small way, stimulates more ‘interaction-based’ hypotheses being asked in Biomedical Sciences, where appropriate.

Multi-factor models can also allow an analysis to account for features of experimental design.  For example, an experiment that includes ‘blocking’ (See the Chapter, ‘Experimental Design’) might include ‘Block’ as a factor in the analysis to account for unwanted (nuisance) variation.  Alternatively, an experiment might intentionally ‘heterogenize’ environments to determine whether a study’s results generalize across researchers and environments   Again, ‘heterogenization’ would involve an analysis of ‘Blocks’.

Here are a couple of examples of these experiments:

The videos in this Chapter begin by comparing 1- vs. 2-Factor GLMs.  We then discuss assumptions of 2- (i.e., multi-) factor GLMs, and highlight the issue of non-independence in the experimental design.  Note that the concepts of experimental design covered in previous chapters (including principles covered in the Chapter addressing 1-factor GLMs) apply to multi-factor GLMs.  With respect to experimental design, we add that a 2-factor GLM can only include an ‘interaction term’ if the experiment has appropriate replication, as discussed in the video, ‘Introductory Example Analysis’. 

Following this introductory example, we explain how to interpret coefficients for a 2-factor GLM, get practice interpreting interactions, and consider how unbalanced data can affect how we calculate p-values.  We then consider two example datasets that illustrate analysis and interpretation when we do (Pyruvate kinase example) and do not (Sexual conflict example) identify good evidence for an interaction between main effects.