Okay, in this video, we're going to walk through an analysis that involves a significant interaction between two different factors. And really arm, our main interest in this video is to consider how we could approach analyzing or interpreting data in that kind of situation. And we're gonna get our data from this really nice paper here. It came out in 2020. And the authors of the paper, we're really generous and they were happy to share their data with me. This paper is a really nice combination of understanding physiology and evolutionary biology. What the authors who were interested in understanding is how physiology or metabolism evolves in response to a change in environment. And they are pretty, they were particularly interested in understanding how metabolism evolves. When, when we have a shift from living in a low altitude environment to living in a high altitude environment. And low and high altitude environments present very different sets of challenges. So at high altitude environments, you have lower oxygen levels and you have lower temperatures. And what I'm talking about high altitude, I really mean high altitude. They studied waterfowl and the Andes. And so we expect you to be big differences between the temperature and oxygen levels at high altitude environments like this compared to low altitude environments, birds like waterfowl have an additional challenge when living in these high altitudes because the air is thinner and that can make it more difficult to fly. So there's lots of challenges to living at high altitudes and we would expect metabolism to evolve in response to those different challenges. The authors of this paper were interested in understanding changes in metabolism generally. And so they looked at a number of different metabolic markers. Markers that I, creatine kinase and hexokinase and pyruvate kinase. We're going to focus on data from the pyruvate kinase dataset created by these authors. So what did the authors do? What the authors did is they went out into the field and they sampled by the field, I mean the real-world places where, where these animals live naturally. And they found a number of species that live both where some populations live at low altitude and other populations live at high altitude. And where they expected these populations to have been separated for long enough. There would've been time for high altitude individuals too. How have experienced enough generations at high altitude, you have had an opportunity to evolve a response to that environment. So here's an example. They've got the ruddy duck, where they found populations at low altitude and at high altitude. Similarly for the crest to duck, they were able to sample individuals from low altitude and high altitude. Same thing for the yellow PIN, yellow build PIN tail, low altitude and high altitude. We have pictures of all these beautiful birds here. In some cases, like we have here. They sampled individuals from low altitude and high altitude environments. Buttons these, in cases like this, taxonomists have decided that these populations were sufficiently different that they should be considered different species. And so I just want to draw that to your attention because I'm, since most of these cases involve high and low altitude populations of particular species. In some cases, we have pairs of organisms, pairs of birds that come from low altitude environments, in high altitude environments that are formerly considered different species. But really, the main thing to consider here is that maybe they shouldn't be calling these species pairs, but just consider these lineages. So here's one lineage where we have low altitude and high altitude lineages for the Ready duck. A low altitude lineage here as compared to a high altitude lineage here, et cetera. The point is reason why am I'm as a couple reasons why I'm pointing this out. One is just to get you oriented to the experimental design, but also just to let you know that you're going to see in a little bit that the dataframe that I've created, but these data has a column called species. And I just want to point out that in some cases the, what we're calling species are not actually the same species. So I would consider this pair of the Magellan goose and the Andean goose, either consider them to be one species pair. Where really though, a more correct terminology would be to say at this is one lineage where you have a low altitude branch of that lineage in a high altitude branch of our lineage. So that's the general experimental design. We have pairs of low altitude and high altitude individuals for a number of different species. And the author is, we're interested in looking at the difference in metabolism between low and high altitude populations, but also contrasting those differences among species to get a sense of whether or not all the species have similar kinds of responses to changes in altitude. I think that's pretty much what I wanted to say here. This, I think we can talk about two more things with this figure. And that is we can talk about the assumptions of the tests that we're going to be using. We're going to analyze these data with the two factor general linear model. And the first two assumptions of that approach are that the data were randomly selected and that they are independent. What the researchers did is they went out into the field and they captured the birds. And, and they took tissue samples from those birds. I imagine that they did their absolute best to randomly sample individuals. The paper, as far as I can tell, doesn't comment on that from my own experience and in doing field work with animals that can be very difficult to randomly sample some animals. And it's possible that they have, do not have a truly random sample. But we're going to assume that they've, they've, they've accomplish that goal. Certainly, I expect that they'll have accomplished it as well as anyone in the field possibly could. As well. They have a number of different individuals from within each lineage. The captured multiple birds within each of these lineages. And those different birds are independent of one another. Okay, so the data meet the assumption of independence as well. I'll just quickly point out that if you're an evolutionary biologist, you might shoot up your hand at this point and protest because you might say that, because the species are related to one another to varying degrees, the species are not truly independent of one another. And they might say, uh, you need to take and do take this, related this into account. That can be true. And the authors take this into account using phylogenetically independent contrast. If you're interested in learning about that perspective, I suggest you read about that in the paper. We're going to ignore that aspect of the data. However, because our real goal here is just to use this as a sample data set in order to understand how to analyze to factor general linear models. Okay, let's start looking at our data. In our case, the data are in this file called Dawson at all, 20-20. That's because that's what I've named paper. And we'll run that and we've stored the data in this object called pk because we're interested in pyruvate kinase. So let's just look at this DataFrame. So just look at the general structure of pk. You can see that there are four different columns. One called species, one called SBP, one called altitude, and one called pk. I'll just tell you that these first two columns are redundant. They both indicate the species or the lineage that the data are associated with. And so, and why do I have these two different columns? While the first column just gives you the full name of each species that the data associated with like cinnamon, teal. And the next column just provides abbreviations of these full species names. And I just include the abbreviations because it's easier to fit these labels on two axes of various plots MY weight might want to produce. Then two presents the full species names as axes labels. So that's why I've just duplicated the information that's in the Species column into this SBP column. Okay? We then have our, our, our factor altitude, which is, has two levels, HA for high altitude, an LA for low altitude. So these are two factors. One factory is species or SBP and the other factor is altitude. Our dependent variable is pk, and that's our measure of pyruvate kinase. Okay? I've said that Pk is our dependent variable because our hypothesis is that shifts an altitude will have caused a change in metabolism and thereby, and Pk is a part of that. And so we're expecting that evolutionary shifts to living at a new altitude will involve a change in pk. We also can expect that not all species will have the same levels of pyruvate kinase. And so we can expect that both species and altitude will be determining our level of pk. And so pk is our dependent variable. Let's look at our data some more, just using the summary function. And there we go, basically the same information. But just another view of it. Let's just look at the top, the DataFrame. Head, pk. We go. I usually use this dysfunction hedge just so I can remind myself sometimes of what, what I've called the various columns. Because look the bottom at the dataset by saying tail. And there we go. And finally, let's just look at the data set itself, the whole thing. Okay? You can see we have a lot of data. Viz, researchers have gone out and sampled a lot of different individuals. We have a 134 individuals from seven separate species. Okay. There's something I want to point out which is just how these data are organized. Each row of data contains all the information that's pertinent to a particular data point. So this value of pk of pyruvate kinase is associated with a bird that was sampled at low altitude from the species cinnamon teal. Likewise, this row contains all the information that we need to know about this data point. So for this value of pk of 426, it was sampled from an individual from a high altitude, again from the species cinnamon teal. So we set up these data so that all the information about a particular data point are listed on each row. Okay? Now before we go any further, let's stop and just make sure that we can wrap our heads fully around this data set to make sure that we analyze it correctly. And what I'm thinking of is we need to check to see whether or not our data are balanced or not. We could check that simply by looking at the number of data points that we have here. So for example, in this first species combination or species outdo combination, we have 12345678 data points. But I'm sure if we scroll down. Yeah, like here you can see in this case for the ruddy duck, whoops. And this case you can see we just have 123456 data points. So we can see that the number of data points in our various treatment combinations is not always going be the same. We can get that just by looking at DataFrame that can be a bit laborious or prone to mistakes. So let's use the function summary by just to do this work for us. So somebody by function is in the library. Dubai. Sit, we'll open that up. And then we're just going to say Summary by were the By is capitalized. And if we're going to say pk, and notice pk is capitalised in our dataset. So p k as a function of species and altitude. The data are in the DataFrame PK. And we want to use the function. Let's just calculate the mean for each combination. But really what we're interested in is the length. Okay, let's, let's walk through what this means. What we've done here is we said, we want to calculate. For every combination of the levels of species and altitude. Okay? So for every combination at the level of species and altitude, so we have seven species and to different altitudes. That gives us 14 different combinations. So for all those 14 combinations, we want to calculate the mean value and the length of the data, pk. And other words for each combination of species and altitude, we want to calculate the mean for PK and also the length of pk. What does the length mean? Length just refers to the number of data points you have. And so this function length as a really nice way of just summarizing our data to get a sense of whether or not our sample sizes are similar among all of our different treatment combinations. And so here's our output. You can see that, yes, indeed, we have 14 treatment combinations. That's important because if we did not have all 14 treatment combinations, then we couldn't actually perform a two factor. Generally new model, at least not one. We can see. I lost myself there for a moment. We have 014 fact we have well 14 combinations and that's what we want to analyze our data with a two factor general linear model. You can see that we have replication. So here's our length, pk length. So this gives our number, this gives us our number of data points for each combination of species and altitude. And you can see we have at least two data points for all those combinations. This means we can model that interaction between species and altitude. But we also see that these number of observations is different for our various treatment combination. So this tells us that our data are on balanced. And since our data are unbalanced, we need to take that into account when we're analyzing our data. Okay. I spent a whole lot of preamble before actually getting into looking at the data. That's good. Because we want to make sure we really understand the situation that we're we want to understand the situation before we jump into just analyzing it. At this point, let's start by just creating a boxplot of our data. I'm going to be lazy and just copy this and put it down here. But I'm going to change this plus two an asterix nets because within the box plot command, if we say, if you put an asterix between two different factors, that will cause box plot to produce boxplots for each combination of all the levels of our two different factors here, which is what we want. And we're going to have pk as our variable on the y-axis. So let's run that. See if I can get okay. It seems that my efforts to make the labels short enough or make the label shorter has not been a good enough effort in order to fit everything on here. I could play with the font size, but I don't really think it's worth it. So what can we learn from this plot? And we'll First, let's start just by assessing our, our assumptions. It looks to me like these box plots are generally pretty symmetrical. And so that would suggest that we can generally expect our data to be normally distributed. Which is nice. What about the assumption of equal variance? Well, we can see a fair amount of difference in the width of these box plots. So that might suggest that we could have a different sneak covariance. So that suggests we could have different variances among our various treatment combinations. It also looks to me a little bit like some of these box plots that are especially wide tend to be the treatment combinations that also have the highest means. I don't think that's necessarily universally true, but that's kind of the impression that I'm getting. Some of the more narrow box plots do tends to be towards do you tend to have mean values that are a bit, little bit lower. So it's possible that we have a trend of increasing variance with the mean. What we'll know more about that when we look at our residuals. It's hard to get much more out of this plots because there's so many different box plots. It's hard for us to really get a sense of any trends in the data. So Let's produce a different kind of plot that can allow us to look for trends in the data more easily. And we're going to use the function interaction plot. The interaction plot function requires that we attach our data set. So we'll say not attack attach pk. When we attach our dataframe that allows our to essentially see the column headings within our DataFrame from within any function without having to tell that function where the data, where those columns are found. So in what DataFrame those columns are found. That can cause problems. Because that leaves us open to referring to columns and a function that we didn't mean to. So in general, it's good practice to not attach DataFrames if we can avoid it. And so as soon as we finished with our interaction plot function, we're going to detach our DataFrame. So say interaction plot. Now the first thing we need to specify is. The variable that goes along the x axis, which we'll say is species. And then we need to specify the factor that will be separated within the body of the, of the plot. And we'll say that that is altitude. And then we need to tell it our Y variable, which is Pk. And then we're going to say type equals. Within quotation marks, type equals b. B refers to both, meaning we want to plot both points and lines. We could just say p if we just want points are just L If we want lines, but we want both. And finally, we want a legend. So say legend equals true. Okay? So let's now run this. And here is our interaction plot. You can see along the x axis we have 1234567 species or seven lineages. And for each of them we have means for high altitude, which is given by this dotted line indicated by our legend. And we have, these are also data for the low altitude, which is given by this solid line. I want you to just pause the video for a moment and look at this interaction plot and ask yourself whether or not you would expect us to find significant interaction based on these data. Just pause the video for a moment, think about that. Okay, now that you've had a chance, I think I'll say it's very difficult to know whether or not we expects to get a significant interaction because we don't have any error bars on here. We have no sense of how well we've been able to estimate each of these means. But if we were to assume that these means we're estimated very well. So the arab i has to be very small. Then this plot would suggest that thing we have an interaction between species and altitude. And that's because in some cases, like for the species ST, we can, we have what looks like a big difference between our pyruvate values between high, between low altitude and high altitude. Whereas in other cases, the difference between low altitude and high altitude, tense B seems to be very small. In other words, it looks like the effect of altitude depends on, or the size of the effect of altitude depends on which species were looking at. In some cases, altitude causes a big effect, or we have big differences between the altitudes for the effective on pyruvate kinase. Whereas in other cases, there's very little difference in pyruvate kinase between the altitudes. So that would lead us to expect that we have an interaction in our data. Okay, with all that in mind, I think we're now in a position to actually analyze our data. So let's start by creating our linear model. Will save our output for linear model and say Pk dot lm. That'll be the objects that holds the output. So we'll say lm and Pk. That's our dependent variable. Remember that was one of the columns in our, in our dataframe. And we're gonna say that that is a function of species plus altitude plus their interaction. Now, you'll remember that our data were unbalanced. And so that means we want to make sure we analyze our data in a way that accounts for that unbalanced nature of the data. In a previous video, I gave one suggestion for how to do that using the ANOVA function with a big a from the car library will do that again in this video. There are other ways of doing it, but we're going to just follow the, the suggestion that I made in a previous video. So to do that, we need to specify our contrasts. This with a contrast equals. And we'll say list. And then we have to list our factors. Soaks, species say is contrast some, and altitude is contrast some. Okay. Make sure I don't have any typos here. Okay, that looks okay. So we can run this now. Often we'd want to look at the summary of the output from this to look at our look at our coefficients, were not going to do that. Because if you may remember from the previous, from a previous video, when we set our contrast in this way, that changes the meaning of our coefficients. And so we're not going to look at the coefficients in this example. When I say look at the coefficients, I mean sane summary and then Pk dot lm. Okay? And then to look at that output, we're not gonna do that in this example because the coefficients have a different meaning from what, how we've examined before. All right, so we've done, we've run our model. Now we need to check our assumptions. So we'll say plot Pk dot lm. And see we get, okay, so this first plot allows us to test the assumption of equal variance. And to my eye, we are seeing some small trend of there being an increase in the variance with the mean. So you can see, so this, this bottom. Situation this bottom treatment combination is, is an exception generally. But from here to the right, you can see that the spread in our residuals tends to increase as you move from left to right. That's something that we started to pick up on them or looking at the box plots earlier. And we'd said earlier that we get a better sense of whether our data violate the assumption of equal variance. When looking at this plot, this seems to suggest to me that we do have some reason to, some reason for concern. At least we're going to see if we can improve the situation. The data are pretty good for normality. So that's alright. Let's run this again. But we're going to log transform our data. Okay? And just to remind ourselves that these are log transformed data, we'll change the name of our of the objects that we save our results in. So let's see, we get this time. Okay, that to me looks improved. The trend that we saw earlier was not huge. But this to me looks improved. And so to my eye, it looks like these. The amount of variation we have it within each of these different treatment combinations is more consistent than we had in the previous week when we're just using the raw data. So, so far I think that log transforming the data has improved our has improved how well the data meet the assumptions. Normality is improved. So that's, that's good. So I think at this point we can say that day to meet the assumptions of random sampling of independence, of equal variance and abnormality. So that means we are now in a position to check our p-values. We're going to use the ANOVA function or the big a from the car library to do that. So I'm gonna say ANOVA. And we want to look at you want the p-values for this object. And where we specifying that we want to calculate a p-value is based on type three sums of squares. And here we go. So here are our main results. Okay? Let's start with the interaction. The p-value for interaction is much smaller than 0.05. And so this suggests that we do have strong reason to. We do have relatively strong evidence that the effect of altitude will depend on the species that we're considering. And that's what we said we might expect based on our interaction plot that we produced earlier. So to reiterate, this last line where we're looking at this interaction. This last line indicates to us that the size of the effect of altitude will depend on the species that we're considering. And vice versa. That the particular values of pyruvate for, particularly for a given species, will depend on which altitude we're considering. Ok. That is really the main result from this analysis. Because an interaction means that the effect of altitude will depend on species and vice versa. That means that we really cannot interpret these main effects in the models. Because these main effects and their p values, these main effects can be understood when there is not a significant interactions. So these main effects are these p-values to these main effects refer to whether or not, are they refer to tests of whether or not they're significant differences among species. When we're assuming that the effect of species would be consistent among our various levels of altitude. And likewise, our p-value for our test of altitude tests whether or not we have a difference in the mean pK value between low and high altitude samples. But we're assuming that that difference would be consistent among our very species. We can see that that's not true. We can see the effect of altitude will depend on species and vice versa. And as a result, we cannot interpret these main effects. Okay, so the rest of our analysis then should be devoted to understanding this interaction. That's where our interests now likes. Having said that when we were, if we're going to write up these results, than we'd want to report all three of these p-values and report the degrees of freedom for each of these cases, the degrees of freedom of 616 and a residual degrees of freedom. And reports the f values for species, F i's for int i for altitude, and the f values for the interaction. Ok? So we have now, we now have our, our mission outlined. We now want to be able to understand this interaction better. So we're going to do that using the EM beans function. And we'll, we'll do this twice, will go through the analysis or the we'll use EM means function very quickly in a case where we do not back transform the data. And then we'll do it again where we do back transform the data. So say library. Em means, whoops. Let's say R1. And now let's say. Pk just dot, dot EM for EM means. And we want EM means, we want the R model output is stored here in this object. And I think it's probably the most interesting to look at differences between altitudes, but do it separately for each species. And so we can do that by saying that we want to look at the effect of altitude by listing that factor within quotation marks. And then saying By equals species. And what this will cause EM means to do is it will calculate the pK values that were worked with in my calculate the the mean pK values or the fitted values from our model for our level of four different levels of altitude, but do it separately for each species. So we've got that. Let's just start by looking at those results. Okay. Here are the results. So for each species now, so we have species is equal to d. For each, we have, for each species we have different output for high altitude and low altitude for the EM means and the standard errors and confidence intervals for our pk measurements. And that's reported separately for each species. Which is really nice. Weeds you remember though, that these values are on the log scale. So we could report these values perhaps in an appendix to our, if we were to publish these data, Dawson old already published these data. So this is just a dream that we could publish these nice data again. And we could publish these, these values here. And note that they are log transformed values. Okay, so on the log transform scale. But can also get our contrasts though. For these means, let's say R1 snouts get pK pairs will look at that. And what you can see here is again, we have a different contrast made for each species. So for this species C, D, you can see that we have this estimate, an estimate of minus 0.25. So that's the difference between the mean level of pk between low altitude in high altitude. This species. But remembering we're working with data on the log transform scale. And here we have a standard error for that effect size. And this p-value tells us that we do formerly have a statistically significant difference between the low altitude and high altitude populations of this species. Ok, so if we were to write up these data, we can include these values here. Note the fact that they're on the log transform scale. But we could report these estimates for these effect sizes and their standard errors. And that can be useful for people who might want to include our analysis in a meta-analysis. Okay. We can see that in another species, we do not get any evidence for a significant difference between the between low and high altitude. So for the species CT, our p-value is greater than 0.05, which means that we do not have particular, we don't have any strong evidence to say that the mean difference between low altitude in high altitude is, are different from one another. Certainly you can see that this estimate for this species group is tends to be smaller than we had for the first species group. And it gets smaller yet again for this next species. Okay. We'll finish off this stage is by getting confidence intervals, confidence intervals for the pairs. And again, let's remember these confidence intervals are on the log transformed scale. Okay? So going back to this first species CD, it can see that our confidence intervals range from having a difference on the log scale of between 0.45.06. So in other words, when we use our log transformed values of p k, The difference between those log transformed data between the low altitude and high altitude populations of the species CD. That difference is likely to be between 0.06.45. Ok. We can compare that to try to find another situation. That's not that different actually from this case. Where you have for the ruddy duck, the difference between low altitude and high altitude. But you can see that the signs are different. So for this species, for CD, you can see that the confidence intervals are such that they, they range from about 0.06 to 0.45. We find the same thing for the Ready duck or conference intervals range from 0.04 instead of oh, six to 0.42 instead of 0.45. But they've run in opposite directions. This means that our estimate of the difference between low altitude and high altitude for species CD suggests that low altitude has higher pK values and HA. And that's because our confidence intervals are negative. So forget negative eyes. That implies that LA, or low altitude has higher values and HA, the opposite is true for ready duck. So in this case, the confidence intervals range from around 0 to around 0.45. And so this range of confidence intervals is pretty similar to what we saw up above. But they move in the opposite direction, which suggested in the ruddy duck, it's actually the high altitude population that has the higher values of PK or pyruvate kinase. Ok. So this kind of difference could be the kind of difference that could lead to our scene, our significant interaction between species and, and altitude. Because we're seeing that our effect sizes may, tends to go in different directions among our different species. I think that these kinds of differences be easier to wrap our heads around, at least for me when we look at back transformed data. So, so let's, let's do that. Let's do that. Now. I'm just going to copy this and make some changes to it. The first change you want to make is we want to be looking at these output on the back transform scale. So we're going to say type equals response. And so we'll add that as an R to denote that we're looking at the response scale. And we'll just update the names of all of our objects here. Okay, so let's run that. And let's look at that output. So here you can see that it means is telling us that our integrals are back transformed from the log scale. That's what we wanted. And like before, we can get mean values with standard errors. For each of our combinations of altitude and species. And we also get confidence intervals for this. We do need to remember however, that these means are generalized means, they're not arithmetic means. And so if we were to report these values than we'd want to make sure that we were explicit with fat. With we need to explicitly tell our readers that these are generalized means or geometric mean. Since working with log transformed data and not arithmetic means. What we're really interested in here though is understanding our effect sizes. Okay? So once again, what's a show that we can do our contrasts on the response scale. You can see that our contrast now are in terms of ratios. And that's something I discussed in a previous video. Basically, that's because, or these are, our contrasts are in terms of ratio is because when we're taking the difference between two log transformed data, that's the same thing as saying log of something minus log of something else. You can, that's equivalent to saying the log of this divided by the other term must shift from doing that properly because I haven't I don't know if this is how your seeing the opposite of what I'm seeing. The point is that whatever term is being subtracted from the other, that will be in the denominator of the ratio that would be in a log. And so we're back transform, we get rid of the log of that ratio. We're left with just a ratio. So we could report these these values, these back transformed effect sizes if we wanted to. Again, we'd need to be clear that these are based on generalized means. I think it's easiest to make these comparisons if we look at the confidence intervals for those contrasts, let's do that. Okay? So here we have, here now we're going to look at the contract that confidence intervals for these contrasts. And Let's take this first example. In this case, you can see that the ratio is equal to about it's equal to 0.773. So that's about three quarters, that's about 0.75. Okay, so I'm just going to think in terms of 0.75, so that's a little bit easier. What that implies then is that the pyruvate kinase levels at the high altitude population are about equal to three quarters of those at the low altitude population. For this species CD case, that's our effect size. And The confidence intervals for that ranged from about 0.6, so 0.63 to about 0.94. Okay, let's see if we can find any other situations where that's really different. And that's kinda been, now that I'm looking at this more carefully, that's kind of an intermediate scale difference. Let, let's, let's focus on, Let's, let's not focus on that first example we had SPC CD. Let's look at this example of st. If I remember right, we can go back here and look at our previous interaction plot. There we go. Yes, it was st. That was the species where we saw this really big difference. Okay? You can see that for this species S t, the ratio is around 0.5. Okay, so that means that the pyruvate kinase measurements we had for the high altitude, we're about half of those that we had at the low altitude. Bouts of that ratio means. And look at the confidence intervals, we can see that this effect size, we can have a fair amount of confidence in the effect size being close to 0.05, because our 95% confidence intervals range from about 0.4 to about 0.6. So that's a fairly tight range, which suggests that this ratio of about 0.5 is, has been estimated. We've been able to estimated with a fair degree of precision. Let's compare this range of effect sizes then. It looks like we can compare it with example directly above. Okay? So the effect size we have for this species is, or the effect size of altitude is somewhere between 0.4.6 for the species St. whereas for this species, this species pair SOP t. In that case, the conference intervals ranged from about 0.7 to around one. So that's our confidence interval for the effect of altitude in this species. And you can see that the confidence intervals for that effect size of altitude, these, this confidence interval does not overlap at all. This confidence interval. And that kind of situation where you know, what this means is that our, our estimate of the effect size for this species is very different from the effective, the aspect of is very different from our estimate of the effect size for this species. And so the fact that we get very different estimates of the effect size for one species compared to, I mean, talking myself in circles will go more slowly. The fact that we get very different effect sizes for the effective altitude. For one species compared to another. That illustrates that the effect of altitude really does depend on which species we're looking at. Okay, so it's this kind of difference and the effect of altitude in one species compared to another. This kind of effect can be what's giving rise to the significant interaction and we found between species and altitude. I think we'll stop our video there. We've done a lot. In this video. We've talked about some pretty cool general biology about adaptation to different altitudes. We've looked at how to determine whether or not a data set is is balanced or not. We've looked, we've plotted our data and gotten a good sense of whether or not the data are likely to meet the assumptions. We've also got an interaction plot, and from that we were able to predict whether or not we're likely to get a significant interaction. And we did, we calculated P-values in a way that accounts for the fact that we had unbalanced data. We did that using log transformed data. And then we saw that we did get a significant interaction between species and altitude. And because of that, we concluded that we could only understand our results by looking at the effect sizes for one of the factors. And we chose altitude and comparing that among various species. And we did that both for data that were not back transformed and when they were transformed. We looked at the data that were not back transforms because those might be data that we'd wants to report and to be included in a meta-analysis. So we would report or effect size and the standard error for those data that were not back transformed. But when we looked at back transforms data, I think at least I found a little bit easier to wrap my head around. What's the effect sizes actually meant or thinking about them in terms of ratios. And we're able to see that our estimates of the effect size of altitude were drastically different among or different species. The last thing we should do, I just realized we haven't actually talked about without these effect sizes are likely to buy biologically meaningful. I have to say that I don't know enough about metabolism to know whether or not a change in p k by a factor of two. In other words, by dropping it in half is likely to be biologically meaningful. But at the very least, I think I should say that we need to talk to someone who knows more about physiology. Knows more about physiology than I do to be able to decide whether or not a ratio like this is biologically meaningful? My gut reaction would be probably, yes, that's entirely naive since I know very little about metabolism. But my gut reaction be to say that my naive reaction be to say that this is probably important. So that, that's my attempts to wrap up what we've been talking about for so long. I hope this video has been helpful. And I'll say, thank you very much.