Okay, I've been teaching one factor, general linear models for awhile. And in my experience, one of the things that people often struggle with is deciding whether or not their data meet the assumptions of the test based on residual plots. So this video, I'm making this video because I want this video to be a tool to help you get a sense of how much? Well, to get a sense of what residual plots may look like. Okay? The way we're going to do this is I'm going to show you a number of residual plots that are all four perfect data. And when I say they're all four perfect data, what I mean is these are data that were simulated. And so we created data points using random number generators in a way that we can be certain that data meet the assumptions of normality and equal variance. Okay? So here are three plots that allow us to test the assumptions of normality and equal variance. These plots come from a program called mini tab. And I'm going to show US plus some are shortly. Okay? So on the left here, we have two ways of assessing normality. One is with this Q-Q plot. And you can see what we're looking for is to see whether or not it's the points fall nicely along the line. And you can see that by enlarge, they fall beautifully along the line. They're not absolutely perfect. But this is actually surprisingly good. Okay. And here's another way of looking at the residuals where we're just plotting a histogram of the residuals. And you can see that from this histogram, there's a relatively nice bell-shaped which is consistent with the normal distribution. Okay? So these plots suggest that yes, our data are normally distributed. And this plot on the right allows us to assess the assumption of equal variance. What we're looking for is to see whether or not the amount to spread that's in our two different groups. So these BI data points, one group, these BI data points from another group. So these data points might come from one factor, generally a model where we have two levels, maybe a drug and a placebo group, for example. And what you can see here is that the spread in these data points is relatively similar. Are they perfect? No. We have kind of more data points bunch together. Here. Are the data points tend to be more bytes together than we ever see over here. We've got some data points that are relatively spread out. But these data are, or this plot is perfectly consistent with data that meet the assumption covariance. Here's another example. Again. These data meet the assumptions of normality. Even though the data fall off the line. And if we plot the histogram, you can see it's even a little bit skewed. Okay, but these data are perfectly consistent with a normal distribution. Even though will these plots don't look perfect? Similarly, these data meet the assumption of equal variance perfectly. Even know It can see that these data are look a little bit tighter together then these data. But this amount of variation between these two groups of data is perfectly consistent with data that meet the assumption of equal variance. I'd be saying that a lot in this video. Okay? And here's a third example. In this case, you can see that most of the data actually fall almost exactly on the line. But we have a couple of data points at the end that tend to fall off. And here's our histogram, which doesn't look perfectly bell-shaped, but this is perfectly acceptable. And we know that these data meet the assumptions because we know that these data come from simulations that perfectly match the assumptions of equal variance and normality. And once again, we can see that the variation between these two groups is not identical. So this tends to be more spread out than this. But again, this is perfectly acceptable. What I'm trying to highlight here is two things. I'm trying to highlight that even for perfect data, the residual plots will rarely be perfect. Okay, that's the first thing I'm trying to highlight. The second thing that I want to highlight is just don't want to be giving you a sense. Amount of variation that we might expect. I'm just flipping back through these examples again. So these plots are the differences among these plots. Give you a sense, or I want them to give you a sense of the amount of variation that we might expect to find in plots like these. Okay? Now these pre-work, these plots were produced in mini tab. We want to do this in R. What I'm going to show you is how we can simulate our own data and use those perfect data to get a sense of what good residuals look like. And we're going to be using code that looks exactly like this. I'm going to call up this code in and RStudio file in a moment. The one difference is that you're going to see that there's going to be using the lm function as opposed to AOV. So AOV is a function for analysis of variance. We're getting me using the lm function, which does general linear models. But in many ways these functions are completely equivalent. In fact, the AOV function uses code from the lm function in order to run what it does. Okay, so let's see how we can create our own data so that we can build our sense of of what good data. Okay? You'll see that I've got a lot of comments here. Okay. I hope they don't aren't distracting. I've put them there because I'm going to make this file available to everyone. And I thought it'd be nice to have this commentary around the commands so that the commentary itself can explain what's going on. I thought that just might make this file a bit easier. I'm not going to be reading the commentary, I'm just going to focus on the actual commands. So we're going, pardon me. We're going to go through the process of creating a DataFrame and then analyzing the data in that DataFrame and looking at the residuals. Okay? So the first thing we're going to do is we want to create a column in our DataFrame that contains the labels for different treatments in our experiment. So we're gonna imagine we have an experiment with three treatments, going to call them, well, cheap into a, treatment B and treatment C. And I'll just show you how this code works. If a highlight just that code. What I'm gonna do here is I'm going to repeat. Rep stands for repeat. And I'm going to repeat the letter a 20 times. Okay, that's we get there. So that if I run all of this code like that, oops, what happened here? Oh, I see. It doesn't like commas, okay. Because I have this see around here. So what I'm going to do, what I'm doing here in this code is I'm repeating letter a 20 times and then repeating, let a be 20 times than repeating letters C 20 times. And then I'm sticking them all together in their own vector. And that's what the C command is doing. Okay? So that's what that looks like. And if I just run this whole thing, we're going to be saving all of that treatment information in this vector which is called treatment. Okay? So this is going to, this fact that we created, It's going to be a part of our dataframe that specifies which treatment datapoint came from. Now we need to do is we need to actually generate data that will go with these labels. We're going to use the rnorm function. Okay? So our norm allows us to draw a random numbers that's the r is from a normal distribution and it takes three arguments. The first or we have 20 here. This specifies the number of data points that we want to draw. Should be no surprise. We're going to try 20 data points because we have 2820 bees. And when he sees. The next argument here where we have 10 at this represents the mean value of the distribution that we're drawing these numbers from. Okay, so our normal distribution that we're drawing our numbers from has a mean of 10. And this last part here specifies the standard deviation of the normal distribution that we're drawing in the numbers from. So this refers to the amount of spread that we have in the normal distribution. So we describe the normal distribution in terms of its mean and a standard deviation. And so that's what these two points allow us to do. They allow us to characterize a normal distribution that we're drawing numbers from. And this first number here says how many random numbers we're going to draw. Okay, so if I just run that highlighted code, you can see we're getting 20 random numbers. If you want. We can plot these as a histogram just to get a sense of what these look like. Okay, there we go. If we wanted to plot more random numbers, just to get a better sense that we can see that with lots of random numbers. Does look like a normal distribution. Now, let's generate our random data for our analysis. Since here three different treatments, we're going to draw three different sets of random numbers. I'm highlighting the code for the data that will corresponds to treatment a. So we're drawing 20 random numbers from a normal distribution because we're using our norm. And this the data for treatment a or gotta have a mean of 10, standard deviation of four. Okay, next comes a data for treatment B, where again, we're drawing of 20 random numbers. The mean is different. So the mean is this time is 15, but the standard deviation is the same as we had for the first treatment. Okay? And finally, we have the data for treatment C. Again, we're dry and 20 random numbers. The mean is higher again, so the mean is 20 and the standard deviation is identical to the other two treatments. So what we're doing here is we're creating a dataset where there are the true, where the truth is there are differences between the mean values, achievement a, B, and C. And we know that because we specified different mean values, a mean of 10, mean of 15 and 20. But these data perfectly meet the assumptions of normality. We know that because we're drawing a random numbers from a normal distribution. And we know that they meet the assumption of equal variance. And that's because we set the standard deviation to be the same in all three of our treatment groups. And recall that the standard deviation is just the square root of the variance. Okay? So if the standard deviation is the same for all these, the variance must be as well. So if we run this code and we look at data, Here's our data set, our random dataset that we've created. So I should say we've say save all these data in a vector which we're calling data. And now we're just going to take these two vectors, data and treatment, and put them together to create a DataFrame. And we're going to call that DataFrame fake data. Okay? So if we say Run, now if we say fake data, here's our DataFrame with 20 data points in each of our two treatments. Okay? So that's how we're going to generate our random data. I've just taken all that code we had above where it was all spread out and just put them all together here. This just repeats exact same code as we had above, but it's just on one spot now. And we're going to take these data be created. We're now going to analyze them. We're going to analyzing these in general linear model. And so we do that by reading the lm function and to analyze these data with the general linear model. The first thing we do is we specify column within our DataFrame that contains the dependent variable. So that's the y variable. And in this example, that's the data in the column called data. Okay? And let me give a little Tilda, which you can think of as being like an equal sign. And then on the right of the Tilda, we specify the name of the column in our DataFrame that contains the treatment information, okay, or the, the names of the levels of our factor, which happens to be called treatment. Okay? And then after that we're just going to tell R where to find these columns. And these columns are in the DataFrame that we've called fake data. And so we're just saying, okay, Are the data are found here in fake data. And we're going to save this output in this object called check fake. Good. So we're checking our fake data. And we're saying that's good because we're remembering that these, these data all meets the assumptions of normality and equal variance perfectly. Now, if you want to check the assumptions, we look at the residuals and we can do that by just saying plot. Let's just use the plot function. And what we plot is just the contents of this object where we saved the output from our general linear model. So a running our model, savor the output in this object. And then if we just take that object and give it to the plot command, this is going to give us four plots that allow us to check our assumptions. And I'll walk you through those four plots in a moment. Okay. So that's how we generate our data. Now let's just run this. Okay, so just highlight and all this code. Let me go. Now let's look at our first plot. Okay? So it's told you there's four plots that you're going to be given. And two of them can be used to check the assumption of equal variance. And one of them can be used to check the assumption of normality. And the third one, we're not going to deal with. And I'll explain why when we get to it. Okay. So this first plot here contains or shows the fitted values against the residuals. Okay? And the fitted values refer to the inferred mean values for each of our groups are for each of our levels of our factor. And let's just notice the values. Remember we had a mean of 10, a mean of 15, and a mean of 20. And you notice here that we have a mean of around 10, a mean of around 15, and a mean of around 20. Okay. They don't fall exactly on 10, 15, and 20 because these data were drawn at random from normal distribution. And so we expect there to be sampling error that influences that's going to cause the mean of our sample data to not equal exactly the true mean. But what, I'll start with a point here. The point here is I want to highlight that each of these columns represent the residuals from each of our three treatments. And these will be the data from treatment a, treatment B, and treatment C. And in this plot, what we're looking for is to see whether or not the spread of the data is relatively similar among each of our groups. And you can see it's not exactly the same, but it's very nice. Okay, so, and that's because we know our data perfectly meet the assumptions. There's also this red line. Ideally, we want this red line to be flat as it is here. Okay? So that's another thing to be looking out for. So this is the first plot that we can use to check the assumption of equal variance. This plot allows us to check the assumption of normality. This is our QQ plot. And you can see that by and large the data fall along the dotted line that we have here. But it's certainly not perfect. Okay, but again, remember, even with this imperfect plot, these data match the assumptions perfectly. This third plot is another way of checking the assumption of equal variance. Okay? And we read it in a similar way to the first plot that we get, that we had. The difference between this plot and the first plot is that the residuals have been standardized in a way that can make it easier to assess whether or not the data meet the assumptions. So given the choice between deciding whether or not your data meet the assumption of equal variance given the choice of this plot. And the first plot. This plot is generally the one that's recommended to use. We read it in the same way as the first plot. So we have this red line would generally wants it to be flat. And we're looking for the data points to be the data points within each of our groups to be roughly centered on the red line. Okay? And you can see you that's true here. So roughly there's was much data beneath the red line as it was above. And that's true for, in general, for all of our treatments. And you can see the amount of spread within each of these columns is also relatively equal. Is it perfectly equal? No, absolutely not. But again, these data are perfectly consistent with data that meet the assumptions of equal variance. Okay? So on this, on this plot here I highlighted the fact that we want the data to be roughly centered around the red line. I forgot to mention that on the first plot. So that's true for both of these plots. Okay, this plot and the first one. Now, this last plot here we're not going to talk about. This plot is one that can be used to look for outliers. And we're not going to use this plot because normally, what we would do is we would plot our data first before actually conducting the analysis. And it's that initial plot that we would use to see whether or not we have any obvious outliers. There are certain types of analyses where this fourth plot can become exceedingly useful. But we'll talk about that when we get to those types of analyses. Okay? So I generally don't use this fourth plot when I'm analyzing general linear models. So it, So when I'm using a one-factor general linear model, pardon me? Because I use my initial. Plot of the data as the time or I be looking for outliers. Okay, so that was a nice tour of how to interpret these various plots we get from our, well, we want to see now is you want to see how much variation we get in these results. Okay, so let's just run this code again. Okay? Can see in this case that again, the data are not perfectly identical with respect equal variance. In fact, we have a couple of points that stick out on the ends here. We have 1 that sticks out there. But by enlarge, we know that this situation is perfectly consistent with equal variance. This looks really nice for normality. But again, not perfect. And here again, you can see that this plot is consistent with equal variance. You notice that the, the red line is not perfectly straight. That's totally fine. And you can see that the data are not perfectly identical in terms of their spread for our three different treatments. Okay, but again, this, this amount of variability that we see here compared to there compared to theirs perfectly expected. Let's run this couple more times just to continue building our intuition. So again, variation here. So we've got some tight data points. There was some points spread out compared to this where it's all relatively nice and even. But that's okay. This is the amount of variation that we can expect with random data. Same thing here. Normally distributed data doesn't look perfect, but the data are in fact perfect. And here's another example, okay, where again, the red line is not perfectly straight and we don't have points that are perfectly spread out identically between our three groups. But that's okay. Let's do this one last time. So once again, not absolutely perfect, but this amount of variation between these groups is completely expected that is close to perfectly normally distributed. And here you can see the red line goes up and down a bit. And you can see that these data maybe looked slightly tighter together than these or that. But this amount of variation again, is totally within the range of what we expect with perfect data. Okay? That's what I wanted to show you. So the main point for this video is that residual plots were rarely be perfect, even, even for perfect data. I've given you this code partly as a 20, sorry, partly as a tool for you to use. So for example, let's imagine that you are analyzing a dataset that had been generated, some results that you weren't really certain about. And you really weren't certain whether or not your data met the assumptions that one of the things you could do is you could take your description of the data. So for example, you could take, you could extract the variance from your model and use that variance as a component of your, I should show you how to do that. Actually. If you want to know the overall standard deviation from your model, you can use this function here. Check, check. Good. If you run that. Whoops, sorry, it's not what I wanted to do. Let's run that again. That's tripped up because I hadn't pressed return for this last command there. Okay? So here you can see that sigma, this is the estimate of the overall standard deviation of our residuals. And you can see it's pretty close to the value that we specified. We'd specified a standard deviation of four, that was the true standard deviation of our data. But you can extract the estimated standard deviation from our, from our data like this. Okay? Why am I telling you this? Well, let's imagine that you had run an experiment and you weren't really sure whether or not your data met the assumptions of normality equal variance. You could go through this process of simulating data like this. But where you could just insert the amount of or the estimated standard deviation from that you've extracted from an analysis of your data. So let's imagine you analyze your data with a model like this and you extracted your standard deviation. You can now plug that standard deviation into code like this. And then you could go through a series of plots and see if your data did actually meet the assumptions. How much variation, what range of possible scenarios would you expect to find any residual plots? And if you do that, say 1020 times, get a good sense of that variation, then that can give you a nice tool to be able to go back and look at your actual data and to see whether or not those residual plots fall within the expected range of what residual plots would look like with perfect data? I don't feel like I've said that in a very articulate manner. The point is, you can go through this exercise yourself for your own data to get a sense of the range of residual plots you would expect to find for perfectly, for perfect data, for perfect ETA in terms of meeting the assumptions. And see how that range compares to the residual plots you actually obtained from your data. And if your actual residual plots look very different from the range you get from these simulations, then you probably have some reason to worry. Okay? And that's what I wanted to show you. Hope this video has been helpful and I'll say, thank you very much.