Okay, in this video, I'm going to walk you through an analysis, an analysis of some data that uses a mixed effects model. The data that we're going to use come from a practical or a lab from a third year neuroscience course at the University of Edinburgh. Because the data come from a practical or come from a lab. There is I have mixed feelings about using this dataset as, as an example of mixed effects models. On the one hand, I think it's excellent to be able to create this video and to share some thoughts about this dataset. Because students at the university can make use of it for this course. The drawback, however, of course, is because this is for a practical or a lab of the eye, that the design is, has been shortcut so that students have enough time to complete the whole experiment. And as a result, there are some, some design flaws with the experiments that I'll point out. And so I'll put a point these out along the way and explain why this isn't the ideal datasets to be explaining this. My, the approach that I want to take however, is to basically lay out what our assumptions are and how we should view the analysis. And then after we've done that, everything that follows should follow nicely. And we can turn this into a nice example. So let's start by getting our data into R. So it's in this CSV file called tadpole three. As you might guess, the analysis involves analyzing data from tadpoles. Let me start by just explaining the data to you. So let's start just by getting a summary of these data. So here's a summary of the data. And I we go. The experiment involves a series of tadpoles. And each tadpole has been measured a number of times in a number of contexts. Specifically, each tadpole has experienced three different treatments. They've, they start out with a control treatment where the animals are intact. And what they do is, or what the students have done is they measured the swimming ability of these tadpoles multiple times within this first treatment. So we have multiple measurements from a particular tadpole within each treatment, sorry. Within this treatment. After that, the students administer a lesion to the tadpoles. And then they repeat this process. They then for each lesions, tadpole vey measure some aspects of swimming, the same aspects of swimming as measured in the first treatment. But again, they get several measurements of each tadpole in this second circumstance. And then after this second treatment has been applied, they then apply a third treatment where they expose the tadpoles to NMDA. And they then repeat this process of causing a tadpole to swim multiple times. So each tadpole swims multiple times in this third treatment as well. And they measure the swimming for each of those swim medications. So there's a couple of things to point out with this experimental design. First is we have multiple measurements on each tadpole within each of these treatments. As a result, we have data that are non-independent. And as a result, we cannot analyze these data with something like a one factor general linear model. And it might seem at first glance like we might be able to do that because you really only have one factor of interest. And that is this factor which is called experiments, which has three levels, either intact, spy analyzed or the NMDA treatment. And it'll be really nice just to perform a one-way general linear model to compare among the means of these three different treatments. But we can't do that because the data within each of these treatments is, is non-independent. We're going to solve that issue of not independence by also modeling the effective tadpole as a random effect. And by doing that, we can model the among tadpole variation. And doing that allows us to analyze the data in a way where the non-independence will be accounted for. In other words, we'll be avoiding a situation where we have pseudo replication. So that's, that's really the main thing that I do want to highlight in this experiment. However, I'm going to point out right now that there are a few issues of this experiment. One of the biggest issues is that these treatments are always applied in the same order. In some cases that can't really be helped because you can't have an intact animal after you've already spinal iced it after you've already administered a lesion. So there are some practical difficulties that that might be difficult to deal with. But the fact that we have all of these treatments administered in the same order introduces a problem here because we now have treatment being confounded with time. In other words, if we always measure intact first and then we always measure spinal I second than always measure NMDA third, then we might find differences among these treatments. But it'll be difficult to sort out the extent to which differences among these treatments occur because of the treatments themselves versus, versus problems with time. Because it could just be that the tadpoles swim perhaps less in the NMDA treatment because it experienced this treatment last and the tadpoles might be say, very tired. There can be other issues with this as well that may have to do with non-independence. To be honest, I have not fully wrap my head around the, the full consequences of this experimental design. So one thing I want to point out at this point is to take the overall conclusions from this experiment. With a grain of salt, maybe a handful of salt. And that's because there are some confounding issues with, with this experiments that we're not going to deal with in this analysis. Instead, I want you to, to solve this problem and to make this a good learning exercise. What I'd like you to do is the following. I would like you to imagine that this experiment was performed differently, where we have this column tadpole. I want you to imagine that instead that this is called family. And so what we have is instead of having the same tadpole experience, experience the same treat. Sorry, instead of having the same tadpole experienced three different treatments. Instead, I want you to imagine that for each level of this thing that we're calling tadpole, we're actually saying that this, this, this factor tadpole, we're gonna now imagine it's family. And we're going to imagine is within this family, we would randomly select a few tadpoles from this family to the intact treatment. A few other tadpoles from a given family. So like this family TE ten. And we'd allocate them to the animal spinal lysed. And then we take animals from the family t ten and allocate them to the NMDA treatment. In other words, for this particular family, which we'll call T1 in our factor, tadpole, which were imagined as being a family. We're taking some tadpoles from this family and allocating to intact. Some allocating to spinal lysed and some allocating to NMDA. In this experimental design, we still have non-independent data because we have data that are highly related to one another because they all belong to the same family. And so we would still wants to model this factor as a random effect. However, if we imagine our data in this way where we're having different animals within families allocated to each of these treatments. But still a sing, each family experiences all of these three treatments. And that case we do not have the the same carry over issues that we had or the same time issues necessarily. For one thing, we could, if we had enough people carry out all these experiments at exactly the same time, we also want to have problems with the tadpoles feeling more tired between the beginning and end of treatment, et cetera. Okay, so that's how I want you to imagine this experiment as, as we go forward. So that's all of my thats all by warnings before we get started. As always, we're going to start by just plotting the data. So we'll start with a boxplot. And we're going to say our dependent variable is swimming. And so we're going to put it on the left of our tilda. And we have two factors. We have tadpole and experiments. We're going to ignore this, this other factor which is, which is included, which is just subgroups. This is another aspect just of the, the groups within the practical lab. And we're not going to deal with this. It's not worth going into. This is something that I asked the person organizing the practical to collect data on. And because I was interested in whether or not there's better ways to design the practical. And this told us, know that there's the practical design just fine. So that's why we're going to ignore this treatment. We're just focusing on tadpole swimming and experiments. So swimming is our dependent variable. We have two independent variables. We have tadpole and experiments. When we were plotting data for a two factor general linear model, when we were putting a box plot, we would normally say something like tadpole star experiments. Because then we'd be able to plot box plots for all combinations of R. Two factors are tadpole and experiments. In this case, it's really not practical to do that because there are so many levels of tadpoles. So instead, we are just going to plot our data as a function of experiments. And how we'll say data equals tad three. So let's start by running that. Well, that looks pretty ugly. And we've got lots of variation here. Very little variation in the intact treatment, et cetera. Well, let's, let's see what happens if we, let's build on this. We can also add on the individual values using strip chart. I'm going to skip that stage, however, just because for the interests of time, what I'd like to do is just jump to see and whether or not log transforming these data help us at all. At the moment. Should be pretty obvious from this plot that the variance is going to be very different between the NMDA treatment and the in tax treatment. Okay, that should be dead obvious. You can also see that these distributions are highly asymmetrical. So the bottom part of this distribution, so underneath the median, is very scrunched together, whereas the top part above the median is very stretched out. So we have data that's do not look particularly normally distributed. What have we log transform the data. That's looking much better. That's looking much better. The variance doesn't look to be exactly equal, but thats, that looks pretty decent and the data are now, or the box plots are now nicely symmetrical. And so we're, we're, we'll probably be satisfied with with the data meeting the assumptions. We can note here that it looks like the values for swimming are lowest for intact. They look to be somewhat higher for the legion to individuals. And then it seems like adding NMDA increases, increases swimming again. So that's what we expect to find in our, in our results in general. Okay, so let's start now. Now that we've looked at our data, let's start by analyzing the data. Okay? We're going to use a function called LME r, which is in the library LM E four. So we're going to bring that up. If you don't have LME for, then you should install it with install packages. Now, we're going to say, we're going to create a linear model called LME JAR to remind us that we're not using the LM function. Instead we're going to use the LM E r function, which is a very common function that's used for mixed effects models. And let's start by running a model where we have not log transform the data. Service a swimming as a function of experiments that we're treating experiments as a fixed effect. Because we are specifically interested in understanding the value of swimming at each of these three predetermined levels. These levels were not determined at random. We specified them in advance because we're interested in them. We can model the effective tadpole in this way. So first of all, we use brackets. Ok, so this is, I'm going to show you LME R's syntax to model a factor as a random effect. So we're gonna put a one and then a pipe, which is this vertical line. And then we're going to have tadpole. And I just can't remember if tadpole is capitalized or not. Yes, it is. And then we say data equals Tad three. Okay? So this is how we're going to specify our random effect. What does this one in front of the pipe actually mean? I'm not going to get into that in this video. There are a number of ways of specifying this random effect, which gets into much deeper issues with random effects, with sorry, with fixed effects, with mixed effects models. And so we're going to talk about why this notation is the way it is in future videos. For the moment, I just want you to appreciate that this is perhaps the most common way of specifying random effects in LME our, Does that mean it's always the best way? No. But usually when if you were to look up in a book that teaches you how to use LME are you'll very commonly see notation like this. And so I'm showing it to you. I'll just say that this type of notation or having the one in front of the pipe, involves certain assumptions about how remodelling this random effect. And again, we'll deal with those assumptions in future videos. So let's start by running this model. Okay? So what we'd like to do is we want to As always start by just checking our assumptions. And we can as always use the plot command to visualize the residuals. One thing that's different, which you'll see is we're only going to get one plot. From this. We're not going to be getting four plots which we normally got from the output of the lm function. This first plot will allow us to assess the assumption of equal variance. And we interpret it in the same way that we did in our previous analyses using the function lm. So we'll just run this. And I think you will agree with me that that is dead ugly. What we want to see is a nice even cloud of data points in, throughout this region. That's not what we have. So that's, that's, that's pretty ugly. Let's see what happens if we log transform our data. Okay? So where log transforming our data's just putting log, putting log function around our dependent variable, right in our function will run this again. And then we'll run this plot again. That much, looks much, much better. So what you can see here is that there's really no systematic trend in these data points as you move from left to right, we have a fairly consistent spread in the residuals. So I'm thinking vertically. So the vertical spread and the residuals is pretty consistent as he walk from left to right here. And so this indicates to us that once we've logged transformed our data, our residuals meet the assumptions of our test nicely. I should say at this point that the assumptions of these mixed effects models are generally very similar to the assumptions of just standard general linear models without a random effect. There are some slight differences, but we check the assumptions by enlarge in a very similar way. I'll talk about those differences in the assumptions in a future video. So here, checked her assumption of equal variance. We can also check our assumption of normality. A quick way of doing that, sorry, is two. Just take our object where we store the output from the LME, our function. So Tad dot LME Are, we have ty dot LME arm. And then we can pull out from it, specifically the residuals. And then we can plot them in histogram. Oh, of course, I'm being an idiot. We don't do that. We do that. Okay. So what we're doing is we're pulling the residuals from our object and then we're plotting those residuals in a histogram. We all make mistakes. And there we go. And you can see that these residuals are nicely symmetrical. Were not quite so concerned with whether or not they're perfectly normally distributed. We're looking to see whether or not they have this general kind of symmetry in general bell-shaped. And those look beautiful. So we can now say that we are satisfied that our data meet the assumptions of our test. We can also say that the data were randomly selected. We're going to assume I wasn't involved in collecting the tadpoles, but we'll assume that had pools were selected at random from a larger population. And we've accounted for the non-independence in our dataset by modelling the tadpole as a random effect. So now we want to look at our results. I want to show you something. So we're just going to say summary. We're going to look at summary of Tad's dot LME are. And I want to emphasize a few things to you. Okay? The first is that a lot of this results, a lot of this output is very similar to what we get from the lm function. So at the top, you'll see that we get a summary of what the model was. So we have the model given here. We're told that the model was fitted using our RAM or Remo, which stands for restricted estimation, maximum likelihood. That's something again, we're going to talk about in a future video. We then have some statements about the residuals. And all of this stuff that comes above is exactly the kinds of things that we saw in the output from the lm function. We're going to skip this for a moment. And we're going to pop down to the bottom. Ok. What we see in the bottom mostly is just again, output that looks very similar to what we saw. And the output for the lm function. We have coefficients which are given by the estimate. We have our intercept which has been allocated to the it looks like the lesion group, I'm guessing. Since we have intact and NMDA treatment there. And so we have the our estimate for our first treatment. And then these other estimates tell us how the, the means of our intact treatment differs from our intercept and how the NMDA treatment differs from the intercept. That's what's given here by these three. Okay? There's something I want you to notice here. And that is that we are not given any p-values. And that is because the authors of this package of LME for recognise that there is some debate on how to calculate P-values for mixed effects models. And they have very strong feelings on this, which is excellent. Although it might seem a bit disheartening to not get these p-values at first. Rest assured we are going to be getting p-values. What I really just want to highlight to you by showing you that these p-values are not given to you by default, is that the authors have left them out because there are a number of different ways of calculating p values. And they involve a number of different kinds of assumptions for how book, for how to calculate P-values. And so when the authors leave the p-values out there, leaving it to us to decide how to calculate the P-values. And again, this is something I'm going to deal with more in a number of other, in later videos. In this video, in just a moment, I'm going to show you 11 way to get some p-values. Okay? So that's the first thing I want to emphasize. The other thing is I want to emphasize that what we have up here is new. Okay? What we have is we have estimates of variance. We have an estimate of variance for tadpole. And we also have, so here we have variance and we have estimates of variance both for tadpole and for the residual. So what this is, is this is the estimate of the variance that occurs among our various tadpoles. In other words, this is the amount of variance in our data set that can be explained by variation among the tadpoles. This value here, this is our measure of the variance for our residuals. So this is our residual variance. The output here also expresses the, this term also has a standard deviation. So this is simply equal to square root that. So I can just prove that to you by saying if we take this value here, multiply it by itself, then you get 5.724 or 4.75. Ok. So those, that's what I want you to see from the summary. You can see that the raw, the raw output from LME r does not GDP values. But we can interpret the amount of variation that is explained by our random effect. That's really nice. So I told you we wanted to get some p-values. To do that. We have to start up another library. We're going to say we want lie library, LME, our test where test is capitalised. So we will start this. Okay? And now what we're going to do is we're going to run this function again because earlier it was only run with LME, are now with LME our test loaded. We can get additional output from this function. Okay? So we're just running our model again. Now we can say summary of Tad's dot LME are, oh, come on. Now you can see that we get p values associated with our, associated with our estimates. Okay? So now we're getting the full kind of output that we're used to getting with the lm function. I'll just highlight however, that LME, our test, uses one particular method. Or it allows you, it actually allows you to use more than one method. We haven't played with the defaults. But by using LME, our test, we're using a method among many in order to calculate these p values, okay, and we're going to revisit those many different ways of getting p-values in future videos. Ok? If we wants to, what we want to do now is we wants to get an overall p-value from for our test. And we can do that using the ANOVA function. I want to highlight that this We're using this even though in previous videos I talked about unbalanced data. And I said When you had unbalanced data, you should use anova with a big a. In this case. Once we have loaded a LME, our test, whoops, didn't mean to do that. Once you've loaded LME our test, it has its own ANOVA function. And so this function does not work exactly the same as a standard ANOVA function. And you'll see that in just a moment. So you can use this Annova function once you loaded LME our test, even if you have unbalanced data. And it will calculate P-values using type three sums of squares, which you will see in just a moment. Okay, so I realize that this notation is really unfortunate. It's so unfortunate that it's so similar to a two notation we've seen previously. You just have to remember that if you are analyzing a mixed effects model using LME are LME art or using LME our test. Then you can use this function because it's actually, as far as I understand it, it comes, it's a different form of ANOVA from this package. So let's now look at this. What I want you to see here is that I'm just going to point out a couple of things. So when ANOVA is calculating this P-value here, it's using something called the sadder or satyr widths method, which is one of the methods of many that can be used to calculate p-values. The other thing that I want you to point, that I want to point out is that it is using type three, type three sums of squares approach. Okay? So let's look at our p-value here. Our p-value is minuscule. It's much smaller than 0.05. So we have good evidence that our swimming speed differs among our three different or three different treatments. So if you were to be writing this up, you could report this p-value. You can report this f value. You can report the denominators degrees of freedom and the numerator degrees of freedom. And you would also specify that these degrees of freedom were calculated using the satyr waves method, which explains, or at least partly explains why we can have denominator degrees of freedom. That's a fraction. This is one of the, maybe the first time you will have seen degrees of freedom that's expressed in terms of a fraction essence as opposed to an integer. Okay? So these are all bits that you could report in and a write-up or in. In a paper, if you were to be writing papers, reading a red NAEP analysis like this for publication. What we want now is we want to understand not just whether or not there are differences, but we want to know how big those differences are among our various treatments. So how do we do that? Well, we can do that in exactly the same way as we've done in our previous analyses using the LM function. So we're going to use a library EM means. Because this library is able to work with mixed effects models or output for mixed effects models. So we're going to open the EM means library. And we're going to say Tag dot EM means. So we're going to say EM means. And where we want to give it the name of our, the name of our object. English is escaping me. So ted dot LME are and we want to understand the differences among the levels of experiment. And we log transformed our data. So we're going to say type equals response. Whoops, scratch that. I forgot for a moment that when we have a mixed effect model, there are a diff, there are, there are additional levels of variants to account for that are not present in a simple one factor general linear model. And as a result, we cannot simply use type equals response. I'd forgotten that when I was writing this, but EM means was very kind and gave us this, this error message. So we're going to move forward without actually trying to back transform our data. Okay. Oh, I see the diamond idiot. It's because it's called Experiments. Let's see if I was just talking total shite earlier. So type equals response. There we go. Okay, take back what I said before about EM means not being able to account for the additional variance. Okay, so, so it is able to do that. And let's just look at our results. So first of all, it tells us that our integrals and back transformed into log scale. These will be generalized means, remember, okay. Because these will not be arithmetic means. But here ours, here's the mean values for our swimming variable and standard errors for each of them, for each of our different experiments, where we have lower confidence intervals and upper confidence intervals. Now we'd like to make pairwise comparisons among these. To do that we can just say tag pairs, say pairs of tad, Tad EM means. And now we'll just look at that output. Let's pick this bit bigger. Let's just run this again. So we can, there we go. Now I've got everything in the same place. So because we're backed transforming, log transformed data are contrasts are being expressed as ratios. And here are our ratio. So this tells us that the based on generalized means, the are swimming value for animal spinal lysed individuals was about 2.5 times the size of that for the intact. Whereas animal spinal lysed was only about 13% that of the NMDA treatment. Okay. And we have p-values and standard errors to report for each of these various comparisons. I'll point out that in this experiment, the way it was designed, the hope was to only compare the NMDA treatment versus intact and spinal lysed versus intact. Because intact works as as a control, it is possible to do that in EM means as opposed to conducting all three of these comparisons. But that again, is a subject for another video, okay? So that kind of thing can be done, but we're not going to deal with it. Here. We're going to keep things as straightforward as possible. The last thing we want to do is we want to calculate confidence intervals for our contrasts. And once again, this is just revisiting things we've already learned to do previously. Where here, now we can have confidence intervals for these ratios, for these effect sizes when we're comparing one treatment to another. And ideally when we were write up results like this perhaps to get published, we would include the conference intervals. With these effect sizes, we would definitely include the standard errors for these effect sizes. Meta-analyses might not just want these results presented in the back transformed state, but they might also want these kinds of comparisons in the transformed state. So you might also want to report the same kinds of values, but where we didn't not back transform the data. Okay? That really wraps up our discussion. So we've touched on a number of quite a lot of topics here. I think. The first important issue is just this experimental design. That the facts that we had this particular experimental design where you have one treatment following another, following another. That opened up a Pandora's box that is beyond the scope of this video. However, is easy to encounter situations like this, like, like we have ourselves. And so I want you to be aware of how experimental designs like this can be much more complicated than we might expect at first glance. So that's our first lesson. The second lesson is that we can use random effects in order to account for non-independence in our data. And when we do that, we can use the function LME R, And we can code a random effect in this way. As I mentioned earlier, there's more than one way to, to call these random effects. So for example, we might say experiments in front of the pipe instead of one, whoops. And this would have a very different meaning. And in future videos we're going to talk about the differences between these ways of specifying our random effects. We also saw that when we use LME are the default does not give us p-values in our summary. And so if we want to be getting p-values and we have to think specifically about how to get them. And there are a number of different ways to calculate p-values, which you will deal with in future videos. In this video, I've highlighted one very common way, which is to use the methods that are implemented in the LME, our test to package. And after that, we've also saw how you could, from the output of our analysis. We can actually quantify the amount of variation or the output gives us a number that quantifies the amount of variation that's explained by our random effect. And then the last point is it, very much of this analysis is exactly like we've done previously. Which I think should be a very nice thing, Fred, Once you appreciate, we'll stop this analysis here. And I'll say, I hope that this has been useful. And I'll say, thank you very much.