Okay, In this video, we're going to start talking about a central element of experimental design, which is randomization. Before we go any further, I'd like to draw your attention to this wonderful book. I've talked about this book in other videos. This picture, the book has a little bit blurry. Unfortunately, it's called experimental design for the life sciences. And it's by Graham Ruxton and Nicole grave. And this book provides a wonderful introduction to many aspects of experimental design. And they do a great job talking about features of randomization or how, when and why randomization is important to experimental design. So if you want to know more beyond what I talk about in these videos, I highly recommend this book has a place to go. We're going to break up the topic of randomization into. So this video is going to talk about, we're going to do with part one. Where in this video we're going to talk about using randomization in order to assign different components of an experiment. What do I mean by that? We'll expand on that very briefly, are very shortly. In the next video, we're going to talk about a different topic, which is sampling populations. In this video, we're talking about how we use randomization in order to make certain decisions. Well, we're creating an experiment. This topic in this video ties very closely to the assumptions that we're going to talk about in other videos with regards to the assumptions of statistical tests. What you'll often see or pretty much always see in the videos that I create. We're talking about particular statistical tests is the most important assumption for these tests is what I call random sampling. And this assumption of random sampling. That's what this video deals with. Okay? The term random sampling is a bit unfortunate. Perhaps I should have called it something different, like random assignment. Because what we're really talking about here and in this video is how we can use randomization in order to ensure that we don't introduce any biases into our experiment. What do I mean by that? Well, that's we're going to talk about more. So we have two overall goals in this video. The first goal is to explore why experiments require random assignment. And we'll give lots of examples for how we can do that. And then the second goal is just to introduce some basic methods for using random assignment when we're designing an experiment. Okay, before we really jump in, let's start with some terminology. We're going to focus on an experimental design like this, where we just have two treatments. We'll call this treatment 1 and we'll call the other a treatment 2. And we can imagine that we have, let's say, a 100 subjects in total for this experiment where we have 50 subjects that had been assigned to treatment 1 and another 50 that had been assigned to treatment 2. We can call an experiment like this a one factor experiment with two levels. We call it a one factor because I probably only dealing with one type of manipulation. For example, we might have a drug treatment and a placebo treatment. Or we might have plants where we might have plants that are left alone and other plants where we've subjected them to herbivory. So we've allowed insects to come along and shoe on their leaves. Okay? So we have one general factor, one general type of manipulation. And within this factor we have what we call two levels. So here's one level and the other level. When we say we have two levels, that's just referring to the number of different kinds of treatments that we have within a factor. So I may not use the word factor very much during this video, but I'm probably going to use the word levels a lot in this video. So when I do, this is the context for that term. Okay. So when we're referring to different levels, were talking about different treatments within a factor. What I want to say next is I want to give an overall goal for when we're designing an experiment or when we conduct an experiment. Okay? When we conduct an experiment and we set up different treatments, our goal is to be able to determine whether or not the specific differences that we impose in our experiment between our treatments will cause some particular biological response. So for example, if we had a, as I said before, a drug treatment here and a placebo treatment here. Our goal is to determine whether or not there is specifically an effect of this drug on whatever it is we're measuring. Okay? So we want to design our experiment in such a way that we can exclude other possible, other possible explanations for a difference between this group and this group. So that if we find a difference between these groups, we can correctly describe that difference to the manipulation that we've imposed in our experiments, like the supply and our subjects with the drug rather than a placebo. Okay? So as a result, when we're designing an experiment, what we want, what we need is to design our experiment in such a way so that the only systematic difference between this treatment or this level. And this treatment, or this level is the difference due to the treatment itself or due to the manipulation that we impose. So say that again, we want to set up our experiments such that the only systematic difference between our two treatments is the treatment effect itself. Now, having said that, it should be fairly easy to recognize that when we have subjects living biological entities, living speak you're living organisms. We don't expect all of our subjects to be identical. So our subjects Mike, different age. They might differ in their health. They might differ in their genetics. And as a result of having all of these differences among our subjects, we want to be really wary when we set up our experiment so that we don't end up assigning our subjects to our treatments in such a way that we could have a systematic difference based on these other based on these other factors. Okay, So for example, we would not want to set up our experiment in such a way that the individuals that were in treatment one happens to be, on average, say little bit older or younger than the individuals and treatment 2. Because if we have a, a systematic difference in age between these treatments, then we'd have to differences between the subjects and treatment 1 and treatment 2, 1 to be the treatment effect itself. But the second would be this difference of age. And as a result, if we found a difference between these two treatments, we wouldn't be able to know whether or not that difference was due to the treatment or due to the effect of age. For example. We call this a special name for these other types of differences that can arise like this example of age that I just gave. And we call those confounding factors. Where confounding factor is. Some other systematic difference that can lie between our two treatments that is confounded with the treatment effects themselves. Okay? So knowing that there's always a potential for confounding factors to creep into our experiment. How can we allocate our subjects in such a way that we can avoid systematic differences in these potential confounding factors. Well, the simplest and most effective way of doing this is just to use randomization. We let, we let luck, decide which subjects will go on one treatment or the other. And when we use randomization, what that will do is that will lead us to the strong expectation that there should be no systematic difference between the subjects between our two treatments, except for the difference that we impose upon them due to the treatments that we add ourselves. Okay, So let's make this a little bit more concrete. This is the cutest possible experiments that I could think of. Let's imagine that our experiment was we want to know whether or not petting pets or petting puppies will change the average rate of tail wagging. So we have two treatments or two levels. We have a nope heading treatment and a padding treatment. And let's imagine that these are some of the puppies that might be involved in our experiment. These puppies my different from one another in a number of ways. They might differ in their size. They, different breeds of puppies can have different temperaments. They might differ in terms of their exposure or the amount of exposure that they've had. Two people. If it turned out that we assigned our puppies to are no petting, petting treatments in such a way that, for example, we ended up having more puppies in one group than another that had lots of experience with people, then that experience with people could be a or would be a confounding factor where the extent of experience of people might be associated with one of the treatments. Like it might have a situation where we might have puppies that had lots of experience with people ending up in the padding treatment. And in that case, if petting seems to be associated with more tail wagging, then we wouldn't know whether or not it's actually the padding that causes increase in tail wagging. It could just be because some the puppies in this group happens to have more experience with people. And so to get around this, what we would do is we would randomly assign our puppies to the no padding or petting treatment. Okay? So I've been so far saying how random assignment of subjects is really important to experimental design. What I'd like to do now is really define more precisely what I really mean by random assignment. And to do that, I want you to imagine and experiments that might involve, let's say, fish in a tank like this, okay? And let's imagine we had 10 fish in this tank. And we had a small experiment where we wanted to put five fish into one treatment and five fish into another treatment. What I'd like you to do it, what I'm going to do now is I'm going to describe one way in which we might consider choosing our subjects to go and one treatment for the other. And I want you to think about whether or not this approach really is random assignment. So what I want you to imagine is that we might just perhaps even close our eyes if we want to get really lucky and reach in with a net into this tank and scoop out one fish. And what we're going to say is that the first five fish that we scoop out are going to be assigned to the first treatment. And this next five fish that we scoop out would be assigned to the second treatment. Would that approach really count is truly random assignment of subjects. To stop the video for a second, to think about. That, and if you think North, think about why you don't think it would be random assignment. Okay, now that you've had a good Think, I'll just say, no, this is, this is not a wise way to assign subjects to treatments. And that's because it's very easy to imagine that we're not choosing our fish by random, by random chance. Instead, we're choosing them in order of how easy they are to catch. So the first five fish that we catch are most likely to be the fish that are most easily caught. Maybe they're the biggest ones, maybe they're the oldest ones and can't move as well. Maybe they're the most sick. All of these possibilities raise the strong possibility of our having confounded factor associated with each of our two treatments. This type or this approach to random assignments is often called haphazard assignment. Where it's not truly random assignments, but where we're kind of trying our best to be objective. And so that's why I suggest you might close your eyes because then you're not actually looking at the fish. You try and introduce some degree of objectivity. But I'll just say it's very hard to escape biases when we're using haphazard assign methods. I'll tell you a little story about that in a little bit. So haphazard approaches like I've just described are not truly random assignments. So what could be some good methods for randomly assigning subjects to treatments? I've listed a few here. The first is very low tech. What we might do is you might assign each subject with a number. And then we might write the numbers of all of our subjects on a piece of paper or on a piece of cardboard. And then cut them all to equal size, pop them into a hat. Give had a really good shake and mix them thoroughly. Close your eyes and select them one at a time. And we might say in that case, the first five numbers that we obtain, those ones will go into treatment 1. And then the remaining five numbers would go into treatment two cakes, that would be one possible approach. I'll just quickly name the rest of these and then I'll actually points to bring them to life a little bit more. We could also use a table of random numbers. What is that'll show you that in a moment. We might bring some dice with us into the lab or wherever we're doing our experiment to be doing a field experiment, bring some dice with you and roll the dice to make your decisions. You could also use a digital stopwatch, either on an actual digital stopwatch or on your phone. Or you can also use a random number generator, which you can get on in many computer programs. Okay? So I'm going to try and bring these suggestions to life a little bit now. Just by showing them to you. So here is an example of a random number table. To get this, I just went into Google and said random number table and then went to images. And this is what they look like. And you can find tables like these at the back of many statistical textbooks. Or you can just go online like this. And you could download this and print it and bring it with you to whoever you're conducting your experiment. So how might you use this? You could use it in a variety of ways. What I would often do is let's say we've, let's say we've numbered our subjects from one to a 100. And then what I might do is, is if I'm trying to assign subjects into various treatments and I've got I want 50 subjects in each treatment. Then what I might do is I might say, Okay, I'm just going to run my finger down here and choose the first. First, choose to look at the first two digits. And in this case we'll see we have number 36 is first. So that would say okay, Numbers 30 thick, number 36. You're going the first treatment. Number 46, you're going the first treatment. Number 31, you're going the first treatment. Number 78, you're going the first treatment and so on. Until you've chosen, until you've chosen your, all of your candidates or all of your subjects to go in the first treatment. And then the remaining candidate. So the remaining subjects, they just automatically go and your second treatment. Okay? So that's, that's a random number table. I'm not this this is an old publication on it was published Let's see, when it was published in 1996. You can Google to get this yourself. I just googled this by saying using digital stopwatch as random number generator. And this came up, is published in, I think the journal ecology. It's published in the, by the ESA, which is the Ecological Society of America. So it'll be in one of their journals. And it just explains how you can use a digital stopwatch as a source of random numbers. And this is one of the tricks that I've used when I've been doing field work. Alternatively, you can use a random number generator. There are functions in Excel that can produce random numbers for you. I'm just going to show you some quick functions in R. So one nice function is our UNF, which allows you to choose random numbers from a uniform distribution. So if we just ask for this, but I already have it up actually, this explains for how we can use this function. Okay? Just as quick example. Here, I'm going to be pulling out 10 random numbers between 1100. Okay? So say run and there are our Random numbers, okay, they're not integers, but they are random numbers. If we want to obtain random integers and we can use this function sample int. Where what we can do here is we can say this number here is tells you how many numbers you want to choose from us. This is going to be choosing from one to a 100. And then, whoops, the next digit here tells us the number of random numbers that we'd like to select. And then stop that. This last option here tells us whether or not we'd like to replace the random numbers that we've drawn. And I'll explain what that means in a moment through example. Okay? So the way I have this setup is we're going to just use one random number from one to a 100. And so they have chosen the number 76. Okay? Let's say instead, I want to choose 10 random numbers. Okay? So we've got now 10 random numbers between one and a 100. Let's say I want to choose a 100 random numbers. Well, what I've done now is, or what this function is done is it's essentially given us a 100 random numbers. But because I've said replace equals false, what I've basically done is just taken the numbers from one to a 100 and shuffled them in terms of their order. And this seems like a good place to explain with this, replace equals false means. What this refers to is to say that if we have the numbers one to a 100, once we choose our first random number, we then have the choice of whether or not we want to replace that number that we've chosen back in with the original 100 numbers? Or do you want to strike it off the list? Are the possible numbers to choose before we take our next random number. Here, I've said that we are not replacing our random, that we're not replace the numbers that we've drawn. And so as a result, when we first drew the number 31, that number was no longer available among the remaining 99 numbers to choose. Um, and so the next thing that sample int is going to do is it's going to choose another random number among the remaining 99 numbers. And that happened to be 82, and so on. Okay. What would we do if we said replace equals true? And I'm just going to start what I want. Let's say we're choosing numbers from one to 10, but we're going to choose them, choose a 100 numbers from one to ten and we're allowing there to be replacement. I don't know in what circumstance this would actually be useful. You might not be useful at all, but I just wanted to show this to you to give you a sense of what replace equals true means. When we say replace equals true, it means we can draw the same random number more than once, which is why we see ten. There. There are ten there, ten there, and so on. Okay? So those are some nice functions that we can use to draw random numbers. What I'd like to do now is just show you some very briefed scripts that allow you to just create. Well, I'm going to create a DataFrame is what I'm gonna do. I don't know why was it lost for words there? Um, what this code does here is it allows me to create a DataFrame where we have two columns. I'm not going to go through this code specifically because that's not really our goal. And I don't want to make this video too long. What I've done here is I've created a setup where we have two columns. One's called treatment and we have treatment a and treatment B. And then we have our subjects which have just called 1, 2, 3, 4, 5, 6, 7, 8, 9, all the way down to 20. We could use some very simple scripts in R In order to randomly assign our subjects to our various treatments. So what I'm imagining is we might assign our subjects, say the numbers from one to 20. And then what we'd like to do is you'd like to use R to randomly assign our subjects to these various treatments. And we can do that with this code here. Where what I'm doing is I'm taking our original DataFrame, which I've called mydata. You can see that's what I call down here in order to call up the DataFrame you've created. And I'm just saying, let's focus on the column called subject. So they're down here is our column called subject. And I'm saying I want the output from the function that we're going to use to go into this column called subject. The output that we're going to create comes from this function called sample. Or what you do is in, what you do to use sample is you supply it with a few bits of information. The first is you say, where's the data that you want to sample from? And what I want to do is I want to sample from this original column of subjects. Okay, So we're going to be drawing values, are drawing random numbers. Come on. Why can't I think I want to go up? There we go. I want to be drawing random numbers from this column. Which is why I've said my data dollar sign subject. Okay, That's how we're pointing to our, to this group of numbers. Next we tell it how many random numbers who want to draw? Well, we want it to draw all of the numbers that are here. And we can tell R to do that by just saying, well, we want to draw as many random numbers as there are rows in this DataFrame. Okay? It can see there's 20 rows in this DataFrame. And so if I say n rho my data, and that means this is going to give us the number 20. Okay, I could have just written the number 20 here. I often find it easier to just use this approach because this approach will never fail or shouldn't say never fail, become, might always do something silly. But this way I don't have to think about how big my DataFrame actually is. And then the last thing that we have here is I'm using this replace equals false command or Option. So that once we've selected a subject, we cannot select it again. So that overall, this approach or this code will simply reshuffle the subjects that we have here into a new order. And I'll just demonstrate that here. So if we say Run, and then we say, my data. Here we go. We've now randomly assigned or randomly shuffled are subjects to treatment a and B. This is a really nice and straightforward approach to assigning subjects randomly. Two were different treatments. Okay? So those are a number of methods we might use to assign or subjects to our treatments. I just want to stop and pause the video minute for a moment here just to summarize where we've been. Okay, the main message that I'm trying to give on from this slide and for everything proceeding is that when we randomize our subjects, we wants to choose a method that will allocate our subjects to our treatments with equal probability. That's not true. Or this, this goal will not be met if we use a haphazard approach. Why do we want to do this? Why do you want to use this random assignment? We want to do this in order to ensure that we do not have any confounding factors associated with our treatment differences are with our treatment levels. And we want to do that. We want to avoid these any confounding effects to make sure that the only systematic difference between our treatments is the manipulation that we induce ourselves. So that if we do find a difference between our treatments, we can be, we can be very confident that that difference arises because of the treatment and not because of some other factor in our experiments that we have not accounted for. Before we go on, I want to point out, we've been talking about a simple experimental design where we have two groups. But these approaches that I was just outlining back here, these approaches will work for other experimental designs as well. So for example, let's imagine we had an experiment where we wanted to investigate how different combinations of drugs might work at treating cancer, for example. So you might have two drugs, cancer drug a and cancer drug B. And we might imagine an experiment where we have 400 subjects. For Drug a, we can either not to give the sub not give a subject drug a or give it drug a. So we have no and yes for drug a. And the same thing's true for drug B. So some subjects will not receive drug B and others will receive drug B. With an experimental design like this, we have four treatments are for treatment combinations that represent all of the various, all the possible combinations of these two factors. Drug a is one factor with two levels, Drug B is another factor with two levels. So here we have a 100 subjects that received neither drug. Here we have a 100 subjects had only received Drug a. We have a 100 subjects had only received Drug B. And here we have another 100 subjects that only received Drug a. I'm sorry, receive both drugs, sorry. Okay. And we could randomly assign our subjects to these treatments using a number of approaches. So one approach might simply be to say, okay, we thought 400 subjects, let's randomly assign half of them to the yes, no treatments of drug a. Do that using the approaches you've already described. And then once we have 200 subjects in the know and 200 for yes, for drug a, then we could, within the NO group, randomly assign those to hundreds so that we have a 100 and no for drug B and a 100 and yes for drug B into the same approach for these remaining 200 subjects. Or alternatively, we might just kind of list all of these different drug combinations like this where we have a no B, no a, no B, yes, yes, no, yes, yes. And we might set up a spreadsheet like we had in our few moments ago, where we might list this combination at 100 times, followed by this a 100 times and so on. And then just shuffle our subjects into these various treatments. And an analogous way that we did in our hue moments ago. Okay? So the main point here is that although we're largely focusing on really simple experimental designs, for the sake of this video, the approaches and the thinking that we're using apply to much more complex designs as well. What I'd like to do now is I really want to relate some of my own personal experience with randomization because, well, I have a story to tell. Okay, What did the steroid like to tell you is comes from the very first real experiment that I did in my, in my training. This was when I was doing my master's degree, which is back in the late 90s. So I'm going to describe an experiments that I did, I think in 1997 and 1998. So in this experiment, what is, what I was interested in doing is determining whether or not the arrangements of flowers on a plant can influence the behavior of the pollinators that visit that plant. Just very briefly, we're interested in doing that because we wanted to have and to understand why different species of plants arrange their flowers in different ways. That was the ultimate goal. That, that's not really the point here. What I did for this experiment is I worked with this species here, which is called del Finney and Glaucon are common law exper. And what I would do is this was working at a wonderful setting in the foothills, the Rocky Mountains. And there was a big field filled with these plants and with lots of bumble bees flying around. And what I would do is I would look around you, the field for a plants that had the characteristics that would allow me to use it. So one criterion or one aspect of this experimental design was that I always needed to have plants that had at least say, 15 flowers being displayed, which is probably something along the lines of what this plant has. Okay. And the reason for that is because when I created my treatments, when I created my, the, the, the, the plants that I wanted to present to bumblebees. I wanted all of the plants that are presented to bumblebees to always have eight flowers. And that's what these circles mean on these various cartoons. So I had three different treatments. This one I called unmanipulated where it just had eight flowers. I had another treatment where it also had eight flowers. But the density of these flowers, it was much lower because I removed every second flower as you moved up the stalk. And then for our third treatment, I removed half the flowers again, but I didn't remove every second flower. What I did is I just removed the flowers on 1.5 on one side of this display. So for example, maybe I'd remove all these flowers, leave these ones intact. And then what I would do is I would take these, these manipulated the call to inflorescences and present them to bumblebees and look at what the bumblebees did. So point is I had to be selective about which, which plants I used in this experiment. Because, for example, I couldn't use plants that produce too few flowers if they had fewer than eight flowers, and that was an impossible plant for me to use. Ideally, I wanted all the plants to have at least 15 so that if I removed half the flowers, I'd still have eight left or 60 and I should say. So just want to walk you through my experience with this. It took me two years to run this experiment. It's a fairly straightforward experiment. Here's what I did in the first year. And the first year, what I did is I went out and I looked around and I chose a and I chose a, an inflorescence or it shows a plants to use. And then I just thought, you know, I'm, I'm going to give this one say. And manipulated treatment. And that's what I did. And I removed the flowers according to this schematic and presented to the B's and got my data. And then I choose another plant that's a valid use a low density treatment on this one. In other words, I was using a haphazard approach to assign my plants to their treatments. Now while I was doing this, I not only recorded the bee's behavior, but I also measured some other aspects of the plants that I selected for the experiment. So for example, I recorded the original number of flowers that were on the plants before I removed any of the flowers, I measured flower size, that sort of thing. And I kept that information in my notebook. And it's a good thing that I did because I, after running this experiment, I went back in the lab and I start to analyze my data. And I found, we found what look like interesting results. But then I want to check to see whether or not there are any systematic differences among the plants that I've chosen to put into our different treatments. And I've found Sure enough, there was a camera up exactly what it was because this was years ago. But I believe with something like, you know, one of the treatments happened to have happens to use plants that had more flowers on them than the other ones. And that was really bad news because that meant that I could no longer say that the difference between that I saw on the bee's behavior among the various treatments was because of what I did to the inflorescences. It could adjustment because there were already some inherent differences among the plants. So for example, if plants were larger as if they're producing more flowers, that maybe they were just more vigorous plants and more vigorous plants might produce more nectar. And so maybe the bees were behaving differently on one sheet or the other. Not because of the manipulation, but because the flowers they were visiting happened to have more nectar in one treatment than the other. So that whole summer of work just had to throw that out the window, had to start over. So I went out again the next year. And this time I was determined I was going to use proper randomization. And so what I did in this case when I started make my work in the field is I would choose a plant. And then I said, Okay, I'm going to use a random number generator or my watched you assign this plants to a particular treatment. And that works great. At least at first it did. But then it turned out that just by random chance, I ended up getting a whole bunch of data for one of the treatments and much less data for the other treatments. And I started becoming concerned that. If this kept up, I might end up having a good sample size for one treatment, but a really poor sample size for another treatment. And so this approach might have helped me to avoid the kinds of biases that I'd had in the first year. But it still wasn't ideal because I could still end up with a dataset where the sample size, it was just too small for one or more treatments. So then I changed the approach. What I did is instead of going out and choosing one plant at a time, I would go out and find three plants and select those three plants. Then based on those three, I would randomly assign a particular treatment to each of those three plants. And then I'd take those plants, manipulate them given to the B's. And by doing that, I will basically move my experiment forward in blocks of 33 data points, one data point from each treatment. And by doing that, I was able to make sure that my data were, I was able to avoid confounding effects and was also able to make sure that my sample sizes were relatively equal. Okay, that was finally a success and that finally worked. But it took a lot of learning to get to that point where I could have a dataset where he's actually confident about the results that I was getting and believe that they're actually real. This final approach that I described where I worked in kind of blocks of three. So choose three plants, then randomly assign those three plants, the three treatments. This final approach ended up getting me what's called a balanced experimental design. So what is balanced mean? A balanced experiment is where you have an equal number of subjects in each of your treatments are in each of your levels. Naturally, this should mean that an unbalanced design is where you have an unequal number of subjects in each of your treatments. Given the choice between a balanced and unbalanced design, balanced designs are more desirable. And that's because balanced designs tend to be more powerful when you're analyzing your data. And they're also less prone to problems when your data violate the assumptions of a test to some degree. In other words, if it looks like your data might violate some of the assumptions that are required for a particular test. We would worry more about violating those assumptions if your data were unbalanced compared to situation where your data are balanced. Okay? So the point here is just to say that when you're assigning subjects to your various treatments randomly, it's ideal to try to do that in a way that will give you a balanced experimental design. I want to shift gears for just a moment. Just to think about randomization from a slightly different perspective and ask how it is randomization matter. Okay? We've to a certain degree of already answered that question because we've already established in this video that without randomization, we can't be certain that we don't have confounding effects. And to run our experiment, which could then lead us to incorrect conclusions. Okay, in other words, without randomization, we can't trust whether or not a treatment effective interest is actually the cause. Some of the observed effects we find between our treatments. So that's we focused on so far. What I'd like to do on this slide is point out that randomization or lack of randomization can have other kinds of effects as well. Where randomization or lack of randomization can influence the effect size that we seem to get from our experiments. And here's an example from a meta-analysis done by Malcolm MacLeod and a number of other people. Where they were, where they looked at a variety of studies that were all studying stroke. And the studies were looking at the effectiveness of this x, y 059, where as I understand it, I'm not an expert on stroke where this is a, a, a scavenger for free radicals. And it's thought that this might help recovery from stroke. As I understand, as I said, I'm not an expert on stroke. And so what Malcolm MacLeod and others wanted to know is what can be learned from looking at all of these, all studies that have looked at this topic. And do we end up getting different kinds of conclusions depending on how the experiments are designed. They were able to, they turned out that not all of the experiments had randomize their subjects. So these two bars here represent the effect size or the amount of change in the variable that they were measuring in response to a treatment with an X, Y, Z are 59. And what they found was that NX, why 59 seems to have a larger effect when the data or when the subjects were not randomized compared to the situation where they were randomized. And so what this suggests is that not randomized and your subjects can actually lead to inflated effect sizes. Or in other words, it can make it look like you're getting a more dramatic response than you actually should be. Are they actually should find. This example here is just one. You can find other examples like this in literature. I'm just highlighting this as yet another reason to emphasize why randomization is so important. So far, we focused on randomizing subjects. So how did we decide which of our treatments are various subjects are going to be allocated? How we can decide how to allocate our subjects to our various treatments. There's much more to an experiment, however, than just simply randomizing your subjects and you various treatments. One obvious component is that experiments take place in a physical environment. And we need to consider aspects of space, aspects of the physical conditions of those environments when we're conducting an experiment. And the reason for this is because the environmental or the experimental conditions can depend on where you are in your area, where you're conducting your experiment. Other words, experimental conditions can vary over space. So this photo here, this is given to me by Genie's Freedman, who's a friend and collaborator and an excellent scientist. She shared this photo with me where this was from an experiments that she conducted a while ago working with a plant called ragweed. And she did this experiment in the glass house. There were lots and lots of plants. And these plants were grown on benches. So there's one big bench, there's another big bench, there's another big bench. And we can easily imagine that the conditions at these plants experience will depend on where they are on the greenhouse or the glass house. So these plants that are right here, they're very close to this end wall, whereas these plants on this bench are farther away. There might be some plants that are near some panels that have newer glass, whereas other ones might have older glass. And the quality of the glass might influence equality, the light that's coming through, et cetera. In other words, we can expect that the, the, the living conditions for the organisms, the organisms in our experiment can depend on where those organisms are in our study space. Okay, I'm enginius this case, we have some nice data, what we can actually quantify this. So engineer's his experiment. She had, I forgot to count maybe 12 different benches. Each of these columns of data represent data that come from a different bench. So we have 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 different benches in this experiment. And what I've plotted here is one aspect of the plants. As they've grown. I planted that, I've plotted the number of branches that plants had on day 239 of the experiment. And what you can see is that the number of branches that plants produce can be really different depending on which bench they were grown on. So the plants grown on this bench had a median number of branches of about five. Whereas on this bench, they had, the median value was more than double that. Okay? So we can see there are big differences among the benches in terms of the growing conditions that should trouble us if we're designing an experiment. So for example, let's imagine we wanted to run an experiment where we might want to say, let's imagine a look at the effect of herbivory on plant growth. And so we might have one treatment where we take herbivores like little insects, me put them on the plants. And then we might have another treatment where there is no herbivore put on the plant. And so we have one sheet and we expect the plants be eaten by something and other treatment where we don't expect the plants be eaten by something. If we set up an experiment such that we only had, say, the non herbivory treatment on this bench. And we only had the herbivory treatment on this bench, then we'd be confounding the the effect of herbivory with the effect of being on one bench versus another. So it's very plausible then that any difference that we see between or herbivory treatments could actually be because of differences in the growing environment. So as a result, what would, you know? How, how should we set up our experiment? Well, we should be doing is setting up our experiments so that we have all of the treatments randomly assigned or are all of our subjects from a various treatments randomly assigned to the various areas of our experiment. So for example, you might say, within this bench, we would randomly assign subjects to this bench. And we'd randomly assign those subjects equally to our various treatments. So if you had two treatments and let's say, let's imagine this bench had enough space for 50 plants than we'd randomly choose the plants that would go on this bench. And we randomly assign those, 25 of those 50 plants to the non herbivory treatment, and randomly assign the other plants to the herbivory treatment, for example. Okay. Now there's, you want to make sure that we set up our experiment in such a way that we do not confound our experimental treatment with differences in space. So I've used this example here to highlight conditions we want to account for and a glasshouse or greenhouse experiment. But these problems can be everywhere. So for example, let's imagine your growing Drosophila in an incubator or growing cell cultures in an incubator. Those incubators might have different shelves and those shelves can represents different growing conditions. As a result, if you put all of your drosophila or cell lines for one treatment on one shelf and for another treatment on another shelf, then you would be confounding your treatment and the location and everything that's different among those locations within your experiment. Okay. So if there are differences between those shelves, then those differences among the shelves could actually be what's driving your, the effect that you're hoping to find figure differences among your treatments. Some people might say, well, incubators that we use, they're really expensive. They're really good. I don't have to worry about that. I would say that you'd simply never know. There are cases where I have spoken to people where they have said, we're working with Drosophila. They have found that there are differences among the shelves within their incubator for the growing conditions for Drosophila. So there are certainly cases where differences among shelves within incubators can be important. If you think that your incubator is particularly special because it is especially important and you don't have to worry about these things. I would say. The only way you can know for sure is to actually run an experiment to proper experiment to determine whether or not there actually is any reason to worry about differences among the shelves. Much easier than any of that, is to simply randomize your locations for where your treatments go to avoid any confounding effects of, of location. That's the easiest thing to do. Another quick example that might be relevant for people working with animals that live in cages. Cages are often held on racks and racks of a vertical component to them. And cages that are higher up, maybe closer to a light source. And how close you are to that light source can affect how stressed the animals are. And as a result, animals that are higher up in Iraq might be more or less stressed and animals lower down in Iraq. So if you set up your experiment such that animals in a cage, the top of the rack experienced one treatment and the bottom of the rack experience another treatment, then you could be confounding your treatment effect with something like stress level due to the environment that the organisms are living in. Again, this is the best approach to avoid this kind of situation is simply to randomize location of your subjects with respect to space. Okay. We're just talked about how you can have variation in space should come as no surprise then you can also have variation in your conditions over time. So over the time course of the experiment. For example, experimental conditions can vary over time. So if you're doing an experiment outside in a field, then the, the weather conditions might change. As your experiment progresses. The age of your subjects is going to change throughout your experiment. So the experiment might, on average start with younger individuals and end with older individuals. That change in age could be particularly important if you're working with short-lived species, the quality of your equipment might change over time. The calibration might change subtly over time. The scientists themselves can also vary. So for example, let's imagine a scientist is focus, focus yet I'm making particular, making a very tricky type of measurement. It could be that as they collect their data and as they get more practice making these measurements, that they get better at making these measurements over time so that the quality of their measurements at the end of the experiment are better than those at the beginning of the experiment. Alternatively, the opposite could happen. We know that collecting data is really hard work. That's why have this picture here of some flopped out on the floor. This is how I felt at the very end of a long day of collecting data. It's entirely feasible that as you get more and more tired, you may be more prone to making mistakes. So the quality, quality of your data at the end of time, sorry, end of time, that sounds, that sounds very ominous. At the end of some time period might be very different from the quality of the data that you collect. Another time period. As a, as a result of all these possible changes over time, what we want to avoid is a situation where we might, let's say, collect data from different treatments at different times. So for example, let's imagine you had two treatments and you started out by collecting all of your data for one treatment first, and then you collected all of your data for another treatment later. For all these reasons we've just talked about. You could end up getting systematic differences between these two different treatments. Not because of the treatment effect, but because of all these possible differences that can arise between your subjects over time. Okay? So as a result, what we really want to do is he wants to randomize elements of our experiment that occur over time, including randomizing the order in which we take measurements. So for example, let's imagine we have collected data from a whole bunch of subjects. We've, let's say we've got are the things that we want to measure, collected and envelopes or maybe. The images stored on a computer desk. And we're going to analyze those images. What we do not want to do is analyze all the data or make our measurements for one treatment first and then all tick of our measures for another treatment second, because they were confounding time with the treatments. What we'd like to do ideally is randomize the order in which we collect our data with respect to treatment. So randomly select a subject to measure from the first treatment. And, or start randomly select one individual for our first measurements that might come from the first treatment, and then for the second treatment, et cetera. Here's an example of this from an experiment that I did. Again, this is from during my master's degree in the, in the late nineties, where I was interested in understanding how bumble bees foraged differently when we arranged flowers in different ways. So I describe that experiment they did in the field earlier. Here's an experiment that I conducted in the lab where I created artificial flowers, use the end of centrifuge tubes and some paint and some glue and stuck them on the end of some pins. And I created these three types of structures that are meant to mimic different types of experimental designs in the field. And when I was studying the behavior of the bees on these different different types of floral displays. What I would do is I would study the bee's behavior on one type and collect all the data for the bee's behavior on that one Type. And then move on to another type and then move on to another type. And there's reasons for why he did it that way. But I'm not going to go into I'm not gonna go into those reasons because that's not really the point. They're more esoteric for this particular experiment. The point here though, is that we can imagine that the bees might get better and better at their foraging as they get more and more experience. And so that towards the end of all this data collection, the bees might be performing better by some metric at the end of the experiment compared to at the beginning. So what I wants to avoid, because that might be true as I wanted to avoid a situation where I examined the bee's behavior on these different designs, always in the same order. So what I did instead is I said, Okay, for the first b, we're going, this first B is going to visit this type of inflorescence, then this one, and then this one. And then for the next B, I just randomize the order in which it experienced these different types of arrangements. So the second B might have gotten this 1 first and then that one, and that one, and the third B might have gotten a different order, et cetera. Okay. So here I've tried to allocate the order in which the bees experience He's treatments randomly. This isn't always possible. It's not always possible to randomize the order of the treatments in which individuals might experience them. So on this slide, I've just want to outline an experiment that's run by third year neuroscience students at the University of Edinburgh, where they want to study the effect of the hindbrain and how it influences movement in zebrafish. And what they do is they start out with a no manipulation treatment where they take the zebrafish and they measure aspects of the fish is swimming, where the fish is in more or less its natural state. And then what they do is they apply a lesion to the hindbrain. So they basically destroy the hindbrain for the zebrafish. And then they measure swimming again. Okay. And then they apply a third treatments where they apply a substance called an NMDA. Okay? In this kind of experiment, you can imagine it's impossible to randomize the order of these treatments. Once you've applied a lesion to a hindbrain, there's no way that you can look at the noumenal, no manipulation treatment afterwards. That's physically impossible. So there are some experiments like this where we have to be very, very cautious in terms of how we interpret any differences that we might find because we are confounding the effect of treatment with the effective time. Okay? So be very conscious of how time could be a confounding factor in your experiments. I just want to make a very brief comment here. These types of types of designs where individual subjects might receive experiment or might receive treatments in different orders. These are often called crossover designs. If you were thinking of using this type of designing your own research. I want to point you to this paper here because often experiments with this type of design are not analyzed appropriately. So if you're thinking about using this type of experimental design, you want to make sure as you analyze your data appropriately. And so I want to point you to this paper. Okay? What I think, what I hope you're getting from this video so far as the overall message should be, we want to randomize every aspect of our experiment as much as possible. Okay? And that point has really made nicely in this paper here, which came out, I believe in 2020 in PLOS Biology, haven't listed the authors because there's so many of them and I didn't have space on my slide. But you can easily get this paper just by Googling this title in to find it. And they list a number of things you might really want to pay attention to when you're designing your experiments in terms of what's to randomize. For instance, they suggest randomized in the day or the time of day at which you might implement a particular experiment. And by that I mean, I think they're referring to collect data for a particular treatment. Okay. You might want to randomize the litter that your animals might come from in terms of which litter is assigned to which particular treatment. You should read as randomize aspects of which cages would say, which cages. Which cages would be assigned to a particular treatment, or which individuals are assigned to which cage or fish tank. If you have more than one investigator, more than one surgeon, you would want to randomize the particular treatments that those surgeons or investigators are actually working with. So for example, if you have more than one surgeon, then you would not want to have a situation where I had two different treatments that both require surgery. You would not want to have one surgeon perform all the surgeries for one treatment, add another surgeon, apply all the surgeries for another type of treatment. Because then you confounding the type of surgery and the surgeon themselves. Instead, you'd want to randomly allocate the particular subjects that receive one type of surgery or another to your very surgeons. The same is true when you are, when you might have different investigators. I'm taking measurements. So let's imagine you have all of your materials that you want to measure listed out and are laid out in front of you. You want to avoid a situation where one researcher might tends to make a measurements from one treatment and another researcher tend to make measurements from another treatment. Instead, you'd want to make sure that each researcher, for example, randomly selects the subjects that they're taking measurements from, uh, from your two different treatments. So that overall, we didn't not confound our treatment with who is actually taking the measurements for those various treatments. They also suggest randomizing which equipment is used for your various treatments. So if you have multiple PCR machines, you do not want to always use one PCR machine for one she went and another PCR for another treatment. You want to randomize which PCR machine is used for which aspect of your experiment. And I'm not going to go through the rest of these ne, detail. You get the idea. I'll just point out that there they also say it's important to randomize location aspects of your experiment. This has been a long video. I just want to end with one final example just to emphasize this point of trying to randomize everything you possibly can. So these two pictures were pictures that I took from an experiment that I ran a number of years ago. Where this was a fairly large experiment where I had over 1000, 500 plants and this glass house. And this experiment involved a number of steps. And I tried to incorporate randomization as much as possible and to each of these steps. So for example, I need to produce seeds. Using this experiment. And so to do that, I would take a plants that I wanted to be the mother of those seeds. And I would randomly choose which other plant I would use as a source of Poland to produce those seeds. And to do that, I used a random number table, okay? Among the seeds that were produced, I haphazardly chose a seeds that I wanted that I was going to use an experiment. But then I randomly assigned those haphazardly chosen seeds to their various treatments. So what do I mean by haphazard? What I mean is I might have taken a fruit that might contain all the seeds from a particular pollination. And then I would take that's that fruit, empty out all of those seeds onto a piece of paper. And then I just haphazardly chose the seeds that I was going to use in the experiment. So I tried to not choose only the biggest seeds. For example, I tried to just use random seeds. But truly choosing seeds at random is a difficult thing to do. So I chose them haphazardly. Important thing though is that when I assigned those seeds to those various treatments, I assigned them to their treatments using a random number generator. When choosing where these plants are going to grow. You can see from this picture here that these plants are grown in groups within trays. Okay? And I would randomly assign which plant that grew from those seeds would be in which tray. So I randomly assigned the seas to particular tray to grow within. And then I randomly assigned these trays to a particular location within the greenhouse. And I did that using a random number generator to create a map of where these, these various trays would go. That on a weekly basis, I would randomize or re-randomize the locations of these trays within the greenhouse. And again, I used a random number generator. Okay? And at the end of the experiment, when we were making measurements on, out on these treatments, I made sure the order of the play of the plant measurements were random with respect to treatment. And also that the yeast trays were randomly assigned to researchers. In other words, the pitch them trying to create here as I tried to use true randomization as much as possible throughout this experiment. The last point I'd like to make in this very long video is I want to raise a question. What do we do if we cannot randomly assign subjects to a level or a treatment? So for example, in biology, we very often want to compare differences between sexes. So or another way of saying that is you might want to compare aspects of biology for individuals have different complements of their sex chromosomes. So this female duck is most, is going to have Zed W sex chromosomes. Whereas this male duck will have zed, zed. That's how sex determination occurs within, within ducts. In this case, we cannot randomly assign a particular subjects to a particular complement of sex chromosomes. What do we do in that situation? Well, in that case, what we really want to do is that when we choose our subjects for our experiment, we want to make sure that we've chosen our female and male subjects in an appropriate manner for our particular experiment. And that is a subject of our next video. And on that note, I'm going to end the video there. I can say hope this video has been helpful. And I will say, thank you very much.