Blog Entry 4: Design of Experiments
Hello. I'll just get straight to the point with this blog because there are a lot of graphs to go over.
​
Basically for the case study data, the first thing I did was transfer the table given into excel for easier picking of data.
So yeah, as you can see on the left is the given table while the right is the table transferred over to EXCEL. My admin number's last two digits is 23 to all the XX on the left table has been replaced with that.
​
Once that was done, I began the process of solving the effects of single factors.
​
This involved getting the value of the response variable (weight of bullets) of each high run for the various factors and vice versa for the low runs. It is a bit hard to explain so I'll just show you what I did.
Yes, as you can see for Factor A, which is the diameter of the bowl containing the popcorn. The first thing to do the sum of the weight of the bullets for the runs where A is high then get the average of the sum. You can see that being done with the highlighted boxes all being divided by 4.
Then I did the same thing but for the runs where A is low, and got a result of 1.4325g for high A runs and 1.58g for low A runs. The last thing to do is to find the difference between the high and low value. The value of such will be illustrated later in the graphs.
​
So that's for Factor A. I repeated this for the other two factors, B (microwaving time) and C (power).
The result is the above image. The average weight of the bullets when B is high or low and when C is high or low. From this I can already tell which factor is the most significant but to make things easier for everyone reading, the table can be converted into a graph... Which I did.
Ok now to interpret the graph. Basically from the previous table, the total effect of each factor will translate to the gradient of the graph. The greater the total effect from 0, the steeper the positive gradient. The smaller the total effect from 0, the steeper the negative gradient.
​
Usually the steeper the gradient, the more significant the factor is but you have to take into consideration what is the goal of the experiment. In this case, the goal is to lessen the amount of bullets (un-popped popcorn kernels), so a negative really steep gradient is desired. Thus, the most significant factor is C, the power setting of the microwave, followed by B and A which has the most gentle gradient. It is one to note that no matter which factor you increase from low to high, there is still a decrease in the response variable, so if one factor cannot be altered, you can still lower the amount of bullets.
​
Summary, as power setting, microwaving time, and diameter of bowl increases, the amount of bullets decreases.
​
So.... to conclude that whole thing the most significant single factor is the power setting of the microwave, as the amount of bullets decreased when the setting was changed from low to high, then microwaving time and lastly diameter of the bowl.
Also before I move on, here's a funny very goofy aah thing I found out while doing this part.
When calculating the averages for A and B, I had to click individually each cell and press the + key on my keyboard in order to sum the values and the type "/4" to get the average. I only remembered when I got to factor C that there was and AVERAGE function. Yeah, that's pretty much it. It saved me a bit of time when creating the tables and stuff. I used the AVERAGE function for the rest of the calculations after this.
​
Next for the data analysis, is figuring out the interactions between the factors.
​
This is even harder to explain without showing, so I'll just go straight into explaining how I did this section.
This example was when I did the A-B interaction.
​
So on the left you can see I made a table and one of the rows is the average of high A response variable values when B is low. That's basically what I did. I took the average weight of the bullets for runs where when A is high and B is low simultaneously. Then I did the same thing for when A and B are low, and lastly for when B is high.
​
And just like the rest of the experiment, you repeat this same format for A-C and B-C interactions. The result should be the image below.
Again, it'll be easier to look at the effects when converted to graphs.
From what I understand, the way to interpret the graphs is to look at the gradients. If there lines are nearly or exactly parallel, there is little to no interaction between the two factors but if the gradients differ greatly, there is a significant interaction. This can also be seen in the tables as "Total effect" it is just the difference between the to high and low average values.
So, looking at the graphs one can tell that there is some interactions between the various factor but not really significant. The best interaction would be B-C followed by A-B then A-C.
​
To conclude the full factorial data analysis, the individual factor that has the greatest influence on reducing the amount of bullets is the power setting of the microwave followed by the time inside the microwave and then diameter of the bowl. When looking at the how the different factors interact with each other, the interaction between the microwave power setting and timing in the microwave has the greatest effect on reducing the amount of bullets.
The previously mentioned data analysis is called "full factorial data analysis" so now, I shall move on to "fractional factorial data analysis".
​
The steps to undergo fractional factorial data analysis is the same as full factorial, however this is used when the total number of tests needed for full factorial is too many to do and too time consuming. For example, doing 64 tests is a lot to do and very time consuming so cutting down the number of tests by half would be more efficient.
​
It is also good to note that even though the number of tests are being cut, there should still be an equal number of low value and high factors in that halved number of tests. I shall use this case study as an example of how I did it.
As you can see in the above image of the the table of test results, I cut down the runs from 8 to 4, and if you count the number of high and low runs for each factor, there is both 2 low and high runs. This is a balanced design and is apparently called 'orthogonal'. It provides good statistical properties.
​
That is one of the biggest differences when doing the fractional factorial data analysis. The rest of the steps is the same with some minor tweaks. Like instead of having 4 low and high runs for each factor, now you have 2 so you would need to divide the sum by 2 to attain the average.
Take the above image again for example, the average of high runs for factor A is now just (3.23+0.32)/2. Subsequently, all the average values will change and the total effect will also change.
As you can see, the table on the left is the table for the effects of the individual factors for the fractional factorial while the table on the right is the full factorial. The values are different.
However, ultimately when looking at the data and the graphs, the final conclusion of the fractional and full factorial data analysis should be the same. The most significant factor based on the fractional factorial data analysis is the power setting of the microwave followed by the microwaving time then the diameter of the bowl. This is completely aligned with the full factorial.
​
The main difference is that fractional factorial data analysis suggests that as the diameter of the bowl increases, the amount of bullets will increase. This is different from the full factorial where an increase in value of every factor would decrease the amount of bullets.
​
The slight difference in the the results is due to the fact that the data is not necessarily complete so there will be some missing information that may or may not end up in the final data analysis.
​
For the interactions, it is also the same just lesser values to use.
The table on the left is the fractional factorial while the right is full factorial. Again, the values are different as this time the average values for the fractional factorial is not even an average anymore, it is just the data because in the full factorial it is the average of 2 values but since it has been cut down, there is only 1 value.
For the fractional factorial interactions, the order of significant interactions has changed. A-C interactions is most significant while I would say that the A-C and B-C interaction have almost the same significance but the B-C interaction still brings about a decrease in the amount of the bullets as at both low and high C there is a decrease, so I would say the interaction for B-C is more significant than A-C.
​
To re-emphasize, the change seen here in the fractional factorial data analysis as compared to the full factorial is largely due to the fact that the data analysis is 'less than full' so there are missing information that has not been analysized.
​
The final conclusion for the fractional factorial data analysis is that the power setting of the microwave is the most significant individual factor followed by the microwaving time then the diameter of the bowl. This is the same as the full factorial data analysis. However for the interaction, the way the diameter of the bowl and the microwaving time gave the most significant interaction which is different from full factorial that was microwaving time and microwave power.
​
Overall, I would say that whether one chooses to do full or fractional data analysis, the end result would not differ so much when done properly. If there is enough time, full factorial data analysis is definitely worth doing especially if the small details are important but factorial would be useful if there is not a lot of time to spend on the testing.
The following link is for the excel sheet for my full factorial data analysis:
Reinard CP5070 Design of Experiment Blog Case Study Full.xlsx
​
The following link is for the excel sheet for my fractional factorial data analysis:
Reinard CP5070 Design of Experiment Case Study Factorial.xlsx
​
Both links can only be accessed if you use the Singapore Polytechnic email.
The last thing I will talk about for the blog is my experience with Design of Experiment.
​
To be very honest, I actually quite enjoyed this topic. I find very useful for me as I wanna do something related to research for my final year project and internship, and ultimately as a job (lab work). All these things that I want to do in the future all involve doing testing and having to analysis the results to I would say that learning and practicing this data analysis skill was very insightful for my learning and I would say I benefited a lot from it.
​
Frankly, when I first encountered this topic during the tutorial it did not occur to me what would be the implications of learning this. I actually had no clue what I was doing during the tutorial practices. I think my head was in another place or I was just really exhausted so trying to understand the slides was difficult as the time. I, however, listened to Dr Noel's explanation and some more parts became clear to me.
​
During the tutorial when the question asked was which factor was the most significant, I remember doing something so unrelated to the proper data analysis taught in this topic that I feel kind of dumb when I think to it. If I try to recall, I think I only looked at factor A and the average of the response variables and somehow concluded A was the most significant without looking at the other factors but yes, as the tutorial progressed I did understand more and more what was going on and that's when it clicked that I may be using this quite a lot in the future.
​
For the pre-practical at the time I thought I was right but now that I've finished this whole topic, I realised that I did the fractional factorial data analysis wrong. When it came to choosing the 4 runs from the 8, I did not choose the runs evenly. I ended up choosing runs where each factor only had 1 high run and 3 runs which is extremely uneven and definitely would not give any good statistical properties. I think that shows that I did not fully understand yet what was going on. At the time I thought that as long there is a change from low to high value for each factor, that would be sufficient but now I see that there is much more to it and that previous idea I had was wrong.
​
Also something that I realised, I like creating the graphs.
​
When it came to the practical, I would say that my understanding of Design of Experiment was already relatively good. I think it was quite fun getting to mess around with the catapults. Clive and I were doing the fractional factorial part of the practical so I made sure to choose the proper runs which consisted of the 2 low and 2 high values for the 3 different factors.
​
Here are some videos from the practical.
The above videos is Clive and I taking turns to launch the balls.
​
I would say that the runs for both the full and fractional factorial was a success as our results for both were similar and the significant factor for both were the same. I also definitely do see the importance of having a lot of runs as the response variable was very inconsistent for a lot of the different factor combinations so getting the average is definitely very important for cases like this.
​
I feel, overall, the group was happy because we were able to utilize the data analysis to get first place for the catapult challenge. We were able to hit 3 of 4 targets. We did not take much time when it came to testing out the how far the ball would fly because we already knew the significant factors.
​
This video is me launching the ball at Dr Ting,
​
I think this video itself shows how much I have learnt from this topic. Comparing to how I analysed the data during the first tutorial to being able to use proper data analysis to troubleshoot the catapult during the challenge to meet the requirements. It was quite fun going through this topic and I will definitely use this in the future.
If I had not gone through this topic, I think I would have been analysing data wrongly during my internship which would be extremely bad lol. I would have never thought that there was such a structured way to analyse data using tables, graphs and even an equation to calculate the number of runs required depending on the number of factors.
11 out of 10 would do it again... I mean I don't really have a choice if I want to do it again or not lol.