Lesson Plans: Experimentation Design and Product Testing

Topic 8 Applied Product Testing
(detailed objectives) (available resources)
Goal: Students learn to design and conduct an experiment and draw analytical conclusions from the observations.
[standards: NM-DATA.6-12.1-3, NM-PROB.REP.PK-12.3, NM-PROB.COMM, PK-12.1-4]

(to review the detailed content, download the low resolution pdf of available teacher presentation)
Summary
This topic presents a convenient opportunity for student to exercise an entire experimentation project (beyond simply conducting the experiment). It is an opportunity to dispel misconceptions that science doesn't allow creative expression. After completing this topic, students should have the skills necessary to complete the testing phase of their major class project.

Background
Like any other project, an experimentation project should have clearly defined phases. Typically these include planning, design, construction, debugging, execution, data analysis and reporting. An experimentalist may want to re-define these or supplement with other phases. In this class, the student team is responsible for the entirety of the experimentation project in support of the testing phase of the class project. To be successful you will need to be keenly aware of two fundamental principles. First, all measurements are uncertain and have some error associated with them. Measurement errors include both bias and precision errors. When design decisions are based on experimental conclusions, the errors need to be accounted for. Second, there are underlying assumptions to any experiment. It is not necessary that an experimental model (or prototype) exactly replicate the final product. However, the assumptions about similarity and the similarity of testing conditions to real-world operating conditions should be well understood.

A "Descriptive Statistic" is any single numerical or graphical representation that describes a data set. They are simply succinct summaries of the data. For data sets involving a single measured variable (univariate statistics), we generally want to describe a dataset using its distribution, or its central tendency, or its dispersion. Most likely some combination of the three. The most common measure of the frequency of individual values or the range of values is the frequency distribution (histogram). The most common measures of the central tendency include the mean, median, and mode. The most common measures for the spread of values around the central tendency include the range and the standard deviation.

The term "normal distribution" is used quite loosely in the presentation. Simply speaking, if the data follows the normal distribution then there are commonly accepted descriptions of the central tendency and variance that we can easily use. For example, if we say we have a dataset that is normally distributed with a mean of 5.2 and a standard deviation of .8, then all statisticians know exactly what we are talking about. There are many advanced tests that can be performed on the dataset to see if it truly comes from a normally distributed population. The short story is that the data might deviate from normal in "skewness" (favoring one side or the other) or kurtosis (too sharp or flat of a peak). If students are advanced enough to calculate the standard deviation, then they are encouraged to report their findings in terms of "confidence intervals." As an example, if the data analysis yielded a mean of 5.2 and a standard deviation of .8, then the 95% confidence interval would be 5.2 +/- 1.96*.8 You then might conclude "based on the data we are 95% confident that the true mean is between 3.6 and 6.7." The following values indicate the various confidence intervals (assuming that a lot of data points are taken).

Fraction    Number of Standard 
of Data    Deviations from Mean

 50.0%           .674
 68.3           1.000
 90.0           1.645
 95.0           1.960
 95.4           2.000
 98.0           2.326
 99.0           2.576
 99.7           3.000

One foundational truth of experimentation is that we cannot have much confidence in the results if we don't take enough samples. However, performing an excessive number of repetitions can waste time without producing any additional insight. The rule of thumb is that below thirty samples is a "small sample size" ... but you can get away with n = [1.96*St.Dev] ² samples in the dataset (95% confidence and standard error =1). [Expert statisticians will recognize some cheating here...but it will likely serve most pre-collegiate students well enough]

Teacher Preparation

Contrive a simple design with slightly varying alternative that can be easily tested. (e.g. different length ramps to go up a hill, different type of material in a column, or different orientation of a members in a beam to support a load)
Prepare simple data sets, diagrams and calculations to demonstrate basic descriptive statistics.

Classroom Activities
Begin with class discussion about uncertainty and assumptions in the context of engineering designs. Explain the need for experimentation and apply the project taxonomy to describe an experiment plan. Discuss common statistics using a pre-defined dataset. Then have the students quickly gather some data as a class and calculate simple statistics for their data. Allow the students to work in small groups to apply what they know about planning and executing an experiment. Then have the students individually exercise the plan that their small group created and individually report the results.

Objectives 8.1 Experimentation Fundamentals Comprehend scope of experimentation beyond simply taking data Comprehend why assumptions are made and why experimentation is required to verify their correctness Know the basic phases of a typical experimentation project Comprehend three basic principles of experimentation	Activities Engage students with teacher presentation.
8.2 Common Statistics Create a histogram from a data set Interpret histograms (central tendency, spread, and outliers) Define descriptive statistic, mean, range Comprehend mean and range and recognize normally distributed data	Engage students with teacher presentation. Class exercise to create and interpret a histogram.
8.3 Planning an Experimentation Project Apply knowledge of phases of an experiment	Work in small groups to plan an experimentation project.
8.4 Executing an Experimentation Plan Apply knowledge of phases of an experiment and common statistics	Individually conduct the experiment and collect data.
8.5 Reporting Results Apply knowledge of phases of an experiment and common statistics Organize data from an experiment	Individually conduct the experiment, collect data and begin writing report.
Additional resources available to licensed users: In addition to 24 animated PowerPoint customizable slides...
Student Resources Supplemental reading materials (printable or web linked)	Practice and Assessment Templates Students are heavily engaged with major project... Students report experimentation project results. Helping students experience other skill specialties.ppt Template for assessing learning objectives