Topic 8 Applied Product Testing
(detailed objectives) (available resources)
Goal: Students learn to design and conduct an experiment and draw analytical conclusions from the observations.
[standards: NM-DATA.6-12.1-3, NM-PROB.REP.PK-12.3, NM-PROB.COMM, PK-12.1-4]
Curriculum for our EST Pipeline
(to review the detailed content, download the low resolution pdf of available teacher presentation)
Summary
This topic presents a convenient opportunity for student to exercise an entire experimentation project (beyond simply conducting the experiment).  It is an opportunity to dispel misconceptions that science doesn't allow creative expression.  After completing this topic, students should have the skills necessary to complete the testing phase of their major class project.

Background
Like any other project, an experimentation project should have clearly defined phases.  Typically these include planning, design, construction, debugging, execution, data analysis and reporting.  An experimentalist may want to re-define these or supplement with other phases.  In this class, the student team is responsible for the entirety of the experimentation project in support of the testing phase of the class project.  To be successful you will need to be keenly aware of two fundamental principles.  First, all measurements are uncertain and have some error associated with them. Measurement errors include both bias and precision errors. When design decisions are based on experimental conclusions, the errors need to be accounted for.  Second, there are underlying assumptions to any experiment.  It is not necessary that an experimental model (or prototype) exactly replicate the final product.  However, the assumptions about similarity and the similarity of testing conditions to real-world operating conditions should be well understood.

A "Descriptive Statistic" is any single numerical or graphical representation that describes a data set.  They are simply succinct summaries of the data.  For data sets involving a single measured variable (univariate statistics), we generally want to describe a dataset using its distribution, or its central tendency, or its dispersion.  Most likely some combination of the three.  The most common measure of the frequency of individual values or the range of values is the frequency distribution (histogram).  The most common measures of the central tendency include the mean, median, and mode.  The most common measures for the spread of values around the central tendency include the range and the standard deviation.

The term "normal distribution" is used quite loosely in the presentation.  Simply speaking, if the data follows the normal distribution then there are commonly accepted descriptions of the central tendency and variance that we can easily use. For example, if we say we have a dataset that is normally distributed with a mean of 5.2 and a standard deviation of .8, then all statisticians know exactly what we are talking about.  There are many advanced tests that can be performed on the dataset to see if it truly comes from a normally distributed population.  The short story is that the data might deviate from normal in "skewness" (favoring one side or the other) or kurtosis (too sharp or flat of a peak).  If students are advanced enough to calculate the standard deviation, then they are encouraged to report their findings in terms of "confidence intervals."  As an example, if the data analysis yielded a mean of 5.2 and a standard deviation of .8, then the 95% confidence interval would be 5.2 +/- 1.96*.8   You then might conclude "based on the data we are 95% confident that the true mean is between 3.6 and 6.7."  The following values indicate the various confidence intervals (assuming that a lot of data points are taken). 
Fraction    Number of Standard 
of Data Deviations from Mean

50.0% .674
68.3 1.000
90.0 1.645
95.0 1.960
95.4 2.000
98.0 2.326
99.0 2.576
99.7 3.000
One foundational truth of experimentation is that we cannot have much confidence in the results if we don't take enough samples.  However, performing an excessive number of repetitions can waste time without producing any additional insight.  The rule of thumb is that below thirty samples is a "small sample size" ... but you can get away with   n = [1.96*St.Dev] 2 samples in the dataset (95% confidence and standard error =1).  [Expert statisticians will recognize some cheating here...but it will likely serve most pre-collegiate students well enough]

Teacher Preparation
  1. Contrive a simple design with slightly varying alternative that can be easily tested. (e.g. different length ramps to go up a hill, different type of material in a column, or different orientation of a members in a beam to support a load)
  2. Prepare simple data sets, diagrams and calculations to demonstrate basic descriptive statistics.
Classroom Activities
Begin with class discussion about uncertainty and assumptions in the context of engineering designs.  Explain the need for experimentation and apply the project taxonomy to describe an experiment plan.  Discuss common statistics using a pre-defined dataset.  Then have the students quickly gather some data as a class and calculate simple statistics for their data.  Allow the students to work in small groups to apply what they know about planning and executing an experiment.  Then have the students individually exercise the plan that their small group created and individually report the results.


Objectives
8.1 Experimentation  Fundamentals
  • Comprehend scope of experimentation beyond simply taking data
  • Comprehend why assumptions are made and why experimentation is required to verify their correctness
  • Know the basic phases of a typical experimentation project
  • Comprehend three basic principles of experimentation
Activities


Engage students with teacher presentation.
8.2 Common Statistics
  • Create a histogram from a data set
  • Interpret histograms (central tendency, spread, and outliers)
  • Define descriptive statistic, mean, range
  • Comprehend mean and range and recognize normally distributed data


Engage students with teacher presentation.
Class exercise to create and interpret a histogram.
8.3 Planning an Experimentation Project
  • Apply knowledge of phases of an experiment


Work in small groups to plan an experimentation project.
8.4 Executing an Experimentation Plan
  • Apply knowledge of phases of an experiment and common statistics


Individually conduct the experiment and collect data.
8.5 Reporting Results
  • Apply knowledge of phases of an experiment and common statistics
  • Organize data from an experiment


Individually conduct the experiment, collect data and begin writing report.
Additional resources available to licensed users:
In addition to 24 animated PowerPoint customizable slides...
Student Resources




Supplemental reading materials (printable or web linked)
Practice and Assessment Templates
Students are heavily engaged with major project...
Students report experimentation project results.
Helping students experience other skill specialties.ppt

Template for assessing learning objectives
These pages and all content Copyright 2005-2006
All rights reserved.  Use of material on this website
is subject to the curriculum license agreement.