Handbook of Clinical Anesthesia

Chapter 9

Experimental Design and Statistics

Practitioners of scientific medicine must be able to read the language of science to independently assess and interpret the scientific literature and the increasing emphasis on statistical methods (Pace NL: Experimental design and statistics. In Clinical Anesthesia. Edited by Barash PG, Cullen BF, Stoelting RK, Cahalan MK, Stock MC. Philadelphia: Lippincott Williams & Wilkins, 2009, pp 192–206).

  1. Design of Research Studies

case report engenders interest and the desire to experiment but does not provide sufficient evidence to advance scientific medicine.

  1. Sampling
  2. sampleis a subset of a target population that is intended to allow the researcher to generalize the results of the small sample to the entire population. The elements of experimental design are intended to prevent and minimize the possibility of bias.
  3. The best hope for a representative sample of the population would be realized if every subject in the population had the same chance of being in the experiment (random sampling). However, most clinical anesthesia studies are limited to using patients who are available (convenience sampling).
  4. Control groupsmay be self-control or parallel control groups versus historical control groups. (Studies using historical controls are more likely than those using self-controls or parallel controls to show a benefit from a new therapy.)
  5. Random allocation of treatment groupsis helpful to avoid research bias in entering patients into specific study groups. Random allocation is most commonly accomplished by computer-generated random numbers.


  1. Blindingrefers to masking from the view of both the patient and experimenter the experimental group to which the subject has been assigned.
  2. In a single-blind study, the patient is unaware of the treatment given. (Patient expectations from a treatment could influence results.)
  3. In a double-blind study, the subject and the data collector are unaware of the treatment group. This is the best way to test a new therapy.
  4. Types of Research Design
  5. Longitudinal studiesevaluate changes over time using research subjects chosen prospectively (cohort) or retrospectively (case-control). Retrospective studies are a primary tool of epidemiology.
  6. Cross-sectional studiesevaluate changes at a certain point in time.
  7. Data and Descriptive Statistics
  8. Statistics is a method for working with sets of numbers (X and Y) and determining if the values are different. Statistical methods are necessary because there are sources of variation in any data set, including random biologic variation and measurement error. These errors make it difficult to avoid bias and to be precise.
  9. Data Structure.Properly assigning a variable to the correct data type is essential for choosing the correct statistical technique (Table 9-1).
  10. Descriptive statisticsare intended to describe the sample of numbers obtained and to characterize the population from which the sample was obtained. The two summary statistics most frequently used are the central location and spread or variability (Table 9-2).

III. Hypotheses and Parameters

  1. Hypothesis Formulation
  2. The researcher starts the work with some intuitive feel for the phenomenon to be studied (biologic hypothesis).
  3. The biologic hypothesis becomes a statistical hypothesis during research planning.
  4. Logic of Proof
  5. If sample values are sufficiently unlikely to have occurred by chance (alpha [p] < 0.05), the null hypothesis


(which assumes there is no difference) is rejected.

Table 9-1 Data Types

Data Types





Data measured with an integer-only scale

Number of teeth


Data measured with a constant scale interval

Blood pressure



Binary data



Qualitative data that cannot be ordered or ranked

Eye color
Drug category


Data are ordered, ranked, or measured without a constant scale interval

ASA physical status score
Pain score

American Society of Anesthesiologists.

  1. Because statistics deal with probabilities rather than certainties, there is a chance that decisions made concerning the null hypothesis are erroneous.
  2. A type I (alpha) erroris wrongly rejecting the null hypothesis (false-positive). The smaller the chosen alpha, the smaller the risk of a type I error.
  3. A type II (beta) erroris failing to reject the null hypothesis (false-negative). Variability in the


population increases the chance of type II error. Increasing the number of subjects (which is very important in research design for controlled clinical trials), raising the alpha value, and dealing with large differences between two conditions decrease the chances of a type II error.

Table 9-2 Descriptive Statistics

Central Location for Interval Variables
Arithmetic mean (average of the numbers in the data set)
Median (middle most number or number that divides the samples into two equal parts; not affected by very high or low numbers)
Mode (number in a sample that appears most frequently)
Spread or Variability
Standard deviation (SD; approximates the spread of the sample data; 1 SD encompasses roughly 68% of the sample and population members, and 3 SDs encompass 99%)
Confidence Intervals
Standard error of the mean (approximates the precision with which the population center is known)

Table 9-3 Information Necessary to Accept or Reject the Null Hypothesis

Confirm that experimental data conform to the assumptions of the intended statistical test.
Choose a significance level (alpha).
Calculate the test statistic.
Determine the degree(s) of freedom.
Find the critical value for the chosen alpha and the degree(s) of freedom from the appropriate theoretical probability distribution.
If the test statistic exceeds the critical value, reject the null hypothesis.
If the test statistic does not exceed the critical value, do not reject the null hypothesis.

  1. Inferential Statistics.The testing of hypotheses or significance testing is the main focus of inferential statistics (Table 9-3).
  2. Statistical Tests and Models
  3. General guidelines relate the variable type and the experimental design to the choice of statistical test (Table 9-4).
  4. tTestThe Student's t test is used to compare the values of the means of two populations. The paired t test is used when each subject serves as his or her own control (before and after measurements in the same patient decrease variability and increase statistical power). An unpaired t test is used when measurements are taken on two groups of subjects.
  5. Analysis of Variance
  6. The most versatile approach for handling comparisons of means among more than two groups is called the analysis of variance(ANOVA).
  7. For parametric statistics (ttests and ANOVA), it is assumed that the populations follow the normal distribution.


Table 9-4 Guidelines for Which Statistical Test to Use

Variable Type

One-Sample Tests

Two-Sample Tests

Multiple-Sample T

Dichotomous or nominal

Binomial distribution

Chi-square test, Fisher's exact test

Chi-square test


Chi-square test

Chi-square test, nonparametric

Chi-square test, nonparametric

Continuous or discrete

z or t distribution

Unpaired t test, paired t test, nonparametric tests

ANOVA, nonparametric analysis of variance

ANOVA = analysis of variance.

  1. Robustness and nonparametric tests canbe used when there is concern that the populations do not follow a normal distribution.
  2. Systematic Reviews and Meta-Analyses.To answer the experimental question, data are obtained from controlled trials (usually randomized) in the medical literature rather than from newly conducted clinical trials.
  3. The American Society of Anesthesiologists has developed a process for the creation of practice parameters that include a variant form of systematic reviews.
  4. Linear Regression.Often the goal of the experiment is to predict the value of one characteristic from knowledge of another characteristic using regression analysis.
  5. Interpretation of Results
  6. Scientific studies do not end with the statistical test. (Statistical significance does not always equate with biologic relevance.)
  7. Even small, clinically unimportant differences between groups can be detected if the sample size is sufficiently large. If the sample size is small, there is a greater chance that confounding variables may explain any difference.
  8. If the experimental groups in a properly designed study are given three or more doses of a drug, the reader should expect to observe a steadily increasing or decreasing dose–response relationship.


Table 9-5 Strength of Evidence (Increasing Order) Concerning Efficacy

Case report
Retrospective study
Prospective study with historical controls
Randomized and controlled clinical trial
Series of randomized and controlled clinical trials

  1. In comparing alternative therapies, the confidence that a claim for a superior therapy is true depends on the study design (Table 9-5).
  2. Conclusions
  3. Guidelines for Reading Journal Articles
  4. Clinicians with limited time should select journal articles to read that are relevant (determined by the specifics of one's anesthetic practice) and credible (function of the merits of the research methods).
  5. Although the statistical knowledge of most physicians is limited, these skills of critical appraisal of the literature can be learned and can greatly increase the efficiency and benefit of journal reading.
  6. Statistics and Anesthesia.Understanding the principles of experimental design can prevent premature acceptance of new therapies from faulty studies.

Editors: Barash, Paul G.; Cullen, Bruce F.; Stoelting, Robert K.; Cahalan, Michael K.; Stock, M. Christine

Title: Handbook of Clinical Anesthesia, 6th Edition

Copyright ©2009 Lippincott Williams & Wilkins

> Table of Contents > Section III - Anatomy and Physiology > Chapter 10 - cardiovascular Anatomy and Physiology