Independent Sample Tests

Overview

Independent sample tests compare two or more groups of data that are not related to each other (e.g., Treatment vs. Control, Male vs. Female). The goal is to determine if the groups come from populations with different parameters—typically observing differences in their means or medians.

Two-Group Comparisons

When comparing exactly two groups:

  • TTEST_IND: Independent samples t-test. Use when data is normally distributed. It comes in two flavors:
    • Student’s t-test: Assumes groups have equal variances.
    • Welch’s t-test: Does not assume equal variances (generally recommended).
  • MANNWHITNEYU: Mann-Whitney U test. A non-parametric alternative to the t-test. Tests if one distribution is stochastically greater than the other (often interpreted as a difference in medians). Robust to outliers and non-normal data.
  • KS_2SAMP: Kolmogorov-Smirnov test for two samples. Tests whether two samples are drawn from the same distribution (sensitive to differences in location, scale, and shape).

Multiple Group Comparisons (ANOVA)

When comparing three or more groups (e.g., Drug A vs. Drug B vs. Placebo):

  • F_ONEWAY: One-way ANOVA. Tests the null hypothesis that all group means are equal. If p < 0.05, at least one group mean is different. Data must be normal with equal variances.
  • KRUSKAL: Kruskal-Wallis H-test. The non-parametric equivalent of ANOVA. Tests if samples originate from the same distribution.
  • ALEXANDERGOVERN: A robust ANOVA alternative that does not assume equal variances (homoscedasticity).
Figure 1: Compare Groups: Boxplots showing three groups with different means. ANOVA (F_ONEWAY) would detect that these groups are significantly different (p < 0.001), prompting post-hoc testing to find pairwise differences.

Homogeneity of Variance

Many parametric tests (like Student’s t-test and standard ANOVA) assume that all groups have the same variance (homoscedasticity).

Native Excel Capabilities

  • T-Tests: Excel’s T.TEST function handles independent samples well, offering options for equal and unequal variance (Welch’s).
  • ANOVA: The “ANOVA: Single Factor” tool in the Analysis ToolPak performs one-way ANOVA, but it produces a static table, not a formula result that updates with data.
  • Missing Non-Parametrics: Excel has no native functions for Mann-Whitney U, Kruskal-Wallis, or Kolmogorov-Smirnov tests. Users must use complex rank formulas or VBA.
  • Missing Variance Tests: No built-in test for equality of variances (Levene/Bartlett), forcing users to assume variances are equal or just default to Welch’s t-test without verification.

Tools

Tool Description
ALEXANDERGOVERN Performs the Alexander-Govern test for equality of means across multiple independent samples with possible heterogeneity of variance.
ANDERSON_KSAMP Performs the k-sample Anderson-Darling test to determine if samples are drawn from the same population.
ANSARI Performs the Ansari-Bradley test for equal scale parameters (non-parametric) using scipy.stats.ansari.
BRUNNERMUNZEL Computes the Brunner-Munzel nonparametric test for two independent samples.
BWS_TEST Performs the Baumgartner-Weiss-Schindler test on two independent samples.
CVM_2SAMP Performs the two-sample Cramér-von Mises test using scipy.stats.cramervonmises_2samp.
DUNNETT Performs Dunnett’s test for multiple comparisons of means against a control group.
EPPS_SINGLE_2SAMP Compute the Epps-Singleton test statistic and p-value for two samples.
F_ONEWAY Performs a one-way ANOVA test for two or more independent samples.
FLIGNER Performs the Fligner-Killeen test for equality of variances across multiple samples.
FRIEDMANCHISQUARE Computes the Friedman test for repeated samples.
KRUSKAL Computes the Kruskal-Wallis H-test for independent samples.
KS_2SAMP Performs the two-sample Kolmogorov-Smirnov test for goodness of fit.
LEVENE Performs the Levene test for equality of variances across multiple samples.
MANNWHITNEYU Performs the Mann-Whitney U rank test on two independent samples using scipy.stats.mannwhitneyu.
MEDIAN_TEST Performs Mood’s median test to determine if two or more independent samples come from populations with the same median.
MOOD Perform Mood’s two-sample test for scale parameters.
POISSON_MEANS_TEST Performs the Poisson means test (E-test) to compare the means of two Poisson distributions.
RANKSUMS Computes the Wilcoxon rank-sum statistic and p-value for two independent samples.
TTEST_IND Performs the independent two-sample t-test for the means of two groups.
TTEST_IND_STATS Perform a t-test for means of two independent samples using summary statistics.