The t-test is the first test most people are taught and the one most often run without its assumptions checked. Magic Stat runs it from the spreadsheet you already have and puts the effect size and a confidence interval next to the p-value, instead of leaving you to compute them elsewhere.
A t-test asks whether a mean differs from something: from another group's mean (two-sample), from a fixed reference value (one-sample), or from itself measured twice (paired). It is a statement about means, and it assumes the sampling distribution of that mean is roughly normal — which is usually fine with enough data and fragile with very few observations or heavy skew.
The choice that changes the answer is the variance assumption. Student's t pools the two groups' variances and is exact when they are equal and the sizes are balanced. Welch's t does not assume equal variances and is the safer default whenever the spread or the group sizes differ. Magic Stat exposes both, and reports Mann-Whitney U, Wilcoxon Rank-Sum and Brunner-Munzel next to them so the rank-based answer is one click away rather than a separate trip to another program.
Where it stops being right: more than two groups is ANOVA, and the same subjects measured twice is the paired route that lives under the repeated-measures dialog. A t-test run repeatedly over many pairs is the classic way to inflate the false-positive rate — control that with a post-hoc or planned-contrast method instead.
Parametric: Student's t-test (equal variance) · Welch's t-test (unequal variance) · One-sample t-test against μ₀.
Non-parametric: Mann-Whitney U · Wilcoxon Rank-Sum · Brunner-Munzel · One-sample Wilcoxon signed-rank.
Effect size: Cohen's d, with a 95% confidence interval by bootstrap (percentile) or normal approximation.
Correction across variables: Benjamini-Hochberg (FDR) · Bonferroni · Holm · Hochberg.
Welch's is the safer default when the two groups differ in variance or in size, because Student's t pools variances it may not be entitled to pool. Student's is the classical choice when the spreads look similar and the sizes are balanced. Both are in the dialog, so the decision is yours and reversible.
If the data are roughly normal, the t-test uses the information better and has more power. If the data are ordinal, strongly skewed, or driven by outliers, a rank-based test answers a question about distributions that may be the one you actually care about. Magic Stat reports both for two groups rather than forcing the choice.
The dialogs for independent groups and for one sample are in this window. The paired t-test fits the repeated-measures dialog, where each row is one subject and each column is one condition; there the app also reports the matched Wilcoxon signed-rank and Cohen's dz.
It reports them — Shapiro-Wilk for normality, Bartlett and Levene for homogeneity of variances — and leaves the judgment to you. Automatically switching tests based on a normality p-value is a decision with consequences that the researcher should own.
No. The analysis runs on your computer. The only signal the app sends automatically is an anonymous installation counter — no data, no names, no e-mail.