Scale work has a sequence: check reliability, look for structure, test the structure, then look at the items themselves. Magic Stat keeps the whole sequence in one dialog — reliability coefficients, exploratory and confirmatory factor analysis, item response theory and DIF — so you are not moving the same columns between four scripts.
The psychometrics module answers the ordinary questions of scale construction: is the scale internally consistent (Cronbach's alpha, McDonald's omega, split-half), how many factors does the item set actually have (exploratory factor analysis with parallel analysis), does the hypothesised structure hold (confirmatory factor analysis with fit indices), and are the items functioning as intended (item response theory).
It is not the whole of psychometrics. The IRT engine here is the unidimensional graded response model for ordered (Likert-type) items — it does not do multidimensional IRT, dichotomous 2PL/3PL models or test equating. If you need those, mirt in R is the answer. For the full range of extraction and rotation options, psych and lavaan remain the reference.
Where the module is useful is the workflow: reliability, EFA, CFA, IRT and DIF share the same data, the same variable picker and the same report, so the numbers a reviewer asks for come from one place instead of being assembled by hand.
EFA rotation: varimax · promax · oblimin · none. Extraction: principal axis · maximum likelihood.
IRT: items coded as ordinal integers (1, 2, 3, …); complete cases are used, and the dialog warns when the sample is small.
DIF: the grouping variable can be categorical (e.g., sex) or continuous (e.g., age), with FDR correction and anchor purification.
Omega is the more defensible coefficient when items differ in their loadings, because alpha assumes equal loadings. Magic Stat reports both, plus split-half, so you can show more than one and explain the choice.
The unidimensional graded response model (Samejima) for ordered items, estimated by marginal maximum likelihood. It is the standard model for Likert scales. For dichotomous 2PL/3PL or multidimensional models, use mirt in R.
Yes — the tables (reliability, loadings, fit, HTMT) and the figures are written into a report with the settings, exportable to .docx, .html or .md, so the method section can be described accurately.
For basic reliability and EFA they are quicker to open and entirely adequate. Magic Stat's difference is having reliability, CFA, IRT and DIF in one place with the same report; if you only need alpha, jamovi or JASP is a fair choice.
No. The analysis runs locally; the only automatic signal is an anonymous installation counter with no data in it.