Simulation Results

We have conducted simulation studies to examine:

1. Type I error rates for approximate methods of testing interaction effects in factorial models
2. The relative power of the various mass univariate methods and the traditional mean amplitude approach.

This work is reported in:
Fields, E. C., & Kuperberg, G. R. (2020). Having your cake and eating it too: Flexibility and power with mass univariate statistics for ERP data. Psychophysiology, 57(2), e13468. https://doi.org/10.1111/psyp.13468

Simulations were carried out by randomly selecting from a large set of real EEG noise trials (i.e., EEG epochs with the average subtracted; see Groppe, Urbach, & Kutas, 2011b). For simulations of power, real ERP effects from studies collected in our lab were added to the ERPs formed from randomly selected noise trials.

We report two main results:

1. Consistent with previous simulation work (Anderson & Ter Braak, 2003; Still & White, 1981; Winkler, Ridgway, Webster, Smith, & Nichols, 2014), our results show that the permutation of residuals method maintains the Type I error rate at acceptable levels with realistic ERP data and sample sizes (n=24).
2. When spatial and temporal assumptions are matched, the permutation-based mass univariate approach has similar or better power than the traditional mean time window and spatial ROI approach. When assumptions are relaxed, the permutation approaches show only modest decreases in power. For widely distributed effects, the cluster mass test shows the best power properties, while the Fmax test has the best power for focal effects.

Code and materials from this work can be found at:
https://github.com/ericcfields/MUSim
https://osf.io/mktqj/