There is a recent rekindled interest concerning the stability, reliability, and predictive validity of skilled performance across repeated measurements. One common phenomenon resulting from the repeated measures of subjects' performance across trials during skills acquisition is the superdiagonal correlation matrix, also known as the simplex matrix, in which correlations of performance decrease as a function of the separation of trials (or as a function of time). The present study collected fifty such correlation matrices from both published and unpublished sources. Next, the standardized item coefficient alphas were calculated from correlations for all possible combinations of adjacent trials to identify rules for which trials should be used to maximize reliability. When early or late adjacent trials showing low correlations were dropped from the computation of the standardized item coefficient alpha, reliability sometimes increased, although not dramatically. The rows of correlations above the diagonal of a superdiagonal matrix were plotted across trials and it was found that the resulting graphs could be used in deciding which adjacent early and/or late trials to drop to maximize the reliability. Seven figures, representative of the different sorts of published matrices, provide graphical decision rules for determining which trials to average to maximize reliability. The standardized coefficient alpha for the entire matrix should also be computed as a benchmark reliability
