The evaluation of assessment dimensionality is a necessary stage in the gathering of evidence to support the validity of interpretations based on a total score, particularly when assessment development and analysis is conducted within an item response theory (IRT) framework. Unidimensionality refers to the existence of one underlying measurement construct (dimension) that accounts for variation in examinee responses. Violating this assumption could severely bias item and ability parameter estimation. In this study, we employ polytomous item responses to compare two methods that have received increased attention in recent years: Rasch model analysis (Rasch, 1960/1980) and Parallel analysis (Horn, 1965) with TETRAD (Glymour, 1982), a method for evaluating assessment structure that is less well-known in the educational measurement community, though confirmatory TETRAD analysis has been developed by Bollen and Ting (1993, 1998, 2000) for identifying causal indicators.

Many methods of investigating unidimensionality are available. Two notable reviews of methods and indices of unidimensionality have been conducted within the last twenty-five years. Hattie’s (1985) review evaluated numerous standard approaches and showed that many lacked empirical support for the adequate assessment of unidimensionality.

More recently, Tate (2003) conducted a review of methods and indices employed with dichotomous items, finding that options for assessing dimensionality had expanded and improved, and that most methods perform effectively “within the limits of their associated perspectives and assumptions.” While this study highlights the TETRAD method, compared to Rasch model and Parallel analyses, there are several established methods that are well-documented and widely used. Test of Essential Dimensionality (DIMTEST) (Stout, 1987, 1990), Dimensionality Evaluation To Enumerate Contributing Traits (DETECT) (Kim, 1994; Zhang & Stout, 1999), and Hierarchical Cluster Analysis with Proximity Matrix (HCA/CCPROX) (Roussos, Stout, & Marden, 1998) are well-established examples, which are nonparametric in essence and can operate in either exploratory or confirmatory mode1. Other methods widely employed to assess dimensionality include confirmatory factor analysis through structural equation modeling (SEM), applied using programs..

Download pdf A comparison of Rasch Modeling, Parallel Analysis, and TETRAD