• 期刊

Nonparametric Adaptive Distribution-Free Procedure for Crossover Design With Repeated Measures

Motivations: The purpose of this research is to develop adaptive nonparametric statistical methods for 2 × 2 cross over designs with repeated measures. Methods: The statistics and their asymptotic distributions are established for testing: (1) the equality for carry-over effects; (2) direct treatment effect when carry-over effects are equal; (3) equality of carry-over effect over time; (4) equality of direct treatment effect over time when carry-over effect over time is equal; (5) the average response for carry-over effects; and (6) the average response for direct treatment effects when average response for carry-over effects are equal. Findings: The methods are applied to the slopes of systolic and diastolic pressures in the study regarding to sleep apnea severity. Implications: Using the sleep apnea severity study as an example, we show that adaptive procedures had advantage of detecting the significance of test while the ranks-only based methods by Johnson and Grender (1993) failed to detect such significant result.

• 期刊

Revising and Evaluating the Expected Likelihood Ratio as a New Item Selection Strategy in Unidimensional Computerized Classification Tests

Purpose: The item selection strategy and termination rule are two crucial factors that determine the measurement efficiency of computerized classification tests (CCTs). The expected likelihood ratio (ELR) strategy combined with the sequential probability ratio test (SPRT) has been proposed to promote the efficiency of CCTs. However, the SPRT only compares two simple hypotheses, whereas the generalized likelihood ratio (GLR) can consider a range of trait levels along the latent continuum. It is a straightforward suggestion that the ELR should also be combined with the GLR in CCT contexts. Because the GLR technique divides the likelihood ratio into two conditions in which its calculation depends on the relative position of the latent trait estimates and boundaries of the indifference region, the ELR is revised accordingly in this study. Design: The performance of this newly proposed ELR strategy in terms of the probability of correct classification (PCC) and average test length (ATL) in CCT contexts was investigated through a series of simulation studies. Findings: The results showed that the revised ELR combined with GLR introduced better measurement efficiency than the two other combinations of methods. The conclusions and implications of this study are discussed.

• 期刊

A Comparison of Pre-Knowledge Detection Procedures With Response Time Modeling

Purpose: This study investigated the effectiveness of two Bayesian procedures in identifying test-takers and items with response time (RT) patterns indicating item pre-knowledge. Design: Various proportions of test-takers with pre-knowledge and items affected by pre-knowledge were manipulated in simulation data to find out: (1) which of the two methods produced lower type-I error, (2) which of the two methods successfully detected items affected by pre-knowledge, and (3) which of the two methods successfully detected test-takers with pre-knowledge. Findings: The results show that the person-fit method could accurately detect items whose RTs are not lognormally distributed but had low detection rates to identify persons with item pre-knowledge. The postpredict method showed high type-I error rates but good detection power when testing for extreme RT patterns for persons. Practical implications: The analyses indicate that the postpredict method is more promising that the person-fit method in detecting test-takers with pre-knowledge of item content. The cross-validated residuals utilized by the postpredict method could effectively alleviate the influence of the biases when detecting RT patterns indicating pre-knowledge. Value: The results from this study not only give lights to which method is more effective in detecting possible item breach, but also provide information about when the detecting methods are most effective.

• 期刊

An Exploration of an Integrated Approach for Enemy Item Identification

Purpose: This study explores an integrated approach to identify enemy items in item bank management in a medical licensure examination. Method: The integrated approach utilizes item bank analysis, natural language processing methods by using Cosine Similarity Index, and content analysis and review by subject matter experts. Results: Results from an empirical study indicate that the integrated approach is efficient in identifying enemy items. Content review of the flagged enemy item pairs is a necessary step to confirm enemy items.