A hierarchical non-parametric Bayesian testlet model for dual local dependence
Loading...
Date
2021
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Namibia
Abstract
The use of psychometric tests to measure the level of individuals on unobservable traits
is common in many fields and item response theory (IRT) models are usually used for
proficiency measurement. Standard IRT models assume local person and item independence and normality assumption for the true ability distribution. However, respondents
are often clustered and test items are often grouped according to sub-content measuring
the same stimuli or sub-component of the trait. This study presents a non-parametric
polytomous multilevel testlet model for simultaneously modelling person and item clustering effects. The grouping variable is assumed unknown and determined from the
data using the Dirichlet Process. The model was compared with a parametric dual
model with groups assumed to be known, the testlet model, the Generalised Partial
Credit Model and the multilevel model accounting for person dependence effects only
in terms of systematic, random and total errors in person and item parameter estimation and test information and reliability, for simulated and real life data. The
effects of ignoring dual dependence effects were evaluated for variant group, sample,
testlet size, number of response options and mixed items tests. Groups of size 5, 20
and 40 were compared for 400, 1000 and 2000 respondents. The effects of ignoring
dependence effects were compared for 6 testlets of 3, 6 and 10 items each for 3, 4
and 5 response categories were compared. Consequences of mis-specifying the slope
and proficiency distributional parameters were evaluated where the competing models
estimated item and person parameters for skewed, bimodal, normal and uniformly distributed traits and constant and stochastic slopes. Three dependency conditions (0,
xi
none; 0.5, medium; 1, large), were considered for both local item dependency (LID)
and local person dependency (LPD). For each simulation study, a fully-crossed factorial
design was employed and the general linear model was employed for comparing estimation errors. Significant different means were detected by use of Cohen’s effect size,
f. In general, ignoring LPD effects resulted in increased bias and total errors in the
estimation of ability parameters while ignorance of LID negatively impacted on item
parameter estimation increasing with sample size, testlet size and number of options.
Failure to account for LID resulted in underestimation of proficiency standard errors,
thus resulting in overestimation of test information and reliability. When dual dependence effects were ignored, both item and ability parameter estimation accuracy was
reduced. The non-parameter model detected the number of groups and group membership well especially for smaller groups, increasingly with sample size, testlet size
and number of categories. However, the non-parametric models requires high computational performance Considering the consequences of ignoring random effects and the
computation efficiency required by the non-parametric model, it is recommended that
the model be used to detect dependence effects and groups, and standard IRT models
be applied for independent persons
Description
A dissertation submitted in fulfillment of the requirements for the Doctor of Philosophy in Science (Statistics)
Keywords
Psychometric tests, Bayesian testlet model, Non-parametric, Dependency, Namibia, University of Namibia, Public health medicine research areas