 |
MSTGen: Windows Application that Simulates CAT Administration Version 1.2 |
Kyung (Chris) T. Han |
|
Preferred Citation: Han, K. T. (2013). MSTGen: simulated data generator for multistage testing. Applied Psychological Measurement, 37(8), 666-668. |
about MSTGen |
Multistage testing, or MST, was developed as an alternative to computerized adaptive testing (CAT) for applications in which it is preferable to administer a test at the level of item sets (i.e., modules). As with CAT, the simulation technique in MST plays a critical role in the development and maintainance of tests. Theoretically, MST is a special case of CAT (likewise, CAT also can be viewed as a special version of MST). Technically, however, MST and CAT, are completely different relative to how test systems work; thus existing commercial or noncommercial CAT simulation programs, for example, CATSim (Weiss & Guyer, 2012) and SimulCAT (Han, 2012), cannot accommodate MST-based tests. MSTGen, a new MST simulation software tool, was developed to serve various purposes ranging from fundamental MST research to technical MST program evaluations. The new CAT simulation software tool supports both traditional MST functioning (MST by routing to preassembled modules after each stage; Luecht & Nungester, 1998) and new MST methods (e.g., MST by shaping a module for each stage; Han & Guo, 2012). It offers a variety of test administration environments and a user-friendly graphical interface.
MSTGen supports different modes of MST
Two different modes for MST are supported in MSTGen. The first mode is the typical, traditional MST in which examinees are routed to one of several preassembled test modules based on their previous responses (Luecht & Nungester, 1998). Users have three different module selection criteria to choose from (the maximum Fisher information, minimum average difficulty difference, and random selection) and can employ sets of multiple parallel modules (i.e., panels) for test exposure control. MSTGen supports up to 990 stages and modules with few limits in the number of items. The second mode is designed for the new MST approach, proposed by Han and Guo (2012), which shapes an item module for each stage on the fly, based on test information function targets. The new MST mode accomplishes item exposure control and content balancing within the module shaping process.
MSTGen simulates various testing environment.
MSTGen supports various test administration options to create test environments that are as close as possible to live testing situations. First, the interim and final score estimates can be calculated using the maximum likelihood (ML), Bayesian maximum a posteriori (MAP), Bayes expected a posteriori (EAP) estimations, or any combination of those. Software users also can set the initial score value, range of score estimates, and restriction in estimate change. Within MSTGen, the number of test takers who are administered simultaneously at each test time slot and the frequency of communication between a test server and client computers (i.e., terminals) can also be conditioned according to the user’s choice.
MSTGen has an intuitive graphical user interface
As a Windows-based application, MSTGen provides a user-friendly graphical interface. Most features of MSTGen can be accessed by just a few simple point-and-click movements. The main interface of MSTGen largely retains the theme of earlier test simulation software tools that the author developed—WinGen (Han, 2007) and SimulCAT (Han, 2012), both of which are widely used in the field. The main interface consists of three easy-to-follow steps: Examinee/Item Data, Module Assembly, and Test Administration.
MSTGen provides powerful research tools.
MSTGen can read user-specified existing data and can generate new data sets as well. Many input and output file formats remain the same as those used in WinGen and SimulCAT . Score distribution can be drawn from a normal, uniform, or beta distribution, and item parameters for an item pool can be generated from normal, uniform, and/or lognormal distributions. The MSTGen tool also offers several graphical analysis tools such as distribution density functions, item response functions, and information functions at both item and pool levels. MSTGen can generate reports on item pool usage and test administrations. For more advanced research, MSTGen provides users with options to input differential item functioning (DIF) or item parameter drift (IPD) information as well as preexisting item exposure data. The software tool also supports the use of syntax files and a cue file for massive simulation studies.
System Requirements, Availability, and Distribution
MSTGen runs on a Microsoft Windows-based operating system with .NET framework 2.0 or higher. Microsoft’s Windows Vista and later editions include the .NET framework, but a machine running an older version of the Windows OS will first need to have .NET framework installed. The software package, a copy of the manual in PDF format, and sample files can be found and downloaded at the following web site: http://www.hantest.net. The software package is free of charge and may be distributed to others without the author’s permission for noncommercial uses only. MSTGen always checks for the latest version and automatically updates itself as long as it is running on a machine with an active Internet connection.
Downloads |
Note: SimulCAT has been developed for Microsoft Windows Vista (32bit or 64bit) or later version (ex. Windows 7). If your system is Windows XP Family and you have been never installed Microsoft .NET framework 2.0 on your system before, it is necessary to install .NET framework 2.0 first (Download Here).
If you are not sure if your system has .NET framework, just run ’setup.exe’ file in simulCAT.zip. The installation program will automatically check your system and download .NET framework from Microsoft website if your system does not have it. However, it could take up to an hour depending on computer.
MSTGen Version 3.2.5 (November. 4, 2019)
Download (MSTGen_Han.zip)
Size: 2,036KB
MSTGen User’s Manual
Download (MSTGen_Manual.pdf)
MSTGen Examples (input/output files)
Download (MSTGen_exampleFiles.zip)
WinGen2 |
IRTEQ |
Last updated:January. 23, 2021
Created by Kyung (Chris) T. Han