ºÚÁϳԹÏ

Developed in-house by ºÚÁÏ³Ô¹Ï to support research and assessment programs, ºÚÁÏ³Ô¹Ï ConQuest is also to use. Version 5 brings new features like the ability to use simulation methods to test item fit, the ability to treat item parameters as random effects, and to estimate models using Markov chain Monte Carlo (MCMC) estimation.

ºÚÁÏ³Ô¹Ï ConQuest 5 is the culmination of more than 25 years of statistical software research and development. Along the way, many large-scale assessments and other research studies have benefited from ºÚÁÏ³Ô¹Ï software.

The story of ºÚÁÏ³Ô¹Ï ConQuest and its predecessor, QUEST, begins in 1983 with the arrival of lead author, . In 1983, there were three key pieces of software being used by ºÚÁϳԹÏ: Itan, an in-house piece of software developed by Lindsay Mackay and Steve Farish to conduct classical test theory (CTT) analysis; , software for fitting the Rasch model to dichotomous data; and CREDIT, developed by Geoff Masters for fitting partial credit and similar models to polytomous data. This software was run on a single DEC VAX-11 minicomputer. Analytic runs were input via a user at a terminal connected to the machine, and output was printed by a line printer.

Figure 1. Digital Equipment Corporation (DEC) VAX-11/750 minicomputer.

As ºÚÁÏ³Ô¹Ï grew with more people undertaking project work, more computing power was required and it became clear that new software development could streamline projects. This included large projects like the NSW Basic Skills Testing Program (1989) – one of the state-based assessment programs that was a precursor to the National Assessment Program (NAP). was developed in 1990 by Ray Adams and Khoo Siek Toon. QUEST brought CTT and items response theory (IRT) analysis into one application that was interactive and ran on desktop machines. This increased the number of people who would work simultaneously. QUEST also expanded the kind of models that could be fit including the and using joint maximum likelihood (JML) estimation.

As the diversity of the analytic work being undertaken increased, there was a need to fit more varied models. To do this a new piece of statistical software, ºÚÁÏ³Ô¹Ï ConQuest, was developed that could fit a highly generalised that could be parametrised to fit specific models, including the classically named models like Partial Credit Model. ºÚÁÏ³Ô¹Ï ConQuest also implemented marginal maximum likelihood (MML) estimation, allowing the drawing of plausible values – a method now central to all large-scale assessments – and could fit multidimensional items response models. ºÚÁÏ³Ô¹Ï ConQuest was first written in the FORTRAN programming language (and at that stage was called MATS: Multi Aspect Test Software) and was later re-written in C++. MATS and ConQuest was written in collaboration with at the University of California, Berkeley.

Figure 2. ºÚÁÏ³Ô¹Ï ConQuest was developed by the ºÚÁÏ³Ô¹Ï (ºÚÁϳԹÏ) in partnership with the University of California, Berkley.

The use of ºÚÁÏ³Ô¹Ï ConQuest grew further when ºÚÁÏ³Ô¹Ï partnered with the International Association for the Evaluation of Educational Achievement () and conducted the IRT scaling of the first Trends in International Mathematics and Science Study (TIMSS ). ºÚÁÏ³Ô¹Ï ConQuest 2 focused on the user experience. After being re-written by Ray Adams and Margaret Wu, ºÚÁÏ³Ô¹Ï ConQuest 2 could do user-friendly things like automatically produce design matrices by iterating over the data. ºÚÁÏ³Ô¹Ï ConQuest 2 ran on the windows OS, including windows NT, but remained compatible with VAX 32 bit machines for large jobs.

Version 3 of ºÚÁÏ³Ô¹Ï ConQuest emerged from ºÚÁϳԹϒs work on the OECD's Programme for International Student Assessment (). ºÚÁÏ³Ô¹Ï led the international consortium undertaking PISA from 1998 until 2012 and continues to conduct the Australian component of the study, including the most recent cycle in 2018. With the biggest team of analysts yet, features like a graphical user interface (GUI) and the ability to generate valid syntax through the use of point-and-click menu commands were developed to support as many users as possible. In addition, native plotting functions were added to export a range of plots including the eponymous .

ºÚÁÏ³Ô¹Ï ConQuest 4 emerged after this, and added new features, including the ability to estimate paired comparison models (Bradley-Terry-Luce (BTL) model), directly manipulate matrices, and added more CTT analysis including the estimation of Mantel–Haenszel tests of differential item functioning (DIF). Perhaps the most notable assessment program to use ºÚÁÏ³Ô¹Ï ConQuest 4 is , which uses a multidimensional item response model with latent regression model to draw Plausible Values (PVs). This expansion of features was in part conducted to support new areas of research, including enabling ºÚÁϳԹϒs Centre for Global Education Monitoring  to assist countries to report on their progress of . ºÚÁÏ³Ô¹Ï ConQuest 4 was also used to scale a new family or regional assessments, including the Pacific Islands Numeracy and Literacy Assessment, , and Monitoring Trends in Educational Growth in Afghanistan.

Despite having only just been released, ºÚÁÏ³Ô¹Ï ConQuest 5 has already supported important research. The took advantage of the ability to estimate high-dimension IRT models to test the veracity of simultaneously estimating abilities on ten outcome domains. ºÚÁÏ³Ô¹Ï is also undertaking research by applying a wider variety of flexible and descriptive models (e.g., , ). Looking ahead, a new generation of assessments will be scaled using version 5, while the next progression in ºÚÁϳԹϒs research agenda will explore the intersection of IRT and new frontiers in analysis, including machine learning.

Find out more:
For further information about ºÚÁÏ³Ô¹Ï ConQuest 5 visit www.acer.org/conquest

If you’d like to delve deeper into education data analysis, our Understanding Rasch Measurement Theory course will teach you everything you need to know. Find out more: /au/professional-learning/