CAT 2012 introduced Linear On the Fly Testing (LOFT), a proven delivery model that improves exam validity, security and fairness.
What is LOFT?
Unlike less sophisticated test construction techniques that simply randomly assign test questions to each candidate, LOFT is a dynamic forms generation model that utilizes advanced computer algorithms and item characteristics to custom-build each exam. A randomly assembled exam can vary from one candidate to another, leading to serious concerns about fairness and validity. On the other hand, LOFT carefully balances each candidate’s exam to ensure proper content coverage and overall difficulty, enabling reliable comparisons of performance from one candidate to another.
How LOFT works?
LOFT is a proven approach to constructing high-stakes, secure examinations with a large volume of candidates. With LOFT, a test is constructed by selecting items from the bank each time an examinee is administered the exam. The number of possible forms is limited only by the size of the item bank and the requirements imposed on the construction process, such as item difficulty and number of content domains and subdomains. Because of the large number of forms, test security is less of an issue than in traditional form-based construction, where a small number of fixed forms are administered to the entire population of candidates.
Each candidate receives a unique exam, with one test question assigned from each difficulty level across all of the available content areas. The system automatically takes into consideration the number of times each item has been presented previously, and adjusts the selection routine accordingly. The ability to select items based not only on content but also on statistical parameters independent of fixed forms on which items appear and independent of a particular candidate group taking pre-specified (fixed) forms is facilitated by use of probabilistic modeling methodologies, generically referred to as Item Response Theory (IRT), rather than the Classical Test Theory (CTT) methodologies frequently used with other exam programs.
Benefits of LOFT
Unique test experience for every candidate: While it is certainly possible that two or three individuals out of about 3 lakh candidates would end up with the same exact test, it is a virtual guarantee that no one else in test center will. The exam a candidate gets is completely and totally individualized. Memorization of any exam would be extremely difficult, ensuring that legitimate, honest candidates are given a fair test experience (and that potential cheaters have no advantage).
Flexibility in scheduling: Since the inherent nature of LOFT item banks is that they need to be much larger to support this testing model, there is a substantially reduced risk of any overexposure. Since the IIMs don’t have to worry about how many people see any one exam question, they are able to sustain a few weeks long testing window without compromising item security. A longer testing window gives candidates a wider choice and more flexibility in scheduling dates, time slots and seats for their exams.
The test and scores for legitimate, honest test takers are fair: Since item exposure is controlled and minimized, concerns on validity of scores that may be due to fixed sets of items becoming known in the community of test takers is also minimized. It is a guarantee that those candidates who achieve a good score on the exam did so fairly, without prior knowledge of any question or test.
The use of IRT statistics to create dynamically-built forms for each candidate increases the measurement precision for the knowledge and skills required of the successful test taker. A deeper and closer evaluation of skill and knowledge for each individual will set apart top performers from the rest.