Prometric to open up on CAT content development

To answer critics of the recently-conducted computer-based Common Admission Test (CAT) for admission to business schools, how it was put together will be disclosed. Prometric, the American company that conducted the test, is doing this to dispel doubts on the question pattern. It will make oublic th details on how the content was developed.

The white paper may be released after the CAT results are released in February, Prometric told Business Standard. The Indian Institutes of Management (IIMs) confirmed the development.

The paper will contain appropriate psychometric analysis to demonstrate the fairness of the exam. A detailed document on the test will be released. This is expected to clear doubts in the student community regarding consistency of difficulty levels of the test this year. The IIMs and Prometric had faced criticism from various quarters regarding the question pattern.

“The development of the test adhered to a process-driven science that both the IIMs and Prometric strictly followed. CAT items were built with an input of more than 95 subject-matter experts and reviewed by IIM professors. In addition, item statistics are being evaluated to verify that all test items performed as intended,” said a spokesperson from Prometric.

This is the first time that CAT has been held in a computer-based format over a period of 11 days. Two more days remain for the test. The test had been traditionally held in a paper-pencil format. CAT scores are mandatory for students who wish to study at the seven Indian Institutes of Management (IIMs) and over 150 other B-schools in India.

Students who took the exam in the first three days alleged that those slotted for the latter days had an advantage, as they would have a better idea of the question paper pattern. Some students and CAT coaching institutes also accused the IIMs of repeating questions from earlier papers — a practice not seen in the earlier editions of the CAT exam. Despite a statement from the IIMs in December 2009 that, despite common situations, key data would be different in each paper, some apprehension continued to remain.

“Unlike the earlier CAT format, which was a single paper distributed to all students, we have gone into computerised testing that requires a new approach to generating questions. This approach is quite common in multi-form tests, such as GRE, GMAT or TOEFLs where tests are conducted on a regular basis. To test the same conceptual understanding across forms, we have generated different items that may have similar context. But the questions are different,” said Satish Deodhar, convenor of CAT.

Candidates do not carry with them the test-booklet in a computerised test, which leads some to misread the questions they see on their screens, the IIMs say. “The unaided memory is short and imprecise. Someone may try to reproduce through unaided memory and give an impression that the particular question will appear next time, and this may mislead candidates. The key data would be different. Also, one wonders if it would be in the interest of competing candidates to share information,” said Deodhar.

Around 8,000-10,000 students are expected to attempt the exam in the second phase of CAT, which will take place on January 30 and 31.