How rational are admission test questions in our universities?

Each year when the admission test result of Dhaka University is published, there is a hue and cry about the poor quality of secondary education in the country—the reason being lower pass rates in the admission tests. But if the quality of education at the secondary level was so poor, then a lot of seats would have been vacant in public universities or medical colleges. Whereas in reality, no seats are vacant.

Students are going abroad for higher studies after doing SAT, GRE, TOEFL or IELTS. They are also participating in international Math or Physics Olympiads and returning home with prizes. Then why is the pass rate so low each year? Some of the reasons could be inadequate time allowed, negative marking or multi-linked conditions for pass mark.

Time is an important factor for a valid test. There are four things a candidate has to do to answer a MCQ. He/she has to read the instructions, try to understand what the question is asking, try to figure out the answer, and fill in the correct circle. Considering these four points, along with the difficulty level of a question, the standard time allotted for a MCQ is at least one minute. If the time limit is not generous enough, candidates may be penalised for being compelled to resort to guessing (the wrong answer) and thus the merit of a candidate will not be accurately reflected. In this respect, the test would not be considered valid.

This is why in any international aptitude test comprising MCQs, at least one minute, on average, is allotted. For example, admission tests for the undergraduate programme of commerce and humanities faculty in Delhi University have a duration of two hours (120 minutes) and a student has to answer 100 MCQs. The GRE test has 25 MCQs for verbal and quantitative reasoning sections with 35 and 40 minutes, respectively.

But the DU admission tests of KA (science), KHA (humanities) or GA (business) units have inadequate time limit for students which leads to negative marking and thus an increase in the fail rate in the admission test. Ninety minutes for 120 MCQs in KA unit and 60 minutes for 100 MCQs in KHA and GA units denote that less than one minute, on average, is allotted for a MCQ in each unit which is less than the time allotted in international tests. Some of the MCQs demand at least two minutes due to extensive reading and calculation. So the total time duration is not enough for the candidates and, as a result, candidates often resort to guessing the answers. Negative marking adds fuel to the fire and this is why the fail rate is so high.

Besides, there are multi-linked conditions for qualifying in KHA and GA units. Obtaining 48 marks will not guarantee a student's success in the admission test unless and until he/she satisfies all the conditions. There is an individual pass mark for English, Bangla and General Knowledge. Securing the pass mark in one subject and not in the others means the student has failed. These multi-linked conditions drag down the pass rate in KHA and GA units of DU.

So if the cut-off point for the pass mark is determined with an invalid time limit, negative marking and multi-linked conditions, a lower pass rate is expected in an elimination test such as DU admission tests. Our assumption is corroborated by the following session-wise low pass rate seen throughout the years (GA unit): 11.19 percent, 9.97 percent, 7.27 percent, 17.56 percent, 5.52 percent, and 14.75 percent in sessions 2009-10,  2010-11, 2011-12, 2015-16,  2016-17 and 2017-18, respectively. If the results were published on a scale score maintaining the rank order as is the practice for aptitude tests (SAT, GRE, GMAT) worldwide, there would be no question of a pass or fail rate.

Now, the question is: To what extent do MCQs in DU admission tests assess the quality of learning of students? Bloom's taxonomy is "a set of three hierarchical models used to classify educational learning objectives into levels of complexity and specificity." Questions demanding analysis, evaluation, problem-solving and application skills of students belong to higher-order skills and have the scope of application of knowledge to formulate an "original" response to an "unknown" situation. Knowledge means reproducing the same learnt knowledge and thus belongs to lower-order skills—assessing the ability of memorisation.

So if a student is asked to write down only the information he can recall from textbooks, his or her lower-order skills will be assessed.

An analysis of this year's DU GA unit MCQ paper shows that almost 99 percent of MCQs in the subjects of business principle, marketing and finance merely require recalling textbook information (e.g. "Who is the innovator of functional organisation?", "Which one of the following is related to the Solar Chain Principle?", "What is meant by Jettison?", etc). There is no room for candidates to show their application or analytical ability necessary for assessing higher-order cognitive skills associated with quality learning.

It's the same case with the MCQs of KHA unit. But the quality of MCQs in KA unit is much better than that of the other two units, i.e. most of the MCQs of KA unit give room to students to formulate an original response to an unknown situation.

RL Thorndike and EP Hagen, in their book Measurement and Evaluation in Psychology and Education (fourth edition), advise against using negative instructions as negative knowledge is not as important as positive knowledge, and negative instructions cause confusion. The MCQ paper of GA unit this year has a lot of questions with negative instructions (10 questions) which led to students being confused; this is an example of not prioritising important learning outcomes. For example, knowing which accounting concept is considered while preparing the balance sheet of a firm (Q16, GA unit, 2018) is much more important than knowing which accounting concept is not relevant from an educational point of view. As it is, students are nervous during an exam. And even a fraction of a second is very important for them. So it is unfair if the nature of questions causes students to be confused. Such questions will inevitably lead to a low pass rate.

Quality education does not mean regurgitation of information from textbooks. It is much more than that. It includes the ability to think critically. Zhu Xi, the famous Chinese philosopher and idolised recluse, had said, "Education is about acquiring knowledge and skills, about development and growth, resulting in betterment of mind and heart. Education is not teaching to test; it is also not about focusing on getting ahead of others."

Robiul Kabir Chowdhury is Senior Specialist (Exam and Evaluation), Bangladesh Examination Development Unit (BEDU), Secondary Education Sector Investment Program (SESIP), Board of Intermediate and Secondary Education (BISE).