Testing Higher Level Skills

A common criticism of objective tests is they are unable to test higher level learning. Certainly there is a abundance of questions that test basic recall of facts rather than more demanding intellectual skills. However many have argued that it is possible to write questions that test a range of cognitive skills (e.g. Bull and McKenna 2001). In discussing the writing of such questions reference is usually made to Bloom's taxonomy of educational objectives . Whilst the ability to test the full range of Bloom's educational objectives might be debatable it is certainly possible to write questions that require students to think more deeply. This section provides some tips for writing questions that address each of Bloom's educational objectives and then explores different question formats that can be used to write more challenging questions.

Think about the learning outcomes for some of your teaching. Make a note of areas where you are testing some of the higher level skills as outlined by Bloom.

Work through the tips and examples below and look for ways in which you might be able to create objective questions that test these skills.

Tips for writing questions for different stages of Bloom's taxonomy

Knowledge and Comprehension

Writing questions that test Knowledge and Comprehension is relatively straight forward. Knowledge questions test basic recall of fact whilst Comprehension questions test that the students have grasped the meaning of the knowledge they are recalling. Comprehension questions will usually require students to understand the meaning of terms used in the stem before they are able to select the write answer.

Application

The testing of Application can be achieved by providing the student with a novel scenario/case study and inviting them to apply some knowledge to the scenario. Care must be taken to ensure the situation is new to the student or there is a risk that you are still testing recall.

Analysis and Synthesis

Analysis and Synthesis are harder to assess by objective testing but questions at these levels can be achieved by presenting students with data, diagrams, images, multimedia etc that require analysis before a question can be answered. A Synthesis question might require students to compare two or more pieces of information .Questions in these categories could, for example, involve students in recognising unstated assumptions, or separating useful information from irrelevant information

Evaluation

Evaluation questions require the student to make a judgement. For example in medicine a question might present the student with detailed information about a patient and requires them to make diagnosis.

Question types that can be used to test higher level skills

The following are examples of types of question that are recommended for their ability to test higher level thinking skills.

Extended matching questions

Extended matching questions is a variation of MCQ where the stem of the question usually contains a scenario followed by a long list of items each of which must be matched with a list of options. Either list of options may be longer than the other and the same answer may be required more than once. These question types are popular in medicine and if well written can test higher level skills (University of Birmingham 2004)

Example

Interpretive exercise questions

Gronlund (1998) Describes interpretive exercise questions as questions that provide a series of "test items" all based on the same introductory material. The introductory material may be text, an image, data,etc. Gronlund states that these type of questions can be used to test more complex learning outcomes for example, "the ability to recognise unstated assumptions", "the ability to identify the meaning of a term", "the ability to identify relationships" and "the ability to recognise warranted and unwarranted inferences."

Example (to follow)

Assertion/reason questions

"The assertion-reason item ...allows you to test more complicated issues and requires a higher level of learning." (CAA Centre 2002). Each item in an assertion/reason questions consists of two items and assertion and a reason that are linked together. The student is required to decide whether assertion is valid and then decide whether the reason is valid

The CAA Centre advises

" Assertion-reason tests can be used to explore cause and effect and identify relationships. When writing assertion-reason questions, keep in mind the following points:

The reason should be a free standing sentence so that it can be considered separately from the assertion.
Avoid using minor reasons. These can result in an ambiguous question.
Repeat options A-E in full for each question.
Use all five options as keys equally."