Test-wiseness

From Citizendium, the Citizens' Compendium
Jump to: navigation, search
This article is developed but not approved.
Main Article
Talk
Definition [?]
Related Articles  [?]
Bibliography  [?]
External Links  [?]
Citable Version  [?]
 
This editable Main Article is under development and not meant to be cited; by editing it you can help to improve it towards a future approved, citable version. These unapproved articles are subject to a disclaimer.

If you are designing a multiple-choice or true-false quiz to assess students' knowledge and mastery of some material, there is a chance that part of what you are assessing is not knowledge of the content but rather how good students are at taking tests (test-wiseness). Basically, students can guess the correct answer or narrow down the options based on things such as wording and grammar (secondary cues[1]) rather than knowledge of the topic being assessed.

For example, in answering the nonsensical multiple-choice question below, you can infer the correct answer by matching the end of the question stem ("an") with the beginning of the choices. The correct answer must be the one that starts with a vowel:

 A zieb always comes after an:
   a) burb
   b) kond
   c) arph
   d) rhoo

Test-wiseness Cues

Here are some criteria[1][2][3][4] that may be used to make test questions less susceptible to test-wiseness:

Categorical Exclusive
Distracters contain words such as "all", "always", or "every." If a choice contains the word "all" or or "every", a student may more easily rule it out. Try not to use those terms in the question or in the choices.
Phrase-Repeat
The correct answer contains a key sound, word, or phrase that is contained in the question’s stem.
Absurd/Implausible Answer
Distracters are unrelated to the stem. Some incorrect choices may be more obviously wrong and easier to eliminate.
Precision/Specificity
Correct answer is more precise, clear, or qualified than the distracters. It is very detailed and provides no room for ambiguity. That suggests it is the correct answer.
Length
Correct answer is longer than the distracters. Try to make each of the choices relatively equal in length.
Grammar
Distracters do not match the verb tense of the stem, or there is not a match between articles ("a", "an", "the").
Give-Away
Correct answer is given away by another item in the test. Sometimes the answer to question 2 can be found in the stem of question 1, for example.
Order of Answer
Don't use a predictable pattern for the answers (for example, making the correct answer usually be C).
Number of Options and Guessing
Out of a hundred multiple-choice items with three answer options each, a student with no knowledge of the content will get 33 correct simply by guessing blindly. However, the more choices you add, the more susceptible the question becomes to test-wiseness, hence four items is most common[5].
Odd Man Out
If you make two or more choices difficult to distinguish from one another conceptually, that usually helps you rule them out. If three answers are very similar and one is very different, the different one may likely be the correct answer.
Spelling
Another one mentioned is that test authors are more likely to make spelling errors on incorrect choices, since they mainly review the correct answers.

Examples

  1. Robert Runt has a webpage[3] with tips for students on taking tests. At the bottom of the page is a sample multiple-choice quiz illustrating many of the above principles.
  2. Gibb's 1964 dissertation[1] contains a 70 item Experimental Test of Testwiseness.
  3. Mahamed et al.[4] created a 20 item test-wiseness questionnaire for pharmaceutical students.


History

Thorndike (1951)[6] first conceptualized "test wiseness" as a variable that can affect test scores. Thorndike viewed test wiseness as a general and lasting characteristic of the individual and a prominent source of score variance on cognitive tests second in its influence only to in­dividual cognitive ability. More specifically, Millman, Bishop, and Ebel ( 1965 , p. 707)[7] defined test wiseness as the "subject's capacity to utilize the characteristics and formats of the test and/or the test taking situation to receive a high score. Test wiseness is logically indepen­dent of the examinee's knowledge of the subject matter for which the items are supposedly measures."[8]

References

  1. 1.0 1.1 1.2 Gibb B.G. (1964). Test-wiseness as secondary cue response. Doctoral Dissertation, Stanford University.
  2. Allen, Kirk; Stone, A., Rhoads, T.R., & Murphy, T.J. (2004). "The Statistics Concepts Inventory: Developing a Valid and Reliable Instrument" (PDF). Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition. Retrieved on 2007-12-01.
  3. 3.0 3.1 Runt, Robert. Tips for Taking Multiple Choice Tests. Retrieved on 2007-12-01.
  4. 4.0 4.1 Mahamed, Anisah; Gregory, P.A.M., & Austin, Z. (December 2006). ""Testwiseness" Among International Pharmacy Graduates and Canadian Senior Pharmacy Students". American Journal of Pharmaceutical Education 70 (6): 131. Retrieved on 2007-12-01.
  5. Kuntz, Patricia (March 1982). "Test-Wiseness Cues in the Options of Mathematics Items". Annual Meeting of the American Educational Research Association (66th, New York, NY). Retrieved on 2007-12-01.
  6. Thorndike, R.L. (1951) Reliability. In Lindquist, E.F. (ed.), Educational Measurement. ACE, Washington DC, pp. 560–620.
  7. Millman J., Bishop, H., & Ebel, R. An analysis of test-wiseness. Educational and Psychological Measurement, 1965, 25, 707-726.
  8. Borrello, Gloria M.; Thompson, B. (1985). "Correlates of Selected Test-Wiseness Skills". Journal of Experimental Education 53. Retrieved on 2007-12-01.