The Hidden facts on how to ace exams

In tests that offer a choice of answers, you can exploit hidden patterns to get full marks if you know how, says William Poundstone. 

Our fates in school and beyond are decided by pop quizzes, final exams, drivers’ exams, and professional exams. Many are true-or-false, or multiple-choice. If you get stumped by a question, it usually feels like you can do no better than guess. But could there be hidden patterns in these tests that point to the right answer?

That’s what I tried to find out. Universities often post online archives of old tests with answer keys. A staggering variety of other tests are available on the web, too. I crunched statistics on a sample of 100 tests, 34 from schools and colleges and 66 from other sources, with a total of 2,456 questions. The tests included school, college, and professional exams; actual and practice drivers’ theory tests from 10 US states; newspaper quizzes on current events, sports, and celebrities; a Cosmopolitan quiz (“50 Guy Phrases”); and safety quizzes for electricity, condoms, and food poisoning. I looked for strategies that would aid a guesser and computed how helpful they might be.

Why is it possible? Journal articles and textbooks instruct educators to vary the location of the right answer randomly, though rarely does the literature acknowledge that this is easier said than done. Human beings are poor at making truly random choices – whether it’s picking lottery numbers or playing cards.

Every test maker is different. Some worry about randomising their answers, while others never give it a thought. My aggregate data suggest, however, that you can bet on some widespread patterns even when going into a test cold.

True or false?

Let’s start with true-false exams. A teacher uses true-false tests because they’re the easiest type of test to make up and the fastest to grade. The test maker who resorts to a true-false exam is cutting corners, and from the strategic guesser’s perspective, that’s good.

Two patterns were evident. One is that “true” answers are more common than “false” ones. The average split was 56% true and 44% false.

This isn’t hard to explain. True statements come more easily to mind. Recalling a fact is quicker than inventing a falsehood. Test makers follow the path of least resistance and produce tests with an excess of trues.

The other finding is that, as expected, there is more true-false-true-false alternation than in a properly random sequence. For example, here’s the answer key to a 20-item test from a college textbook (Plummer, McGeary, Carlson’s Physical Geology, ninth edition): FTTFTFFTTFTTFTTTFTTF.

Here’s the same sequence as a series of black and white squares:

 

This is not as random as it looks. One way to judge randomness is to count how many times a correct answer (true or false) is followed by the same correct answer. This occurs just seven times out of 19 (the 20th answer has no successor).

To put it another way, the chance that the next answer will be different from the present one is 63%. That’s more than the expected 50% for a random sequence.

You won’t be guessing on every item, let’s hope. For the most part you will know the correct answers to the questions before and after the difficult ones. That permits this true-false test strategy:

– Go through the entire test, marking the answers you know, before attempting to make any guesses.
– Look at the known correct answers of the items before and after the one(s) that’s left you stumped. When both neighbouring answers are the same (both false, let’s say), guess the opposite (true).
– Should the before and after answers be different, guess true (because true answers are more likely overall).

There is a rich folklore on multiple-choice test guessing. I remember being advised to pick the centre choice. Based on my data, that tactic wouldn’t do much good. On tests with three choices (call them A, B, and C), the options were about equally likely to be correct. With four options, the second answer (B) was slightly favoured, being correct about 28% of the time. That’s compared to the expected 25% for four answers.

With five options, the last answer (E) was the most commonly correct one (23%). The middle choice (C) was the least favoured (17%).

Random struggle

It appears that test makers intuitively get the proportions right for three choices but have trouble doing so when there are more than three. This is in line with experimental findings that the quality of randomising decreases as the number of options increases.

So to recap, pick the second answer (B) on four-choice tests and the fifth answer (E) on five-choice tests.

Another surprising thing I found was that the answers “none of the above” or “all of the above” were wildly more likely to be correct. In one college textbook with four-choice questions, “none”/“all” answers were right 65% of the time! And in my total sample, a none/all answer was correct 52% of the time that there was such an answer. Assuming this is even close to being representative, it’s astonishing.

Another interesting rule of thumb alleges that the longest multiple-choice
answer is most likely to be correct. On this question from the Washington state driver’s exam, the longest answer (c) is indeed correct:

To turn right, you should be in:
a. The left lane.
b. The centre lane.
c. The lane that’s closest to the direction you want to go.
d. Any one of the lanes.

Test makers have to make sure that right answers are indisputably right. Often this demands some qualifying language. They may not try so hard with wrong answers.

Continuity errors

Still another trick is to check for what Hollywood calls continuity errors. Here’s an example from the guidelines for Brigham Young University’s faculty in the US.

A word used to describe a noun is called an:
a. Adjective.
b. Conjunction.
c. Pronoun.
d. Verb.

The hapless professor used the article an, hearing the correct answer, adjective, in his head. Then he wrote three alternatives, not stopping to think that they begin with consonants. A student could use that slip to deduce the answer.

Like true-false tests, multiple-choice tests show too much alternation.

It wasn’t too uncommon to find short tests in which no correct choice ever repeated twice in a row. The answer key played a game of hopscotch.

For example, for the three-choice tests in my sample (A,B,C), the correct choice repeated its predecessor only 25% of the time (versus the expected 33% for a random sequence). This means that a test taker could gain an easy advantage when guessing just by avoiding the previous question’s answer.

Deja vu

One final rule: Always trust your instinct. It works, even without a system.

When you’re at a complete loss, ask yourself which of the answers sounds most familiar. Correct answers are more likely to ring familiar. It may be that you were exposed to the answer once and have forgotten it. All that’s left is a faint sense of deja vu. Guess the deja vu answer.
None of these methods can replace preparation, of course. But if your hard work seems to have been in vain, you could do a lot worse than relying on your gut instincts and smarter guesses.

-bbc