turned around and entered a new estimate to the question. They didn’t have to reach consensus. We just took the average of the independent estimates, the average after discussion, and compared those averages to see if the second one was more accurate than the first. If it was, then discussion helped. Across a variety of estimation
problems, we found that we had questions for which people got better after they talked, questions for which there was no effect, and then questions for which they got much worse after discussion. We had a situation along the lines of what we wanted. Discussion had a variety of effects, and the big question for us was, “What predicts when discussion helps?”
KNOWLEDGE@WHARTON HIGH SCHOOL
Knowledge@Wharton: What did you find? What was the key factor at play?
Silver: The thing that we found that we’re most excited about is this idea of collective confidence calibration. To give you a sense of what we mean by that, calibration in general refers to an appropriate correspondence between knowledge and confidence, which is to say that if you know the answer, you’re confident, and if you don’t know the answer, you’re unconfident. We would call someone who has that property [of being confident when they are also right] well-calibrated. But in this project, we were looking at calibration across a group of individuals. To measure calibration, we asked participants to say how confident they were in their initial answer. Then we calculated for each group a collective confidence calibration score, which essentially captured the extent to which individuals who had better pre-discussion answers were
more confident and individuals who had worse pre-discussion answers were less confident. Sometimes we found that confidence and knowledge lined up that way, and we would call the group well-calibrated. On the other hand, sometimes we found groups in which individuals who had worse pre-discussion answers were actually more confident, and individuals who had better pre-discussion estimates were less confident. That would be a group we would call poorly calibrated. Then we tried to predict the likelihood that a group’s average answer would improve after discussion using these pre-discussion calibration scores.
Knowledge@Wharton: Confidence can be a minefield. Barbara, in a recent interview that we did about how noise impacts predictions, you
mentioned that overconfidence is one of the major biases that you might encounter in a group. Is there any danger for that in this context? Can too much confidence among members of the group lead to worse judgments?
Mellers: Sure, that’s always a concern. The aspect of confidence that Ike and I were interested in was the relative confidence among group members. We weren’t really interested in the level, just correlation between confidence and accuracy. Did the most accurate person also say that he or she was most confident or not? Was the most accurate person the least confident? And when the correlation was in the right direction, it was a great predictor. It was an excellent signpost of whether the average of group estimates
Concorde STC’s now available for the Cessna 510, 525C, and 680 models.
POWER UPGRADE
INCREASED CAPACITY TO MORE MAIN BATTERY UPGRADES
IMPROVED STARTING PERFORMANCE
CONCORDEBATTERY.COM 626.813.1234 | ISO 9001 + AS9100
Crafted for Quality in the U.S.A. 35
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52
