check in real life

check in real life

At the heart of some quick assessment methods is an interesting principle called the wisdom of crowds, also known as collective intelligence.

The principle is described in detail in the book “The Wisdom of Crowds” by James Surwicki. It examines numerous studies and examples that confirm the effectiveness of the wisdom of the crowd in various areas – from financial markets and scientific research to political decisions and public events. According to this principle, the collective opinion of a group of people can be more accurate and effective than the opinion of individual experts. In this way, a wider range of information, knowledge and experience can be taken into account, which leads to better and more accurate forecasts and decisions.

Trust but verify

I decided to test the effect of the method in real life. We recently visited the Church of Saint Elias in Protaras, Cyprus. If you are there, be sure to visit it, it is already 600 years old, it is beautiful there.

Saint Elias Church (Protaras, Cyprus)

The peculiarity of the church is that it is located on a hill and you have to climb a lot of stairs to get to it.

Stairs

We had a group the size of one product team -10 people. The composition happened to be diverse, children, parents and grandmothers, that is, the team consisted of juniors, middles and seniors. No one had previously read guidebooks and reviews, and did not know any information about the church. The task was to count the number of stairs. In general, it is not difficult, everyone had relevant experience. The evaluations were very different, but, in my opinion, the method showed itself very well.

I will present the results in the table:

Evaluation results

Results:

  1. “Senior” gave the most accurate result. It’s cool to have a senior on your team, but not all teams are lucky. And here the main thing is not to decide that next time you just need to ask the most experienced, he can also be wrong. It is necessary to try somehow on some other examples.

  2. The estimation error was only 7%. Quite good, considering the speed of evaluation and the fact that we had a new team, in this composition we gathered for the first time. I think everyone’s familiarity with the subject area played a role. And even if you remove the “senior” score, the result remains almost the same.

  3. The largest spread was found among middlemen. Apparently, someone had more experience, and they remortgage just in case. Someone is too optimistic, but in any case it is interesting to discuss it in the team, although we did not)

  4. The junas were most upset by the wrong assessment, because prizes were awarded for the correct assessment, but the seigneur shared the reward with the junas.

Conclusions

The principle works, it turned out very close to what happened when evaluating product tasks, and the following recommendations can be derived:

  • Experience and familiarity with the subject area greatly increase the accuracy of the assessment, but even without this, the method can produce excellent results.

  • People with different experience can evaluate the same tasks completely differently, the evaluations should be discussed, gradually the accuracy will increase.

  • If the estimates turned out to be incorrect, do not be upset about it. Evaluation is not an end in itself.

I continue to use it. 🙂

ps If you have read to the end, you will probably like my tg channel Inspired Product Manager, where I look for non-standard ways of conveying information, dilute it with my experience and vivid examples! Subscribe 🔥

Related posts