Crowdtesting

Crowdtesting is part of the croudsourcing concept. Its purpose is to use the crowd as tester for applications, softwares, etc. instesd of a test team.

Definition

Crowdtesting is a manner of managing a test. The crowd is used to test a software, a product, an application, etc. It has appeared as an opposing option to the well-known test team usually trained to that purpose. Crowdtesting achieves results that are generally much less focused that a test team. Always depending on the project, such an approach can be extremely useful. A crowd can cover much more development scenari and patterns of use.

Poeple involved

All this process is realized to satisfy the client who will in the end use the product. In order to solve every possible problems that the product might have, the producer hires someone who organizes a test (organizer). After that the crowd has to play its role by using the product and giving feedbacks. In the end, the results have to be analyzed and the product corrected in order to be as optimal as possible.

Process

The first step consists of the definition of the project, and its analysis, so that it is clear that crowdtesting is better adapted than a classical test team. After that, it is necessary to define the questions that will lead to the discovery of the possible bugs of the product. The test is ready and the crowd enters the process by using the product. People have to document their use, so that when a problem is found, it is possible to know exactly how to reproduce it and solve it. The documentation (texts, images, screanshots, etc.) is extremely important since the testers are not taking the test under the surveillance of the organizer, but at home or in the train or anywhere else. After that, all the results have to be collected and analyzed. The last step is to solve all the problems found.

Each project implies a different way of managing each step, since the product can have variable purposes.

Once the product is on the market, it is almost impossible to correct the problems and still be successful!

Money

There are different option in managing the money :

• “Pay-per-bug” is the most used option and the person taking the test only gets money if founding a significant problem and proving that it exists.
• Another option is to be paid a certain amount of money for taking the test and in addition to that be paid again if finding a significant problem.
• The last option is to be paid per hour spent taking the test and in addition to that be paid again if finding a significant problem.

Motivations

The main motivation in participating in a crowdtesting is the money. There are other influent factors, but they are far less mentioned than money : competence, altruism, fun, challenge oneself, etc.

Benefits

The benefits of crowdtesting are for instance coverage, reduction of costs, availability, speed, impartiality, diversity, real conditions.

• Coverage is a very important factor since an extremely large amount of devices is used as well as system configurations, whereas a test team could not cover such a large diversity of devices and therefore notice all the possible problems.

• Reduction of costs also plays an important role for the enterprises that organize a test and are looking for results. As descibed above, there are three different ways to manage the money but the one that is most used is "pay-per-bug" which is often applied regardless of the importance of the bug.

• Availability concerns the time as well as the location. Indeed, a test taken by the crowd runs at any time of the days because it involves people from all over the world.

• Speed is a result of the availybility but also of the fact that the amount of people is much larger than the one of a test team.

• Impartiability results of the fact that a crowd consists of people whose education, interests, knowledges, etc. are indefenitely variable.

• Diversity concerns as well the people as the devices. The crowd may help the developping process of a product by enlightening a more intuitive manner of using it.

• Real conditions refers to the fact that these products are created for the people, so using the people to test it is as close as possible to the real situation in which it is going to be after its release.

Risks

The risks or challenges in crowdtesting are for example the lack of ownership, the lack of control, the lack of measurement, the reward concept, the communication with the crowd, the security, the continuous involvement, etc

Examples

Ebay, Amazon, Microsoft, Google, Facebook, etc.