Ian Livingston, Lennart Nacke, and Regan Mandryk. 2010. Critic-Proofing: Robust Validation Through Data-Mining. In Proceedings of Fun and Games 2010: 81-94. Leuven, Belgium.
Critic-proofing is a modified heuristic evaluation technique, specifically designed to provide a fine-grained, prioritized list of heuristic violations. The critic-proofing technique weights the severity of a problem based on the frequency that similar problems are found in similar games. The severity ratings are calculated using data collected from game reviews, and the severity assigned to a problem during the heuristic evaluation process. However, heuristic techniques have had limited adoption within the video game industry. One reason for this is the perceived lack of validity and robustness of game specific heuristic principles. In this paper, we introduce and outline a new data- mining project designed to validate game-specific heuristic techniques, especially the critic-proofing technique by using the popular game-review aggregation website Metacritic.