Data and Code for Paper – Improving Vis Design for Effective Multi-objective Decision Making

Decision-makers across many professions are often required to make multi-objective decisions over increasingly larger volumes of data with several competing criteria. Data visualization is a powerful tool for exploring these complex ‘solution spaces’, but there is little research on its ability to support multi-objective decisions. In this paper, we explore the effects of visualization design and data volume on decision quality in multi-objective scenarios with complex trade-offs. We look at the impact of four common multidimensional chart types (scatter plot matrices, parallel coordinates, heat maps, radar charts), the number of options and dimensions, the ratio of number of dimensions considered to the number of dimensions shown, and participant demographics on decision time and accuracy when selecting the ‘optimal option’. As objectively evaluating the quality of multi-objective decisions and the trade-offs involved is challenging, we employ rank- and score-based accuracy metrics. Our findings show that accuracy is comparable across all four visualizations, but that it improves when users are shown less options and consider less dimensions in their decision. Similarly, considering less dimensions imparts a speed advantage, with heat maps being the fastest among the four charts types. Participants who use charts frequently were observed to perform significantly faster, suggesting that users can potentially be trained to effectively use visualizations in their decision-making.