An explorative analysis of user evaluation studies in information visualisation

Geoffrey Ellis; Alan Dix

doi:10.1145/1168149.1168152

An explorative analysis of user evaluation studies in information visualisation

Geoffrey Ellis^*, Alan Dix

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

82 Citations (Scopus)

Abstract

This paper presents an analysis of user studies from a review of papers describing new visualisation applications and uses these to highlight various issues related to the evaluation of visualisations. We first consider some of the reasons why the process of evaluating visualisations is so difficult. We then dissect the problem by discussing the importance of recognising the nature of experimental design, datasets and participants as well as the statistical analysis of results. We propose explorative evaluation as a method of discovering new things about visualisation techniques, which may give us a better understanding of the mechanisms of visualisations. Finally we give some practical guidance on how to do evaluation correctly.

Original language	English
Title of host publication	Proceedings of BELIV'06
Subtitle of host publication	BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference
DOIs	https://doi.org/10.1145/1168149.1168152
Publication status	Published - 23 May 2006
Externally published	Yes
Event	BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference - Venice, Italy Duration: 23 May 2006 → 23 May 2006

Publication series

Name	Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference

Conference

Conference	BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference
Country/Territory	Italy
City	Venice
Period	23/05/06 → 23/05/06

Keywords

Case study
Evaluation
Explorative evaluation
Information visualisation

Access to Document

10.1145/1168149.1168152

Cite this

Ellis, G., & Dix, A. (2006). An explorative analysis of user evaluation studies in information visualisation. In Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference Article 1168152 (Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference). https://doi.org/10.1145/1168149.1168152

Ellis, Geoffrey ; Dix, Alan. / An explorative analysis of user evaluation studies in information visualisation. Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference. 2006. (Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference).

@inproceedings{961b2619e4074bc3a8c30adc48fb4fa8,

title = "An explorative analysis of user evaluation studies in information visualisation",

abstract = "This paper presents an analysis of user studies from a review of papers describing new visualisation applications and uses these to highlight various issues related to the evaluation of visualisations. We first consider some of the reasons why the process of evaluating visualisations is so difficult. We then dissect the problem by discussing the importance of recognising the nature of experimental design, datasets and participants as well as the statistical analysis of results. We propose explorative evaluation as a method of discovering new things about visualisation techniques, which may give us a better understanding of the mechanisms of visualisations. Finally we give some practical guidance on how to do evaluation correctly.",

keywords = "Case study, Evaluation, Explorative evaluation, Information visualisation",

author = "Geoffrey Ellis and Alan Dix",

year = "2006",

month = may,

day = "23",

doi = "10.1145/1168149.1168152",

language = "English",

isbn = "1595935622",

series = "Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference",

booktitle = "Proceedings of BELIV'06",

note = "BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference ; Conference date: 23-05-2006 Through 23-05-2006",

}

Ellis, G & Dix, A 2006, An explorative analysis of user evaluation studies in information visualisation. in Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference., 1168152, Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference, BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference, Venice, Italy, 23/05/06. https://doi.org/10.1145/1168149.1168152

An explorative analysis of user evaluation studies in information visualisation. / Ellis, Geoffrey; Dix, Alan.
Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference. 2006. 1168152 (Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - An explorative analysis of user evaluation studies in information visualisation

AU - Ellis, Geoffrey

AU - Dix, Alan

PY - 2006/5/23

Y1 - 2006/5/23

N2 - This paper presents an analysis of user studies from a review of papers describing new visualisation applications and uses these to highlight various issues related to the evaluation of visualisations. We first consider some of the reasons why the process of evaluating visualisations is so difficult. We then dissect the problem by discussing the importance of recognising the nature of experimental design, datasets and participants as well as the statistical analysis of results. We propose explorative evaluation as a method of discovering new things about visualisation techniques, which may give us a better understanding of the mechanisms of visualisations. Finally we give some practical guidance on how to do evaluation correctly.

AB - This paper presents an analysis of user studies from a review of papers describing new visualisation applications and uses these to highlight various issues related to the evaluation of visualisations. We first consider some of the reasons why the process of evaluating visualisations is so difficult. We then dissect the problem by discussing the importance of recognising the nature of experimental design, datasets and participants as well as the statistical analysis of results. We propose explorative evaluation as a method of discovering new things about visualisation techniques, which may give us a better understanding of the mechanisms of visualisations. Finally we give some practical guidance on how to do evaluation correctly.

KW - Case study

KW - Evaluation

KW - Explorative evaluation

KW - Information visualisation

UR - http://www.scopus.com/inward/record.url?scp=34547225590&partnerID=8YFLogxK

U2 - 10.1145/1168149.1168152

DO - 10.1145/1168149.1168152

M3 - Conference contribution

AN - SCOPUS:34547225590

SN - 1595935622

SN - 9781595935625

T3 - Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference

BT - Proceedings of BELIV'06

T2 - BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference

Y2 - 23 May 2006 through 23 May 2006

ER -

Ellis G, Dix A. An explorative analysis of user evaluation studies in information visualisation. In Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference. 2006. 1168152. (Proceedings of BELIV'06: BEyond time and errors - novel EvaLuation methods for Information Visualization. A workshop of the AVI 2006 International Working Conference). doi: 10.1145/1168149.1168152

An explorative analysis of user evaluation studies in information visualisation

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Cite this