Assessing design research quality: Investigating verification and validation criteria
Year: 2013
Editor: Udo Lindemann, Srinivasan V, Yong Se Kim, Sang Won Lee, John Clarkson, Gaetano Cascini
Author: Le Dain, Marie-Anne; Blanco, Eric; Summers, Joshua David
Series: ICED
Institution: 1: Clemson University, United States of America; 2: Grenoble-INP / UJF-Grenoble 1 / CNRS, G-SCOP UMR5272 Grenoble, F-38031, France
Page(s): 183-192
ISBN: 978-1-904670-45-2
ISSN: 2220-4334
Abstract
Engineering design research spans many disparate approaches to study, from simulation to case studies. A challenge for the community is to ensure and evaluate the research quality, taking into account the variety of approaches. This paper explores four classes of evaluation criteria, truth value, applicability, consistency, and neutrality, based on interdisciplinary research literature. These are decomposed into several elemental metrics and distinguished by validation or verification type. The metrics are applied to two reported research studies (empirical and simulation) on similar topics. In the comparison, it is found that the authors of both papers explicitly addressed validation related criteria on the results of the research. However, the empirical study authors also included more details on the verification based metrics as they relate to the research process. The issue of how the verification and validation criteria can be operationalized according to the research method is identified as critical future research.
Keywords: Verification, validation, empirical research, case study, simulation, research assessment