The structured interview: Manipulating structuring criteria and the effects on validity, reliability, and practicality
Description
Whereas the superior reliability and validity of structured interviews over unstructured interviews has been consistently indicated (e.g., Wright, Lichtenfels, & Pursell, 1989), the semi-structured interview has not been formally examined. Depending on its structure, the semi-structured interview's reliability and validity will vary. Defined dimensions should enhance validity, whereas standard questions and benchmark responses should increase reliability. The semi-structured interview could be equally reliable or valid as the structured interview and with its optimal structure, the most practical. The interrater reliability, predictive validity, and practicality of structured, semi-structured, and unstructured interviews were examined in a medical school setting involving assessing non-cognitive and cognitive skills of medical school students Three interview formats were developed to predict performance of 93 medical students during their clinical clerkship year. Staff, residents, and advanced students acted as interviewers. Structured interviewers used defined dimensions, standard and situational questions, and benchmark responses. Semi-structured interviewers used defined dimensions only. Unstructured interviewers received only the dimensional titles. Each student was assigned to one of the formats and interviewed twice. Also, students and interviewers evaluated the practicality of the interviews. Criteria were constructed using evaluations of the students' clerkship performance and written exams The structured interview had the best interrater reliability over six scales (for a composite of two interviewers, mean BARS $r\sb{tt}$ =.61; mean graphic scale $r\sb{tt}$ =.54), compared to the semi-structured (mean $r\sb{tt}$ =.27) and unstructured (mean $r\sb{tt}$ =.09) interviews. However, the structured and semi-structured interviews had nearly equal significant mean predictive validities over six scales for various criteria: an academic composite, a non-academic composite, and a summary clerkship rating (coefficients ranged from.40s to low.50s). The unstructured interview had little predictive validity. Hypotheses regarding the interrelationships among cognitive and non-cognitive predictor variables and academic and non-academic criterion variables, respectively, were not supported. Various theories were discussed as accounting for the results, including common method variance, multi-factored variables, and hidden biases of interviewers and raters toward knowledge measures. The structured interview was perceived as the most practical by interviewers and students. Implications for the use of different interview formats in medical school settings were discussed