Title: On the Generalizability of Results from Interactive Information Retrieval Research
Speaker:Professor Diane Kelly
Time: October 11th 2018 (14:00)
Venue: FIT 1-415
Abstract:
The idealized model of conducting empirical research starts with a theory, which is then used to derive one or more hypotheses, which are then evaluated using an appropriate method. In this idealized model, statistical methods are used to evaluate the hypothesis. Very often, especially in fields that do not have a strong tradition of theory-building and testing, such as interactive information retrieval, researchers deduce hypotheses from past empirical research reports. But what if these research reports cannot be trusted or generalized? What if the findings are strictly a function of the time at which the studies were conducted, or the environments in which they were conducted? What if the measures themselves produce findings that are unlikely to be observed in another study context even when the same instruments are used? Concerns about the generalizability, replicability and reproducibility of research is of growing interest to those working in many research specialties, including information retrieval. This talk focuses on one aspect of generalizability – the extent to which the findings from one research study can be used to make predictions about what will happen in another research study – and considers how different community practices with respect replicability and reproducibility might help us address result generalizability so we can begin to construct more lasting and useful theories about information search behaviors.
Bio:
Diane Kelly is Professor and Director of the School of Information Sciences at the University of Tennessee. Prior to this, she was a professor at the University of North Carolina at Chapel Hill. Her research and teaching interests are in interactive information search and retrieval, information search behavior, and research methods. She is the recipient of the 2014 ASIST Research Award, the 2013 British Computer Society’s IRSG Karen Spärck Jones Award, the 2009 ASIST/Thomson Reuters Outstanding Information Science Teacher Award and the 2007 SILS Outstanding Teacher of the Year Award. She is the current chair of ACM SIGIR, associate editor of ACM Transactions on Information Systems and serves on the editorial boards of several journals including, Information Processing & Management, and Information Retrieval Journal. Kelly received a PhD, MLS and a graduate certificate in cognitive science from Rutgers University and an undergraduate degree from the University of Alabama.