Showing 1–3 of 3 results for author: Georgila, K

Report from the NSF Future Directions Workshop on Automatic Evaluation of Dialog: Research Directions and Challenges

Authors: Shikib Mehri, Jinho Choi, Luis Fernando D'Haro, Jan Deriu, Maxine Eskenazi, Milica Gasic, Kallirroi Georgila, Dilek Hakkani-Tur, Zekang Li, Verena Rieser, Samira Shaikh, David Traum, Yi-Ting Yeh, Zhou Yu, Yizhe Zhang, Chen Zhang

Abstract: This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggested promising directions for future work in this important and very rapidly changing area of research. This is a report on the NSF Future Directions Workshop on Automatic Evaluation of Dialog. The workshop explored the current state of the art along with its limitations and suggested promising directions for future work in this important and very rapidly changing area of research. △ Less

Submitted 18 March, 2022; originally announced March 2022.

Comments: Report from the NSF AED Workshop (http://dialrc.org/AED/)

Towards Understanding End-of-trip Instructions in a Taxi Ride Scenario

Authors: Deepthi Karkada, Ramesh Manuvinakurike, Kallirroi Georgila

Abstract: We introduce a dataset containing human-authored descriptions of target locations in an "end-of-trip in a taxi ride" scenario. We describe our data collection method and a novel annotation scheme that supports understanding of such descriptions of target locations. Our dataset contains target location descriptions for both synthetic and real-world images as well as visual annotations (ground truth… ▽ More We introduce a dataset containing human-authored descriptions of target locations in an "end-of-trip in a taxi ride" scenario. We describe our data collection method and a novel annotation scheme that supports understanding of such descriptions of target locations. Our dataset contains target location descriptions for both synthetic and real-world images as well as visual annotations (ground truth labels, dimensions of vehicles and objects, coordinates of the target location,distance and direction of the target location from vehicles and objects) that can be used in various visual and language tasks. We also perform a pilot experiment on how the corpus could be applied to visual reference resolution in this domain. △ Less

Submitted 11 July, 2018; originally announced July 2018.

Comments: to appear in Fourteenth Joint ACL - ISO Workshop on Interoperable Semantic Annotation, Corresponding author: Ramesh Manuvinakurike

A Dialogue Annotation Scheme for Weight Management Chat using the Trans-Theoretical Model of Health Behavior Change

Authors: Ramesh Manuvinakurike, Sumanth Bharadwaj, Kallirroi Georgila

Abstract: In this study we collect and annotate human-human role-play dialogues in the domain of weight management. There are two roles in the conversation: the "seeker" who is looking for ways to lose weight and the "helper" who provides suggestions to help the "seeker" in their weight loss journey. The chat dialogues collected are then annotated with a novel annotation scheme inspired by a popular health… ▽ More In this study we collect and annotate human-human role-play dialogues in the domain of weight management. There are two roles in the conversation: the "seeker" who is looking for ways to lose weight and the "helper" who provides suggestions to help the "seeker" in their weight loss journey. The chat dialogues collected are then annotated with a novel annotation scheme inspired by a popular health behavior change theory called "trans-theoretical model of health behavior change". We also build classifiers to automatically predict the annotation labels used in our corpus. We find that classification accuracy improves when oracle segmentations of the interlocutors' sentences are provided compared to directly classifying unsegmented sentences. △ Less

Submitted 11 July, 2018; originally announced July 2018.

Comments: to appear in Fourteenth Joint ACL - ISO Workshop on Interoperable Semantic Annotation