research-article

Towards crowd-based customer service: a mixed-initiative tool for managing Q&A sites

Authors:

Tiziano Piccardi,

Gregorio Convertino,

Massimo Zancanaro,

Cedric ArchambeauAuthors Info & Claims

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2014

Pages 2725 - 2734

https://doi.org/10.1145/2556288.2557202

Published: 26 April 2014 Publication History

Abstract

In this paper, we propose a mixed-initiative approach to integrate a Q&A site based on a crowd of volunteers with a standard operator-based help desk, ensuring quality of customer service. Q&A sites have emerged as an efficient way to address questions in various domains by leveraging crowd knowledge. However, they lack sufficient reliability to be the sole basis of customer service applications. We built a proof-of-concept mixed-initiative tool that helps a crowd-manager to decide if a question will get a satisfactory and timely answer by the crowd or if it should be redirected to a dedicated operator. A user experiment found that our tool reduced the participants' cognitive load and improved their performance, in terms of their precision and recall. In particular, those with higher performance benefited more than those with lower performance.

References

[1]

Agresti, A. 2002. Categorical Data Analysis. Wiley NJ.

[2]

Anderson A., Huttenlocher D., Kleinberg J., Leskovec J. (2012). Discovering value from community activity on focused question answering sites: a case study of stack overflow. In Proc. KDD '12., 850--858.

Digital Library

[3]

Asaduzzaman M., Mashiyat A.S., Roy C.K., Schneider K.A. (2013). Answering questions about unanswered questions of stack overflow. In Proc. MSR '13. 97--100.

Digital Library

[4]

Bottou L. and Bousquet O. The Tradeoffs of Large Scale Learning, Advances in Neural Information Processing Systems, 20, MIT Press, Cambridge, 2008.

[5]

Budzik, J. and Hammond, K. J. (1999). Q&A: A System for the Capture, Organization and Reuse of Expertise. In Proc. of ASIS. Learned Inf. Inc., Medford.

[6]

ColorBrew: http://colorbrewer2.org/ (08/31/2013)

[7]

De Liddo, A., Sándor A., Buckingham Shum, S. (2012). Contested Collective Intelligence. CSCW, 21 417'448.

Digital Library

[8]

Hanrahan B. V., Convertino G., and Nelson L. (2012). Modeling problem difficulty and expertise in stackoverflow. In Proc. of CSCW '12. ACM, 91--94.

Digital Library

[9]

Harper F.M, Raban D., Rafaeli S., and Konstan J.A. (2008). Predictors of answer quality in online Q&A sites. In Proc. of CHI '08. ACM, NY, USA, 865--874.

Digital Library

[10]

Harper R., Rodden T., Rogers Y., Sellen A. (2008) Being Human: Human-Computer Interaction in the Year 2020. Publisher: Microsoft Research Ltd.

[11]

Horowitz D. and Kamvar S.D. (2010). The anatomy of a large-scale social search engine. In Proceedings of WWW '10. ACM, New York, NY, USA, 431--440.

Digital Library

[12]

Horvitz, E. Principles of mixed-initiative user interfaces. In Proc. of CHI (1999), ACM, pp. 159'166.

Digital Library

[13]

http://meta.stackoverflow.com/questions/187450/understan ding-stack-overflows-unanswered-questions 09/15/2013

[14]

http://stackoverflow.com/ (09/15/2013)

[15]

Inselberg A. and Dimsdale B. (1991) Parallel Coordinates, Human-Mach. Interactive Systems, 1991.

[16]

Jubis, R. M. T. (1990) Coding effects on performance in a process control task with uniparameter and multiparameter displays. Human Factors, 32(3).

Digital Library

[17]

Li B. and King I. 2010. Routing questions to appropriate answerers in community question answering services. In Proceedings of CIKM '10.

Digital Library

[18]

Liu J., Song Y.-I. and Lin C.-Y. 2011. Competition-based user expertise score estimation. Proc. SIGIR '11

Digital Library

[19]

Liu Y., Agichtein E. (2008) You've Got Answers: Towards Personalized Models for Predicting Success in Community Question Answering. ACL: 97--100

Digital Library

[20]

Mamykina, L., Manoim, B., Mittal, M., Hripcsak, G. and Hartmann, B. (2011) Design lessons from the fastest Q&A site in the west. In Proc. of CHI '11, 2857'2866

Digital Library

[21]

Mitchell, T. Machine Learning. McGraw-Hill, 1997.

Digital Library

[22]

Montgomery, D.A., Sorkin, R.D. (1996) Observer sensitivity to element reliability in a multi-element visual display. Human Factors Volume 38 484'494.

[23]

Nowell, L.; Schulman, R.; Hix, D., Graphical encoding for information visualization: an empirical study, Information Visualization, Infovis2002, IEEE, 43--50.

Digital Library

[24]

Pirolli P and S Card. (2005). Sensemaking Processes of Intelligence Analysts and Possible Leverage Points as Identified Through Cognitive Task Analysis. In Proc. of the 2005 Int. Conf. on Intelligence Analysis, Virginia.

[25]

Pal A. F. Harper M., and Konstan J.A. (2012). Exploring Question Selection Bias to Identify Experts and Potential Experts in Community Question Answering. ACM Trans. Inf. Syst. 30, 2(10).

Digital Library

[26]

Poulin B, Eisner R, Szafron D, Lu P, Greiner R, Wishart D.S., Fyshe A (2008) Visual Explanation of Evidence in Additive Classifiers, AAAI Spring Symp.

Digital Library

[27]

Scott S., Lesh N., Klau W. (2002) Investigating HumanComputer Optimization, In Proc. of CHI 2002.

Digital Library

[28]

Shneiderman B. (1994). Dynamic Queries for Visual Information Seeking. IEEE Softw. 11, 6, 70--77.

Digital Library

[29]

Smallman, H.S. (2008): JIGSAW - Joint Intelligence Graphical Situation Awareness Web for Collaborative Intelligence Analysis.

[30]

TLX: http://humansystems.arc.nasa.gov/groups/TLX/

[31]

Treude C., Barzilay O., and Storey M.A. (2011). How do programmers ask and answer questions on the web'. In Proc. of ICSE '11. ACM, NY, USA, 804--807.

Digital Library

[32]

Witten I.H., Frank E., Hall M.A. (2011) Data Mining: Practical Machine Learning Tools and Tech,.Elsevier.

Digital Library

[33]

www.crowdengineering.com (02/01/2014)

[34]

www.lithium.com/solutions/social-support/ (02/01/2014)

[35]

Yang L., Bao S., Lin Q., Wu X., Han D., Su Z., Yu Y., (2011) Analyzing and predicting not-answered questions in community-based question answering services, In Proc. of AAAI 2011.

[36]

Zhang J., Ackerman M.S. and Admic L. (2007) Expertise Networks in Online Communities: Structure and Algorithms. In Proc. WWW 2007, Banff, Canada.

Digital Library

[37]

Zhou T.C, Lyu M. T, King I. 2012. A classification-based approach to question routing in community question answering. In WWW '12 Companion.

Digital Library

[38]

Zhou Y., Cong G., Cui B., Jensen C.S., and Yao J. 2009. Routing Questions to the Right Users in Online Communities. In Proc. of IEEE ICDE '09.

Digital Library

Cited By

Hwang SKim BLee K(2019)A Data-Driven Design Framework for Customer Service ChatbotDesign, User Experience, and Usability. Design Philosophy and Theory10.1007/978-3-030-23570-3_17(222-236)Online publication date: 3-Jul-2019
https://doi.org/10.1007/978-3-030-23570-3_17
Xu ALiu ZGuo YSinha VAkkiraju RMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)A New Chatbot for Customer Service on Social MediaProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025496(3506-3510)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025496
Srba IBielikova M(2016)A Comprehensive Survey and Classification of Approaches for Community Question AnsweringACM Transactions on the Web10.1145/293468710:3(1-63)Online publication date: 16-Aug-2016
https://dl.acm.org/doi/10.1145/2934687
Show More Cited By

Index Terms

Towards crowd-based customer service: a mixed-initiative tool for managing Q&A sites
1. Human-centered computing
  1. Collaborative and social computing
2. Information systems
  1. Information retrieval
  2. Information storage systems

Recommendations

Speaking with the crowd
UIST Adjunct Proceedings '12: Adjunct proceedings of the 25th annual ACM symposium on User interface software and technology

Automated systems are not yet able to engage in a robust dialogue with users due the complexity and ambiguity of natural language. However, humans can easily converse with one another and maintain a shared history of past interactions. In this paper, we ...
Read More
A Customer Management Dilemma: When Is It Profitable to Reward One's Own Customers?

This study attempts to answer a basic customer management dilemma facing firms: when should the firm use behavior-based pricing BBP to discriminate between its own and competitors' customers in a competitive market? If BBP is profitable, when should the ...
Read More
Modus Operandi of Crowd Workers: The Invisible Role of Microtask Work Environments

The ubiquity of the Internet and the widespread proliferation of electronic devices has resulted in flourishing microtask crowdsourcing marketplaces, such as Amazon MTurk. An aspect that has remained largely invisible in microtask crowdsourcing is that ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2014

4206 pages

ISBN:9781450324731

DOI:10.1145/2556288

General Chairs:
Matt Jones
Swansea University, Wales, UK
,
Philippe Palanque
Université Paul Sabatier, France
,
Program Chairs:
Albrecht Schmidt
University of Stuttgart, Germany
,
Tovi Grossman
Autodesk Research, Canada

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '14

Sponsor:

SIGCHI

CHI '14: CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2014

Ontario, Toronto, Canada

Acceptance Rates

CHI '14 Paper Acceptance Rate 465 of 2,043 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI PLAY '24

Sponsor:
sigchi

The Annual Symposium on Computer-Human Interaction in Play

October 14 - 17, 2024

Tampere , Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
450
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)6

Other Metrics

View Author Metrics

Citations

Cited By

Hwang SKim BLee K(2019)A Data-Driven Design Framework for Customer Service ChatbotDesign, User Experience, and Usability. Design Philosophy and Theory10.1007/978-3-030-23570-3_17(222-236)Online publication date: 3-Jul-2019
https://doi.org/10.1007/978-3-030-23570-3_17
Xu ALiu ZGuo YSinha VAkkiraju RMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)A New Chatbot for Customer Service on Social MediaProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025496(3506-3510)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025496
Srba IBielikova M(2016)A Comprehensive Survey and Classification of Approaches for Community Question AnsweringACM Transactions on the Web10.1145/293468710:3(1-63)Online publication date: 16-Aug-2016
https://dl.acm.org/doi/10.1145/2934687
Candello H(2016)User Methods and Approaches to Design Cognitive SystemsDesign, User Experience, and Usability: Design Thinking and Methods10.1007/978-3-319-40409-7_23(231-242)Online publication date: 22-Jun-2016
https://doi.org/10.1007/978-3-319-40409-7_23
Srba IBielikova M(2016)Design of CQA Systems for Flexible and Scalable Deployment and EvaluationWeb Engineering10.1007/978-3-319-38791-8_30(439-447)Online publication date: 25-May-2016
https://doi.org/10.1007/978-3-319-38791-8_30

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents