-
Privacy Risks of General-Purpose AI Systems: A Foundation for Investigating Practitioner Perspectives
Authors:
Stephen Meisenbacher,
Alexandra Klymenko,
Patrick Gage Kelley,
Sai Teja Peddinti,
Kurt Thomas,
Florian Matthes
Abstract:
The rise of powerful AI models, more formally $\textit{General-Purpose AI Systems}$ (GPAIS), has led to impressive leaps in performance across a wide range of tasks. At the same time, researchers and practitioners alike have raised a number of privacy concerns, resulting in a wealth of literature covering various privacy risks and vulnerabilities of AI models. Works surveying such risks provide di…
▽ More
The rise of powerful AI models, more formally $\textit{General-Purpose AI Systems}$ (GPAIS), has led to impressive leaps in performance across a wide range of tasks. At the same time, researchers and practitioners alike have raised a number of privacy concerns, resulting in a wealth of literature covering various privacy risks and vulnerabilities of AI models. Works surveying such risks provide differing focuses, leading to disparate sets of privacy risks with no clear unifying taxonomy. We conduct a systematic review of these survey papers to provide a concise and usable overview of privacy risks in GPAIS, as well as proposed mitigation strategies. The developed privacy framework strives to unify the identified privacy risks and mitigations at a technical level that is accessible to non-experts. This serves as the basis for a practitioner-focused interview study to assess technical stakeholder perceptions of privacy risks and mitigations in GPAIS.
△ Less
Submitted 2 July, 2024;
originally announced July 2024.
-
Supporting Human Raters with the Detection of Harmful Content using Large Language Models
Authors:
Kurt Thomas,
Patrick Gage Kelley,
David Tao,
Sarah Meiklejohn,
Owen Vallis,
Shunwen Tan,
Blaž Bratanič,
Felipe Tiengo Ferreira,
Vijay Kumar Eranti,
Elie Bursztein
Abstract:
In this paper, we explore the feasibility of leveraging large language models (LLMs) to automate or otherwise assist human raters with identifying harmful content including hate speech, harassment, violent extremism, and election misinformation. Using a dataset of 50,000 comments, we demonstrate that LLMs can achieve 90% accuracy when compared to human verdicts. We explore how to best leverage the…
▽ More
In this paper, we explore the feasibility of leveraging large language models (LLMs) to automate or otherwise assist human raters with identifying harmful content including hate speech, harassment, violent extremism, and election misinformation. Using a dataset of 50,000 comments, we demonstrate that LLMs can achieve 90% accuracy when compared to human verdicts. We explore how to best leverage these capabilities, proposing five design patterns that integrate LLMs with human rating, such as pre-filtering non-violative content, detecting potential errors in human rating, or surfacing critical context to support human rating. We outline how to support all of these design patterns using a single, optimized prompt. Beyond these synthetic experiments, we share how piloting our proposed techniques in a real-world review queue yielded a 41.5% improvement in optimizing available human rater capacity, and a 9--11% increase (absolute) in precision and recall for detecting violative content.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
Understanding Help-Seeking and Help-Giving on Social Media for Image-Based Sexual Abuse
Authors:
Miranda Wei,
Sunny Consolvo,
Patrick Gage Kelley,
Tadayoshi Kohno,
Tara Matthews,
Sarah Meiklejohn,
Franziska Roesner,
Renee Shelby,
Kurt Thomas,
Rebecca Umbach
Abstract:
Image-based sexual abuse (IBSA), like other forms of technology-facilitated abuse, is a growing threat to people's digital safety. Attacks include unwanted solicitations for sexually explicit images, extorting people under threat of leaking their images, or purposefully leaking images to enact revenge or exert control. In this paper, we explore how people seek and receive help for IBSA on social m…
▽ More
Image-based sexual abuse (IBSA), like other forms of technology-facilitated abuse, is a growing threat to people's digital safety. Attacks include unwanted solicitations for sexually explicit images, extorting people under threat of leaking their images, or purposefully leaking images to enact revenge or exert control. In this paper, we explore how people seek and receive help for IBSA on social media. Specifically, we identify over 100,000 Reddit posts that engage relationship and advice communities for help related to IBSA. We draw on a stratified sample of 261 posts to qualitatively examine how various types of IBSA unfold, including the mapping of gender, relationship dynamics, and technology involvement to different types of IBSA. We also explore the support needs of victim-survivors experiencing IBSA and how communities help victim-survivors navigate their abuse through technical, emotional, and relationship advice. Finally, we highlight sociotechnical gaps in connecting victim-survivors with important care, regardless of whom they turn to for help.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
How Knowledge Workers Think Generative AI Will (Not) Transform Their Industries
Authors:
Allison Woodruff,
Renee Shelby,
Patrick Gage Kelley,
Steven Rousso-Schindler,
Jamila Smith-Loud,
Lauren Wilcox
Abstract:
Generative AI is expected to have transformative effects in multiple knowledge industries. To better understand how knowledge workers expect generative AI may affect their industries in the future, we conducted participatory research workshops for seven different industries, with a total of 54 participants across three US cities. We describe participants' expectations of generative AI's impact, in…
▽ More
Generative AI is expected to have transformative effects in multiple knowledge industries. To better understand how knowledge workers expect generative AI may affect their industries in the future, we conducted participatory research workshops for seven different industries, with a total of 54 participants across three US cities. We describe participants' expectations of generative AI's impact, including a dominant narrative that cut across the groups' discourse: participants largely envision generative AI as a tool to perform menial work, under human review. Participants do not generally anticipate the disruptive changes to knowledge industries currently projected in common media and academic narratives. Participants do however envision generative AI may amplify four social forces currently shaping their industries: deskilling, dehumanization, disconnection, and disinformation. We describe these forces, and then we provide additional detail regarding attitudes in specific knowledge industries. We conclude with a discussion of implications and research challenges for the HCI community.
△ Less
Submitted 20 March, 2024; v1 submitted 10 October, 2023;
originally announced October 2023.
-
SoK: Safer Digital-Safety Research Involving At-Risk Users
Authors:
Rosanna Bellini,
Emily Tseng,
Noel Warford,
Alaa Daffalla,
Tara Matthews,
Sunny Consolvo,
Jill Palzkill Woelfer,
Patrick Gage Kelley,
Michelle L. Mazurek,
Dana Cuomo,
Nicola Dell,
Thomas Ristenpart
Abstract:
Research involving at-risk users -- that is, users who are more likely to experience a digital attack or to be disproportionately affected when harm from such an attack occurs -- can pose significant safety challenges to both users and researchers. Nevertheless, pursuing research in computer security and privacy is crucial to understanding how to meet the digital-safety needs of at-risk users and…
▽ More
Research involving at-risk users -- that is, users who are more likely to experience a digital attack or to be disproportionately affected when harm from such an attack occurs -- can pose significant safety challenges to both users and researchers. Nevertheless, pursuing research in computer security and privacy is crucial to understanding how to meet the digital-safety needs of at-risk users and to design safer technology for all. To standardize and bolster safer research involving such users, we offer an analysis of 196 academic works to elicit 14 research risks and 36 safety practices used by a growing community of researchers. We pair this inconsistent set of reported safety practices with oral histories from 12 domain experts to contribute scaffolded and consolidated pragmatic guidance that researchers can use to plan, execute, and share safer digital-safety research involving at-risk users. We conclude by suggesting areas for future research regarding the reporting, study, and funding of at-risk user research
△ Less
Submitted 1 September, 2023;
originally announced September 2023.
-
"There's so much responsibility on users right now:" Expert Advice for Staying Safer From Hate and Harassment
Authors:
Miranda Wei,
Sunny Consolvo,
Patrick Gage Kelley,
Tadayoshi Kohno,
Franziska Roesner,
Kurt Thomas
Abstract:
Online hate and harassment poses a threat to the digital safety of people globally. In light of this risk, there is a need to equip as many people as possible with advice to stay safer online. We interviewed 24 experts to understand what threats and advice internet users should prioritize to prevent or mitigate harm. As part of this, we asked experts to evaluate 45 pieces of existing hate-and-hara…
▽ More
Online hate and harassment poses a threat to the digital safety of people globally. In light of this risk, there is a need to equip as many people as possible with advice to stay safer online. We interviewed 24 experts to understand what threats and advice internet users should prioritize to prevent or mitigate harm. As part of this, we asked experts to evaluate 45 pieces of existing hate-and-harassment-specific digital-safety advice to understand why they felt advice was viable or not. We find that experts frequently had competing perspectives for which threats and advice they would prioritize. We synthesize sources of disagreement, while also highlighting the primary threats and advice where experts concurred. Our results inform immediate efforts to protect users from online hate and harassment, as well as more expansive socio-technical efforts to establish enduring safety.
△ Less
Submitted 29 August, 2023; v1 submitted 15 February, 2023;
originally announced February 2023.
-
Practicing Information Sensibility: How Gen Z Engages with Online Information
Authors:
Amelia Hassoun,
Ian Beacock,
Sunny Consolvo,
Beth Goldberg,
Patrick Gage Kelley,
Daniel M. Russell
Abstract:
Assessing the trustworthiness of information online is complicated. Literacy-based paradigms are both widely used to help and widely critiqued. We conducted a study with 35 Gen Zers from across the U.S. to understand how they assess information online. We found that they tended to encounter -- rather than search for -- information, and that those encounters were shaped more by social motivations t…
▽ More
Assessing the trustworthiness of information online is complicated. Literacy-based paradigms are both widely used to help and widely critiqued. We conducted a study with 35 Gen Zers from across the U.S. to understand how they assess information online. We found that they tended to encounter -- rather than search for -- information, and that those encounters were shaped more by social motivations than by truth-seeking queries. For them, information processing is fundamentally a social practice. Gen Zers interpreted online information together, as aspirational members of social groups. Our participants sought information sensibility: a socially-informed awareness of the value of information encountered online. We outline key challenges they faced and practices they used to make sense of information. Our findings suggest that like their information sensibility practices, solutions and strategies to address misinformation should be embedded in social contexts online.
△ Less
Submitted 5 March, 2023; v1 submitted 17 January, 2023;
originally announced January 2023.
-
SoK: A Framework for Unifying At-Risk User Research
Authors:
Noel Warford,
Tara Matthews,
Kaitlyn Yang,
Omer Akgul,
Sunny Consolvo,
Patrick Gage Kelley,
Nathan Malkin,
Michelle L. Mazurek,
Manya Sleeper,
Kurt Thomas
Abstract:
At-risk users are people who experience elevated digital security, privacy, and safety threats because of what they do, who they are, where they are, or who they are with. In this systematization work, we present a framework for reasoning about at-risk users based on a wide-ranging meta-analysis of 85 papers. Across the varied populations that we examined (e.g., children, activists, women in devel…
▽ More
At-risk users are people who experience elevated digital security, privacy, and safety threats because of what they do, who they are, where they are, or who they are with. In this systematization work, we present a framework for reasoning about at-risk users based on a wide-ranging meta-analysis of 85 papers. Across the varied populations that we examined (e.g., children, activists, women in developing regions), we identified 10 unifying contextual risk factors--such as oppression or stigmatization and access to a sensitive resource--which augment or amplify digital-safety threats and their resulting harms. We also identified technical and non-technical practices that at-risk users adopt to attempt to protect themselves from digital-safety threats. We use this framework to discuss barriers that limit at-risk users' ability or willingness to take protective actions. We believe that the security, privacy, and human-computer interaction research and practitioner communities can use our framework to identify and shape research investments to benefit at-risk users, and to guide technology design to better support at-risk users.
△ Less
Submitted 13 December, 2021;
originally announced December 2021.
-
Designing Toxic Content Classification for a Diversity of Perspectives
Authors:
Deepak Kumar,
Patrick Gage Kelley,
Sunny Consolvo,
Joshua Mason,
Elie Bursztein,
Zakir Durumeric,
Kurt Thomas,
Michael Bailey
Abstract:
In this work, we demonstrate how existing classifiers for identifying toxic comments online fail to generalize to the diverse concerns of Internet users. We survey 17,280 participants to understand how user expectations for what constitutes toxic content differ across demographics, beliefs, and personal experiences. We find that groups historically at-risk of harassment - such as people who identi…
▽ More
In this work, we demonstrate how existing classifiers for identifying toxic comments online fail to generalize to the diverse concerns of Internet users. We survey 17,280 participants to understand how user expectations for what constitutes toxic content differ across demographics, beliefs, and personal experiences. We find that groups historically at-risk of harassment - such as people who identify as LGBTQ+ or young adults - are more likely to to flag a random comment drawn from Reddit, Twitter, or 4chan as toxic, as are people who have personally experienced harassment in the past. Based on our findings, we show how current one-size-fits-all toxicity classification algorithms, like the Perspective API from Jigsaw, can improve in accuracy by 86% on average through personalized model tuning. Ultimately, we highlight current pitfalls and new design directions that can improve the equity and efficacy of toxic content classifiers for all users.
△ Less
Submitted 4 June, 2021;
originally announced June 2021.
-
"Why wouldn't someone think of democracy as a target?": Security practices & challenges of people involved with U.S. political campaigns
Authors:
Sunny Consolvo,
Patrick Gage Kelley,
Tara Matthews,
Kurt Thomas,
Lee Dunn,
Elie Bursztein
Abstract:
People who are involved with political campaigns face increased digital security threats from well-funded, sophisticated attackers, especially nation-states. Improving political campaign security is a vital part of protecting democracy. To identify campaign security issues, we conducted qualitative research with 28 participants across the U.S. political spectrum to understand the digital security…
▽ More
People who are involved with political campaigns face increased digital security threats from well-funded, sophisticated attackers, especially nation-states. Improving political campaign security is a vital part of protecting democracy. To identify campaign security issues, we conducted qualitative research with 28 participants across the U.S. political spectrum to understand the digital security practices, challenges, and perceptions of people involved in campaigns. A main, overarching finding is that a unique combination of threats, constraints, and work culture lead people involved with political campaigns to use technologies from across platforms and domains in ways that leave them--and democracy--vulnerable to security attacks. Sensitive data was kept in a plethora of personal and work accounts, with ad hoc adoption of strong passwords, two-factor authentication, encryption, and access controls. No individual company, committee, organization, campaign, or academic institution can solve the identified problems on their own. To this end, we provide an initial understanding of this complex problem space and recommendations for how a diverse group of experts can begin working together to improve security for political campaigns.
△ Less
Submitted 1 June, 2021;
originally announced June 2021.
-
"A cold, technical decision-maker": Can AI provide explainability, negotiability, and humanity?
Authors:
Allison Woodruff,
Yasmin Asare Anderson,
Katherine Jameson Armstrong,
Marina Gkiza,
Jay Jennings,
Christopher Moessner,
Fernanda Viegas,
Martin Wattenberg,
and Lynette Webb,
Fabian Wrede,
Patrick Gage Kelley
Abstract:
Algorithmic systems are increasingly deployed to make decisions in many areas of people's lives. The shift from human to algorithmic decision-making has been accompanied by concern about potentially opaque decisions that are not aligned with social values, as well as proposed remedies such as explainability. We present results of a qualitative study of algorithmic decision-making, comprised of fiv…
▽ More
Algorithmic systems are increasingly deployed to make decisions in many areas of people's lives. The shift from human to algorithmic decision-making has been accompanied by concern about potentially opaque decisions that are not aligned with social values, as well as proposed remedies such as explainability. We present results of a qualitative study of algorithmic decision-making, comprised of five workshops conducted with a total of 60 participants in Finland, Germany, the United Kingdom, and the United States. We invited participants to reason about decision-making qualities such as explainability and accuracy in a variety of domains. Participants viewed AI as a decision-maker that follows rigid criteria and performs mechanical tasks well, but is largely incapable of subjective or morally complex judgments. We discuss participants' consideration of humanity in decision-making, and introduce the concept of 'negotiability,' the ability to go beyond formal criteria and work flexibly around the system.
△ Less
Submitted 1 December, 2020;
originally announced December 2020.
-
Explainability Case Studies
Authors:
Ben Zevenbergen,
Allison Woodruff,
Patrick Gage Kelley
Abstract:
Explainability is one of the key ethical concepts in the design of AI systems. However, attempts to operationalize this concept thus far have tended to focus on approaches such as new software for model interpretability or guidelines with checklists. Rarely do existing tools and guidance incentivize the designers of AI systems to think critically and strategically about the role of explanations in…
▽ More
Explainability is one of the key ethical concepts in the design of AI systems. However, attempts to operationalize this concept thus far have tended to focus on approaches such as new software for model interpretability or guidelines with checklists. Rarely do existing tools and guidance incentivize the designers of AI systems to think critically and strategically about the role of explanations in their systems. We present a set of case studies of a hypothetical AI-enabled product, which serves as a pedagogical tool to empower product designers, developers, students, and educators to develop a holistic explainability strategy for their own products.
△ Less
Submitted 2 October, 2020; v1 submitted 1 September, 2020;
originally announced September 2020.
-
Exciting, Useful, Worrying, Futuristic: Public Perception of Artificial Intelligence in 8 Countries
Authors:
Patrick Gage Kelley,
Yongwei Yang,
Courtney Heldreth,
Christopher Moessner,
Aaron Sedley,
Andreas Kramm,
David T. Newman,
Allison Woodruff
Abstract:
As the influence and use of artificial intelligence (AI) have grown and its transformative potential has become more apparent, many questions have been raised regarding the economic, political, social, and ethical implications of its use. Public opinion plays an important role in these discussions, influencing product adoption, commercial development, research funding, and regulation. In this pape…
▽ More
As the influence and use of artificial intelligence (AI) have grown and its transformative potential has become more apparent, many questions have been raised regarding the economic, political, social, and ethical implications of its use. Public opinion plays an important role in these discussions, influencing product adoption, commercial development, research funding, and regulation. In this paper we present results of an in-depth survey of public opinion of artificial intelligence conducted with 10,005 respondents spanning eight countries and six continents. We report widespread perception that AI will have significant impact on society, accompanied by strong support for the responsible development and use of AI, and also characterize the public's sentiment towards AI with four key themes (exciting, useful, worrying, and futuristic) whose prevalence distinguishes response to AI in different countries.
△ Less
Submitted 18 May, 2021; v1 submitted 27 December, 2019;
originally announced January 2020.
