-
To Build Our Future, We Must Know Our Past: Contextualizing Paradigm Shifts in Natural Language Processing
Authors:
Sireesh Gururaja,
Amanda Bertsch,
Clara Na,
David Gray Widder,
Emma Strubell
Abstract:
NLP is in a period of disruptive change that is impacting our methodologies, funding sources, and public perception. In this work, we seek to understand how to shape our future by better understanding our past. We study factors that shape NLP as a field, including culture, incentives, and infrastructure by conducting long-form interviews with 26 NLP researchers of varying seniority, research area,…
▽ More
NLP is in a period of disruptive change that is impacting our methodologies, funding sources, and public perception. In this work, we seek to understand how to shape our future by better understanding our past. We study factors that shape NLP as a field, including culture, incentives, and infrastructure by conducting long-form interviews with 26 NLP researchers of varying seniority, research area, institution, and social identity. Our interviewees identify cyclical patterns in the field, as well as new shifts without historical parallel, including changes in benchmark culture and software infrastructure. We complement this discussion with quantitative analysis of citation, authorship, and language use in the ACL Anthology over time. We conclude by discussing shared visions, concerns, and hopes for the future of NLP. We hope that this study of our field's past and present can prompt informed discussion of our community's implicit norms and more deliberate action to consciously shape the future.
△ Less
Submitted 11 October, 2023;
originally announced October 2023.
-
LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs
Authors:
Tongshuang Wu,
Haiyi Zhu,
Maya Albayrak,
Alexis Axon,
Amanda Bertsch,
Wenxing Deng,
Ziqi Ding,
Bill Guo,
Sireesh Gururaja,
Tzu-Sheng Kuo,
Jenny T. Liang,
Ryan Liu,
Ihita Mandal,
Jeremiah Milbauer,
Xiaolin Ni,
Namrata Padmanabhan,
Subhashini Ramkumar,
Alexis Sudjianto,
Jordan Taylor,
Ying-Jui Tseng,
Patricia Vaidos,
Zhijin Wu,
Wei Wu,
Chenyang Yang
Abstract:
LLMs have shown promise in replicating human-like behavior in crowdsourcing tasks that were previously thought to be exclusive to human abilities. However, current efforts focus mainly on simple atomic tasks. We explore whether LLMs can replicate more complex crowdsourcing pipelines. We find that modern LLMs can simulate some of crowdworkers' abilities in these "human computation algorithms," but…
▽ More
LLMs have shown promise in replicating human-like behavior in crowdsourcing tasks that were previously thought to be exclusive to human abilities. However, current efforts focus mainly on simple atomic tasks. We explore whether LLMs can replicate more complex crowdsourcing pipelines. We find that modern LLMs can simulate some of crowdworkers' abilities in these "human computation algorithms," but the level of success is variable and influenced by requesters' understanding of LLM capabilities, the specific skills required for sub-tasks, and the optimal interaction modality for performing these sub-tasks. We reflect on human and LLMs' different sensitivities to instructions, stress the importance of enabling human-facing safeguards for LLMs, and discuss the potential of training humans and LLMs with complementary skill sets. Crucially, we show that replicating crowdsourcing pipelines offers a valuable platform to investigate (1) the relative strengths of LLMs on different tasks (by cross-comparing their performances on sub-tasks) and (2) LLMs' potential in complex tasks, where they can complete part of the tasks while leaving others to humans.
△ Less
Submitted 19 July, 2023; v1 submitted 19 July, 2023;
originally announced July 2023.
-
Linguistic representations for fewer-shot relation extraction across domains
Authors:
Sireesh Gururaja,
Ritam Dutt,
Tinglong Liao,
Carolyn Rose
Abstract:
Recent work has demonstrated the positive impact of incorporating linguistic representations as additional context and scaffolding on the in-domain performance of several NLP tasks. We extend this work by exploring the impact of linguistic representations on cross-domain performance in a few-shot transfer setting. An important question is whether linguistic representations enhance generalizability…
▽ More
Recent work has demonstrated the positive impact of incorporating linguistic representations as additional context and scaffolding on the in-domain performance of several NLP tasks. We extend this work by exploring the impact of linguistic representations on cross-domain performance in a few-shot transfer setting. An important question is whether linguistic representations enhance generalizability by providing features that function as cross-domain pivots. We focus on the task of relation extraction on three datasets of procedural text in two domains, cooking and materials science. Our approach augments a popular transformer-based architecture by alternately incorporating syntactic and semantic graphs constructed by freely available off-the-shelf tools. We examine their utility for enhancing generalization, and investigate whether earlier findings, e.g. that semantic representations can be more helpful than syntactic ones, extend to relation extraction in multiple domains. We find that while the inclusion of these graphs results in significantly higher performance in few-shot transfer, both types of graph exhibit roughly equivalent utility.
△ Less
Submitted 7 July, 2023;
originally announced July 2023.
-
Flexural response of concurrently 3D printed sandwich composite
Authors:
Bharath H S,
Dileep Bonthu,
Suhasini Gururaja,
Pavana Prabhakar,
Mrityunjay Doddamani
Abstract:
Among many lightweight materials used in marine applications, sandwich structures with syntactic foam core are promising because of lower water uptake in foam core amid face-sheets damage. HDPE (high-density polyethylene) filament is used to 3D print sandwich skin, and glass microballoon (GMB) reinforced HDPE syntactic foam filaments are used for the core. The optimized parameters are used to prep…
▽ More
Among many lightweight materials used in marine applications, sandwich structures with syntactic foam core are promising because of lower water uptake in foam core amid face-sheets damage. HDPE (high-density polyethylene) filament is used to 3D print sandwich skin, and glass microballoon (GMB) reinforced HDPE syntactic foam filaments are used for the core. The optimized parameters are used to prepare blends of 20, 40, and 60 volume % of GMB in HDPE. These foamed blends are extruded in filament form to be subsequently used in commercially available fused filament fabrication (FFF) based 3D printers. The defect-free syntactic foam core sandwich composites are 3D printed concurrently for characterizing their flexural behavior. The printed HDPE, foam cores, and sandwiches are tested under three-point bending mode. The addition of GMB increases both specific modulus and strength in sandwich composites and is highest for the sandwich having a core with 60 volume % of GMB. The flexural strength, fracture strength, and strain of foam core sandwiches registered superior response than their respective cores. The experimental results are found in good agreement compared with theoretical predictions. Finally, the failure mode of the printed sandwich is also discussed.
△ Less
Submitted 22 July, 2020;
originally announced July 2020.
