-
Semgrex and Ssurgeon, Searching and Manipulating Dependency Graphs
Authors:
John Bauer,
Chloe Kiddon,
Eric Yeh,
Alex Shan,
Christopher D. Manning
Abstract:
Searching dependency graphs and manipulating them can be a time consuming and challenging task to get right. We document Semgrex, a system for searching dependency graphs, and introduce Ssurgeon, a system for manipulating the output of Semgrex. The compact language used by these systems allows for easy command line or API processing of dependencies. Additionally, integration with publicly released…
▽ More
Searching dependency graphs and manipulating them can be a time consuming and challenging task to get right. We document Semgrex, a system for searching dependency graphs, and introduce Ssurgeon, a system for manipulating the output of Semgrex. The compact language used by these systems allows for easy command line or API processing of dependencies. Additionally, integration with publicly released toolkits in Java and Python allows for searching text relations and attributes over natural text.
△ Less
Submitted 24 April, 2024;
originally announced April 2024.
-
Federated Evaluation of On-device Personalization
Authors:
Kangkang Wang,
Rajiv Mathews,
Chloé Kiddon,
Hubert Eichner,
Françoise Beaufays,
Daniel Ramage
Abstract:
Federated learning is a distributed, on-device computation framework that enables training global models without exporting sensitive user data to servers. In this work, we describe methods to extend the federation framework to evaluate strategies for personalization of global models. We present tools to analyze the effects of personalization and evaluate conditions under which personalization yiel…
▽ More
Federated learning is a distributed, on-device computation framework that enables training global models without exporting sensitive user data to servers. In this work, we describe methods to extend the federation framework to evaluate strategies for personalization of global models. We present tools to analyze the effects of personalization and evaluate conditions under which personalization yields desirable models. We report on our experiments personalizing a language model for a virtual keyboard for smartphones with a population of tens of millions of users. We show that a significant fraction of users benefit from personalization.
△ Less
Submitted 22 October, 2019;
originally announced October 2019.
-
Towards Federated Learning at Scale: System Design
Authors:
Keith Bonawitz,
Hubert Eichner,
Wolfgang Grieskamp,
Dzmitry Huba,
Alex Ingerman,
Vladimir Ivanov,
Chloe Kiddon,
Jakub Konečný,
Stefano Mazzocchi,
H. Brendan McMahan,
Timon Van Overveldt,
David Petrou,
Daniel Ramage,
Jason Roselander
Abstract:
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and…
▽ More
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and future directions.
△ Less
Submitted 22 March, 2019; v1 submitted 4 February, 2019;
originally announced February 2019.
-
Federated Learning for Mobile Keyboard Prediction
Authors:
Andrew Hard,
Kanishka Rao,
Rajiv Mathews,
Swaroop Ramaswamy,
Françoise Beaufays,
Sean Augenstein,
Hubert Eichner,
Chloé Kiddon,
Daniel Ramage
Abstract:
We train a recurrent neural network language model using a distributed, on-device learning framework called federated learning for the purpose of next-word prediction in a virtual keyboard for smartphones. Server-based training using stochastic gradient descent is compared with training on client devices using the Federated Averaging algorithm. The federated algorithm, which enables training on a…
▽ More
We train a recurrent neural network language model using a distributed, on-device learning framework called federated learning for the purpose of next-word prediction in a virtual keyboard for smartphones. Server-based training using stochastic gradient descent is compared with training on client devices using the Federated Averaging algorithm. The federated algorithm, which enables training on a higher-quality dataset for this use case, is shown to achieve better prediction recall. This work demonstrates the feasibility and benefit of training language models on client devices without exporting sensitive user data to servers. The federated learning environment gives users greater control over the use of their data and simplifies the task of incorporating privacy by default with distributed training and aggregation across a population of client devices.
△ Less
Submitted 28 February, 2019; v1 submitted 8 November, 2018;
originally announced November 2018.
