Showing 1–34 of 34 results for author: Gandhi, D

BraTS-PEDs: Results of the Multi-Consortium International Pediatric Brain Tumor Segmentation Challenge 2023

Authors: Anahita Fathi Kazerooni, Nastaran Khalili, Xinyang Liu, Debanjan Haldar, Zhifan Jiang, Anna Zapaishchykova, Julija Pavaine, Lubdha M. Shah, Blaise V. Jones, Nakul Sheth, Sanjay P. Prabhu, Aaron S. McAllister, Wenxin Tu, Khanak K. Nandolia, Andres F. Rodriguez, Ibraheem Salman Shaikh, Mariana Sanchez Montano, Hollie Anne Lai, Maruf Adewole, Jake Albrecht, Udunna Anazodo, Hannah Anderson, Syed Muhammed Anwar, Alejandro Aristizabal, Sina Bagheri , et al. (54 additional authors not shown)

Abstract: Pediatric central nervous system tumors are the leading cause of cancer-related deaths in children. The five-year survival rate for high-grade glioma in children is less than 20%. The development of new treatments is dependent upon multi-institutional collaborative clinical trials requiring reproducible and accurate centralized response assessment. We present the results of the BraTS-PEDs 2023 cha… ▽ More Pediatric central nervous system tumors are the leading cause of cancer-related deaths in children. The five-year survival rate for high-grade glioma in children is less than 20%. The development of new treatments is dependent upon multi-institutional collaborative clinical trials requiring reproducible and accurate centralized response assessment. We present the results of the BraTS-PEDs 2023 challenge, the first Brain Tumor Segmentation (BraTS) challenge focused on pediatric brain tumors. This challenge utilized data acquired from multiple international consortia dedicated to pediatric neuro-oncology and clinical trials. BraTS-PEDs 2023 aimed to evaluate volumetric segmentation algorithms for pediatric brain gliomas from magnetic resonance imaging using standardized quantitative performance evaluation metrics employed across the BraTS 2023 challenges. The top-performing AI approaches for pediatric tumor analysis included ensembles of nnU-Net and Swin UNETR, Auto3DSeg, or nnU-Net with a self-supervised framework. The BraTSPEDs 2023 challenge fostered collaboration between clinicians (neuro-oncologists, neuroradiologists) and AI/imaging scientists, promoting faster data sharing and the development of automated volumetric analysis techniques. These advancements could significantly benefit clinical trials and improve the care of children with brain tumors. △ Less

Submitted 11 July, 2024; originally announced July 2024.

SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts

Authors: Raghu Prabhakar, Ram Sivaramakrishnan, Darshan Gandhi, Yun Du, Mingran Wang, Xiangyu Song, Kejie Zhang, Tianren Gao, Angela Wang, Karen Li, Yongning Sheng, Joshua Brot, Denis Sokolov, Apurv Vivek, Calvin Leung, Arjun Sabnis, Jiayu Bai, Tuowen Zhao, Mark Gottscho, David Jackson, Mark Luttrell, Manish K. Shah, Edison Chen, Kaizhao Liang, Swayambhoo Jain , et al. (5 additional authors not shown)

Abstract: Monolithic large language models (LLMs) like GPT-4 have paved the way for modern generative AI applications. Training, serving, and maintaining monolithic LLMs at scale, however, remains prohibitively expensive and challenging. The disproportionate increase in compute-to-memory ratio of modern AI accelerators have created a memory wall, necessitating new methods to deploy AI. Composition of Expert… ▽ More Monolithic large language models (LLMs) like GPT-4 have paved the way for modern generative AI applications. Training, serving, and maintaining monolithic LLMs at scale, however, remains prohibitively expensive and challenging. The disproportionate increase in compute-to-memory ratio of modern AI accelerators have created a memory wall, necessitating new methods to deploy AI. Composition of Experts (CoE) is an alternative modular approach that lowers the cost and complexity of training and serving. However, this approach presents two key challenges when using conventional hardware: (1) without fused operations, smaller models have lower operational intensity, which makes high utilization more challenging to achieve; and (2) hosting a large number of models can be either prohibitively expensive or slow when dynamically switching between them. In this paper, we describe how combining CoE, streaming dataflow, and a three-tier memory system scales the AI memory wall. We describe Samba-CoE, a CoE system with 150 experts and a trillion total parameters. We deploy Samba-CoE on the SambaNova SN40L Reconfigurable Dataflow Unit (RDU) - a commercial dataflow accelerator architecture that has been co-designed for enterprise inference and training applications. The chip introduces a new three-tier memory system with on-chip distributed SRAM, on-package HBM, and off-package DDR DRAM. A dedicated inter-RDU network enables scaling up and out over multiple sockets. We demonstrate speedups ranging from 2x to 13x on various benchmarks running on eight RDU sockets compared with an unfused baseline. We show that for CoE inference deployments, the 8-socket RDU Node reduces machine footprint by up to 19x, speeds up model switching time by 15x to 31x, and achieves an overall speedup of 3.7x over a DGX H100 and 6.6x over a DGX A100. △ Less

Submitted 13 May, 2024; originally announced May 2024.

The Brain Tumor Segmentation in Pediatrics (BraTS-PEDs) Challenge: Focus on Pediatrics (CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs)

Authors: Anahita Fathi Kazerooni, Nastaran Khalili, Xinyang Liu, Deep Gandhi, Zhifan Jiang, Syed Muhammed Anwar, Jake Albrecht, Maruf Adewole, Udunna Anazodo, Hannah Anderson, Ujjwal Baid, Timothy Bergquist, Austin J. Borja, Evan Calabrese, Verena Chung, Gian-Marco Conte, Farouk Dako, James Eddy, Ivan Ezhov, Ariana Familiar, Keyvan Farahani, Andrea Franson, Anurag Gottipati, Shuvanjan Haldar, Juan Eugenio Iglesias , et al. (46 additional authors not shown)

Abstract: Pediatric tumors of the central nervous system are the most common cause of cancer-related death in children. The five-year survival rate for high-grade gliomas in children is less than 20%. Due to their rarity, the diagnosis of these entities is often delayed, their treatment is mainly based on historic treatment concepts, and clinical trials require multi-institutional collaborations. Here we pr… ▽ More Pediatric tumors of the central nervous system are the most common cause of cancer-related death in children. The five-year survival rate for high-grade gliomas in children is less than 20%. Due to their rarity, the diagnosis of these entities is often delayed, their treatment is mainly based on historic treatment concepts, and clinical trials require multi-institutional collaborations. Here we present the CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs challenge, focused on pediatric brain tumors with data acquired across multiple international consortia dedicated to pediatric neuro-oncology and clinical trials. The CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs challenge brings together clinicians and AI/imaging scientists to lead to faster development of automated segmentation techniques that could benefit clinical trials, and ultimately the care of children with brain tumors. △ Less

Submitted 11 July, 2024; v1 submitted 23 April, 2024; originally announced April 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2305.17033

Training and Comparison of nnU-Net and DeepMedic Methods for Autosegmentation of Pediatric Brain Tumors

Authors: Arastoo Vossough, Nastaran Khalili, Ariana M. Familiar, Deep Gandhi, Karthik Viswanathan, Wenxin Tu, Debanjan Haldar, Sina Bagheri, Hannah Anderson, Shuvanjan Haldar, Phillip B. Storm, Adam Resnick, Jeffrey B. Ware, Ali Nabavizadeh, Anahita Fathi Kazerooni

Abstract: Brain tumors are the most common solid tumors and the leading cause of cancer-related death among children. Tumor segmentation is essential in surgical and treatment planning, and response assessment and monitoring. However, manual segmentation is time-consuming and has high inter-operator variability, underscoring the need for more efficient methods. We compared two deep learning-based 3D segment… ▽ More Brain tumors are the most common solid tumors and the leading cause of cancer-related death among children. Tumor segmentation is essential in surgical and treatment planning, and response assessment and monitoring. However, manual segmentation is time-consuming and has high inter-operator variability, underscoring the need for more efficient methods. We compared two deep learning-based 3D segmentation models, DeepMedic and nnU-Net, after training with pediatric-specific multi-institutional brain tumor data using based on multi-parametric MRI scans.Multi-parametric preoperative MRI scans of 339 pediatric patients (n=293 internal and n=46 external cohorts) with a variety of tumor subtypes, were preprocessed and manually segmented into four tumor subregions, i.e., enhancing tumor (ET), non-enhancing tumor (NET), cystic components (CC), and peritumoral edema (ED). After training, performance of the two models on internal and external test sets was evaluated using Dice scores, sensitivity, and Hausdorff distance with reference to ground truth manual segmentations. Dice score for nnU-Net internal test sets was (mean +/- SD (median)) 0.9+/-0.07 (0.94) for WT, 0.77+/-0.29 for ET, 0.66+/-0.32 for NET, 0.71+/-0.33 for CC, and 0.71+/-0.40 for ED, respectively. For DeepMedic the Dice scores were 0.82+/-0.16 for WT, 0.66+/-0.32 for ET, 0.48+/-0.27, for NET, 0.48+/-0.36 for CC, and 0.19+/-0.33 for ED, respectively. Dice scores were significantly higher for nnU-Net (p<=0.01). External validation of the trained nnU-Net model on the multi-institutional BraTS-PEDs 2023 dataset revealed high generalization capability in segmentation of whole tumor and tumor core with Dice scores of 0.87+/-0.13 (0.91) and 0.83+/-0.18 (0.89), respectively. Pediatric-specific data trained nnU-Net model is superior to DeepMedic for whole tumor and subregion segmentation of pediatric brain tumors. △ Less

Submitted 30 January, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Computational framework for the generation of one-dimensional vascular models accounting for uncertainty in networks extracted from medical images

Authors: Michelle A Bartolo, Alyssa M Taylor-LaPole, Darsh Gandhi, Alexandria Johnson, Yaqi Li, Emma Slack, Isaiah Stevens, Zachary Turner, Justin D Weigand, Charles Puelz, Dirk Husmeier, Mette S Olufsen

Abstract: Patient-specific computational modeling is a popular, non-invasive method to answer medical questions. Medical images are used to extract geometric domains necessary to create these models, providing a predictive tool for clinicians. However, in vivo imaging is subject to uncertainty, impacting vessel dimensions essential to the mathematical modeling process. While there are numerous programs avai… ▽ More Patient-specific computational modeling is a popular, non-invasive method to answer medical questions. Medical images are used to extract geometric domains necessary to create these models, providing a predictive tool for clinicians. However, in vivo imaging is subject to uncertainty, impacting vessel dimensions essential to the mathematical modeling process. While there are numerous programs available to provide information about vessel length, radii, and position, there is currently no exact way to determine and calibrate these features. This raises the question, if we are building patient-specific models based on uncertain measurements, how accurate are the geometries we extract and how can we best represent a patient's vasculature? In this study, we develop a novel framework to determine vessel dimensions using change points. We explore the impact of uncertainty in the network extraction process on hemodynamics by varying vessel dimensions and segmenting the same images multiple times. Our analyses reveal that image segmentation, network size, and minor changes in radius and length have significant impacts on pressure and flow dynamics in rapidly branching structures and tapering vessels. Accordingly, we conclude that it is critical to understand how uncertainty in network geometry propagates to fluid dynamics, especially in clinical applications. △ Less

Submitted 8 May, 2024; v1 submitted 15 September, 2023; originally announced September 2023.

Comments: 42 pages, 10 figures

Permutationally Invariant Networks for Enhanced Sampling (PINES): Discovery of Multi-Molecular and Solvent-Inclusive Collective Variables

Authors: Nicholas S. M. Herringer, Siva Dasetty, Diya Gandhi, Junhee Lee, Andrew L. Ferguson

Abstract: The typically rugged nature of molecular free energy landscapes can frustrate efficient sampling of the thermodynamically relevant phase space due to the presence of high free energy barriers. Enhanced sampling techniques can improve phase space exploration by accelerating sampling along particular collective variables (CVs). A number of techniques exist for data-driven discovery of CVs parameteri… ▽ More The typically rugged nature of molecular free energy landscapes can frustrate efficient sampling of the thermodynamically relevant phase space due to the presence of high free energy barriers. Enhanced sampling techniques can improve phase space exploration by accelerating sampling along particular collective variables (CVs). A number of techniques exist for data-driven discovery of CVs parameterizing the important large scale motions of the system. A challenge to CV discovery is learning CVs invariant to symmetries of the molecular system, frequently rigid translation, rigid rotation, and permutational relabeling of identical particles. Of these, permutational invariance have proved a persistent challenge in frustrating the the data-driven discovery of multi-molecular CVs in systems of self-assembling particles and solvent-inclusive CVs for solvated systems. In this work, we integrate Permutation Invariant Vector (PIV) featurizations with autoencoding neural networks to learn nonlinear CVs invariant to translation, rotation, and permutation, and perform interleaved rounds of CV discovery and enhanced sampling to iteratively expand sampling of configurational phase space and obtain converged CVs and free energy landscapes. We demonstrate the Permutationally Invariant Network for Enhanced Sampling (PINES) approach in applications to the self-assembly of a 13-atom Argon cluster, association/dissociation of a NaCl ion pair in water, and hydrophobic collapse of a C45H92 n-pentatetracontane polymer chain. We make the approach freely available as a new module within the PLUMED2 enhanced sampling libraries. △ Less

Submitted 16 August, 2023; originally announced August 2023.

Chemellia: An Ecosystem for Atomistic Scientific Machine Learning

Authors: Anant Thazhemadam, Dhairya Gandhi, Venkatasubramanian Viswanathan, Rachel C. Kurchin

Abstract: Chemellia is an open-source framework for atomistic machine learning in the Julia programming language. The framework takes advantage of Julia's high speed as well as the ability to share and reuse code and interfaces through the paradigm of multiple dispatch. Chemellia is designed to make use of existing interfaces and avoid ``reinventing the wheel'' wherever possible. A key aspect of the Chemell… ▽ More Chemellia is an open-source framework for atomistic machine learning in the Julia programming language. The framework takes advantage of Julia's high speed as well as the ability to share and reuse code and interfaces through the paradigm of multiple dispatch. Chemellia is designed to make use of existing interfaces and avoid ``reinventing the wheel'' wherever possible. A key aspect of the Chemellia ecosystem is the ChemistryFeaturization interface for defining and encoding features -- it is designed to maximize interoperability between featurization schemes and elements thereof, to maintain provenance of encoded features, and to ensure easy decodability and reconfigurability to enable feature engineering experiments. This embodies the overall design principles of the Chemellia ecosystem: separation of concerns, interoperability, and transparency. We illustrate these principles by discussing the implementation of crystal graph convolutional neural networks for material property prediction. △ Less

Submitted 19 May, 2023; originally announced May 2023.

Training Large Language Models Efficiently with Sparsity and Dataflow

Authors: Venkat Srinivasan, Darshan Gandhi, Urmish Thakker, Raghu Prabhakar

Abstract: Large foundation language models have shown their versatility in being able to be adapted to perform a wide variety of downstream tasks, such as text generation, sentiment analysis, semantic search etc. However, training such large foundational models is a non-trivial exercise that requires a significant amount of compute power and expertise from machine learning and systems experts. As models get… ▽ More Large foundation language models have shown their versatility in being able to be adapted to perform a wide variety of downstream tasks, such as text generation, sentiment analysis, semantic search etc. However, training such large foundational models is a non-trivial exercise that requires a significant amount of compute power and expertise from machine learning and systems experts. As models get larger, these demands are only increasing. Sparsity is a promising technique to relieve the compute requirements for training. However, sparsity introduces new challenges in training the sparse model to the same quality as the dense counterparts. Furthermore, sparsity drops the operation intensity and introduces irregular memory access patterns that makes it challenging to efficiently utilize compute resources. This paper demonstrates an end-to-end training flow on a large language model - 13 billion GPT - using sparsity and dataflow. The dataflow execution model and architecture enables efficient on-chip irregular memory accesses as well as native kernel fusion and pipelined parallelism that helps recover device utilization. We show that we can successfully train GPT 13B to the same quality as the dense GPT 13B model, while achieving an end-end speedup of 4.5x over dense A100 baseline. △ Less

Submitted 11 April, 2023; originally announced April 2023.

A Federated Approach for Hate Speech Detection

Authors: Jay Gala, Deep Gandhi, Jash Mehta, Zeerak Talat

Abstract: Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that… ▽ More Hate speech detection has been the subject of high research attention, due to the scale of content created on social media. In spite of the attention and the sensitive nature of the task, privacy preservation in hate speech detection has remained under-studied. The majority of research has focused on centralised machine learning infrastructures which risk leaking data. In this paper, we show that using federated machine learning can help address privacy the concerns that are inherent to hate speech detection while obtaining up to 6.81% improvement in terms of F1-score. △ Less

Submitted 18 February, 2023; originally announced February 2023.

Comments: EACL 2023 Main Conference (Short Paper)

Nonequilibrium Electrochemical Phase Maps: Beyond Butler-Volmer Kinetics

Authors: Rachel C. Kurchin, Dhairya Gandhi, Venkatasubramanian Viswanathan

Abstract: Electrochemical kinetics at electrode-electrolyte interfaces are crucial to understand high-rate behavior of energy storage devices. Phase transformation of electrodes is typically treated under equilibrium thermodynamic conditions, while realistic operation is at finite rates. Analyzing phase transformations under nonequilibrium conditions requires integrating nonlinear electrochemical kinetic mo… ▽ More Electrochemical kinetics at electrode-electrolyte interfaces are crucial to understand high-rate behavior of energy storage devices. Phase transformation of electrodes is typically treated under equilibrium thermodynamic conditions, while realistic operation is at finite rates. Analyzing phase transformations under nonequilibrium conditions requires integrating nonlinear electrochemical kinetic models with thermodynamic models. This had only previously been demonstrated for Butler-Volmer kinetics, where it can be done analytically. In this work, we develop a kinetic modeling package in the Julia language capable of efficient numerical inversion of rate relationships for general kinetic models using automatic differentiation. We demonstrate building nonequilibrium phase maps, including for models such as Marcus-Hush-Chidsey that require computation of an integral, and also discuss the impact of a variety of assumptions and model parameters (such as temperature, reorganization energy, activity, and ideal solution interaction energy), particularly on high-rate phase behavior. Even for a fixed set of parameters, the magnitude of the critical current can vary by in excess of a factor of two amongst kinetic models. △ Less

Submitted 13 December, 2022; originally announced December 2022.

Comments: main text + supplementary info (15+6 pages, 3+2 figures), as submitted to ACS Energy Letters

A Federated Approach to Predicting Emojis in Hindi Tweets

Authors: Deep Gandhi, Jash Mehta, Nirali Parekh, Karan Waghela, Lynette D'Mello, Zeerak Talat

Abstract: The use of emojis affords a visual modality to, often private, textual communication. The task of predicting emojis however provides a challenge for machine learning as emoji use tends to cluster into the frequently used and the rarely used emojis. Much of the machine learning research on emoji use has focused on high resource languages and has conceptualised the task of predicting emojis around t… ▽ More The use of emojis affords a visual modality to, often private, textual communication. The task of predicting emojis however provides a challenge for machine learning as emoji use tends to cluster into the frequently used and the rarely used emojis. Much of the machine learning research on emoji use has focused on high resource languages and has conceptualised the task of predicting emojis around traditional server-side machine learning approaches. However, traditional machine learning approaches for private communication can introduce privacy concerns, as these approaches require all data to be transmitted to a central storage. In this paper, we seek to address the dual concerns of emphasising high resource languages for emoji prediction and risking the privacy of people's data. We introduce a new dataset of $118$k tweets (augmented from $25$k unique tweets) for emoji prediction in Hindi, and propose a modification to the federated learning algorithm, CausalFedGSD, which aims to strike a balance between model performance and user privacy. We show that our approach obtains comparative scores with more complex centralised models while reducing the amount of data required to optimise the models and minimising risks to user privacy. △ Less

Submitted 11 November, 2022; originally announced November 2022.

Comments: EMNLP2022 Main Track Short Paper

Score-Based Generative Models for Molecule Generation

Authors: Dwaraknath Gnaneshwar, Bharath Ramsundar, Dhairya Gandhi, Rachel Kurchin, Venkatasubramanian Viswanathan

Abstract: Recent advances in generative models have made exploring design spaces easier for de novo molecule generation. However, popular generative models like GANs and normalizing flows face challenges such as training instabilities due to adversarial training and architectural constraints, respectively. Score-based generative models sidestep these challenges by modelling the gradient of the log probabili… ▽ More Recent advances in generative models have made exploring design spaces easier for de novo molecule generation. However, popular generative models like GANs and normalizing flows face challenges such as training instabilities due to adversarial training and architectural constraints, respectively. Score-based generative models sidestep these challenges by modelling the gradient of the log probability density using a score function approximation, as opposed to modelling the density function directly, and sampling from it using annealed Langevin Dynamics. We believe that score-based generative models could open up new opportunities in molecule generation due to their architectural flexibility, such as replacing the score function with an SE(3) equivariant model. In this work, we lay the foundations by testing the efficacy of score-based models for molecule generation. We train a Transformer-based score function on Self-Referencing Embedded Strings (SELFIES) representations of 1.5 million samples from the ZINC dataset and use the Moses benchmarking framework to evaluate the generated samples on a suite of metrics. △ Less

Submitted 7 March, 2022; originally announced March 2022.

droidlet: modular, heterogenous, multi-modal agents

Authors: Anurag Pratik, Soumith Chintala, Kavya Srinet, Dhiraj Gandhi, Rebecca Qian, Yuxuan Sun, Ryan Drew, Sara Elkafrawy, Anoushka Tiwari, Tucker Hart, Mary Williamson, Abhinav Gupta, Arthur Szlam

Abstract: In recent years, there have been significant advances in building end-to-end Machine Learning (ML) systems that learn at scale. But most of these systems are: (a) isolated (perception, speech, or language only); (b) trained on static datasets. On the other hand, in the field of robotics, large-scale learning has always been difficult. Supervision is hard to gather and real world physical interacti… ▽ More In recent years, there have been significant advances in building end-to-end Machine Learning (ML) systems that learn at scale. But most of these systems are: (a) isolated (perception, speech, or language only); (b) trained on static datasets. On the other hand, in the field of robotics, large-scale learning has always been difficult. Supervision is hard to gather and real world physical interactions are expensive. In this work we introduce and open-source droidlet, a modular, heterogeneous agent architecture and platform. It allows us to exploit both large-scale static datasets in perception and language and sophisticated heuristics often used in robotics; and provides tools for interactive annotation. Furthermore, it brings together perception, language and action onto one platform, providing a path towards agents that learn from the richness of real world interactions. △ Less

Submitted 25 January, 2021; originally announced January 2021.

MIT SafePaths Card (MiSaCa): Augmenting Paper Based Vaccination Cards with Printed Codes

Authors: Joseph Bae, Rohan Sukumaran, Sheshank Shankar, Saurish Srivastava, Rohan Iyer, Aryan Mahindra, Qamil Mirza, Maurizio Arseni, Anshuman Sharma, Saras Agrawal, Orna Mukhopadhyay, Colin Kang, Priyanshi Katiyar, Apurv Shekhar, Sifat Hasan, Krishnendu Dasgupta, Darshan Gandhi, Sethuramen TV, Parth Patwa, Ishaan Singh, Abhishek Singh, Ramesh Raskar

Abstract: In this early draft, we describe a user-centric, card-based system for vaccine distribution. Our system makes use of digitally signed QR codes and their use for phased vaccine distribution, vaccine administration/record-keeping, immunization verification, and follow-up symptom reporting. Furthermore, we propose and describe a complementary scanner app system to be used by vaccination clinics, publ… ▽ More In this early draft, we describe a user-centric, card-based system for vaccine distribution. Our system makes use of digitally signed QR codes and their use for phased vaccine distribution, vaccine administration/record-keeping, immunization verification, and follow-up symptom reporting. Furthermore, we propose and describe a complementary scanner app system to be used by vaccination clinics, public health officials, and immunization verification parties to effectively utilize card-based framework. We believe that the proposed system provides a privacy-preserving and efficient framework for vaccine distribution in both developed and developing regions. △ Less

Submitted 21 January, 2021; v1 submitted 19 January, 2021; originally announced January 2021.

Comments: 8 pages, 4 Figures, 1 Table

Spatial K-anonymity: A Privacy-preserving Method for COVID-19 Related Geospatial Technologies

Authors: Rohan Iyer, Regina Rex, Kevin P. McPherson, Darshan Gandhi, Aryan Mahindra, Abhishek Singh, Ramesh Raskar

Abstract: There is a growing need for spatial privacy considerations in the many geo-spatial technologies that have been created as solutions for COVID-19-related issues. Although effective geo-spatial technologies have already been rolled out, most have significantly sacrificed privacy for utility. In this paper, we explore spatial k-anonymity, a privacy-preserving method that can address this unnecessary… ▽ More There is a growing need for spatial privacy considerations in the many geo-spatial technologies that have been created as solutions for COVID-19-related issues. Although effective geo-spatial technologies have already been rolled out, most have significantly sacrificed privacy for utility. In this paper, we explore spatial k-anonymity, a privacy-preserving method that can address this unnecessary tradeoff by providing the best of both privacy and utility. After evaluating its past implications in geo-spatial use cases, we propose applications of spatial k-anonymity in the data sharing and managing of COVID-19 contact tracing technologies as well as heat maps showing a user's travel history. We then justify our propositions by comparing spatial k-anonymity with several other spatial privacy methods, including differential privacy, geo-indistinguishability, and manual consent based redaction. Our hope is to raise awareness of the ever-growing risks associated with spatial privacy and how they can be solved with Spatial K-anonymity. △ Less

Submitted 4 January, 2021; originally announced January 2021.

COVID-19 Tests Gone Rogue: Privacy, Efficacy, Mismanagement and Misunderstandings

Authors: Manuel Morales, Rachel Barbar, Darshan Gandhi, Sanskruti Landage, Joseph Bae, Arpita Vats, Jil Kothari, Sheshank Shankar, Rohan Sukumaran, Himi Mathur, Krutika Misra, Aishwarya Saxena, Parth Patwa, Sethuraman T. V., Maurizio Arseni, Shailesh Advani, Kasia Jakimowicz, Sunaina Anand, Priyanshi Katiyar, Ashley Mehra, Rohan Iyer, Srinidhi Murali, Aryan Mahindra, Mikhail Dmitrienko, Saurish Srivastava , et al. (5 additional authors not shown)

Abstract: COVID-19 testing, the cornerstone for effective screening and identification of COVID-19 cases, remains paramount as an intervention tool to curb the spread of COVID-19 both at local and national levels. However, the speed at which the pandemic struck and the response was rolled out, the widespread impact on healthcare infrastructure, the lack of sufficient preparation within the public health sys… ▽ More COVID-19 testing, the cornerstone for effective screening and identification of COVID-19 cases, remains paramount as an intervention tool to curb the spread of COVID-19 both at local and national levels. However, the speed at which the pandemic struck and the response was rolled out, the widespread impact on healthcare infrastructure, the lack of sufficient preparation within the public health system, and the complexity of the crisis led to utter confusion among test-takers. Invasion of privacy remains a crucial concern. The user experience of test takers remains low. User friction affects user behavior and discourages participation in testing programs. Test efficacy has been overstated. Test results are poorly understood resulting in inappropriate follow-up recommendations. Herein, we review the current landscape of COVID-19 testing, identify four key challenges, and discuss the consequences of the failure to address these challenges. The current infrastructure around testing and information propagation is highly privacy-invasive and does not leverage scalable digital components. In this work, we discuss challenges complicating the existing covid-19 testing ecosystem and highlight the need to improve the testing experience for the user and reduce privacy invasions. Digital tools will play a critical role in resolving these challenges. △ Less

Submitted 7 May, 2021; v1 submitted 5 January, 2021; originally announced January 2021.

Comments: 22 pages, 2 figures

Challenges of Equitable Vaccine Distribution in the COVID-19 Pandemic

Authors: Joseph Bae, Darshan Gandhi, Jil Kothari, Sheshank Shankar, Jonah Bae, Parth Patwa, Rohan Sukumaran, Aviral Chharia, Sanjay Adhikesaven, Shloak Rathod, Irene Nandutu, Sethuraman TV, Vanessa Yu, Krutika Misra, Srinidhi Murali, Aishwarya Saxena, Kasia Jakimowicz, Vivek Sharma, Rohan Iyer, Ashley Mehra, Alex Radunsky, Priyanshi Katiyar, Ananthu James, Jyoti Dalal, Sunaina Anand , et al. (3 additional authors not shown)

Abstract: The COVID-19 pandemic has led to a need for widespread and rapid vaccine development. As several vaccines have recently been approved for human use or are in different stages of development, governments across the world are preparing comprehensive guidelines for vaccine distribution and monitoring. In this early article, we identify challenges in logistics, health outcomes, user-centric matters, a… ▽ More The COVID-19 pandemic has led to a need for widespread and rapid vaccine development. As several vaccines have recently been approved for human use or are in different stages of development, governments across the world are preparing comprehensive guidelines for vaccine distribution and monitoring. In this early article, we identify challenges in logistics, health outcomes, user-centric matters, and communication associated with disease-related, individual, societal, economic, and privacy consequences. Primary challenges include difficulty in equitable distribution, vaccine efficacy, duration of immunity, multi-dose adherence, and privacy-focused record-keeping to be HIPAA compliant. While many of these challenges have been previously identified and addressed, some have not been acknowledged from a comprehensive view accounting for unprecedented interactions between challenges and specific populations. The logistics of equitable widespread vaccine distribution in disparate populations and countries of various economic, racial, and cultural constitutions must be thoroughly examined and accounted for. We also describe unique challenges regarding the efficacy of vaccines in specialized populations including children, the elderly, and immunocompromised individuals. Furthermore, we report the potential for understudied drug-vaccine interactions as well as the possibility that certain vaccine platforms may increase susceptibility to HIV. Given these complicated issues, the importance of privacy-focused, user-centric systems for vaccine education and incentivization along with clear communication from governments, organizations, and academic institutions is imperative. These challenges are by no means insurmountable, but require careful attention to avoid consequences spanning a range of disease-related, individual, societal, economic, and security domains. △ Less

Submitted 27 April, 2022; v1 submitted 24 November, 2020; originally announced December 2020.

Comments: 18 pages, 3 figures

Digital Landscape of COVID-19 Testing: Challenges and Opportunities

Authors: Darshan Gandhi, Rohan Sukumaran, Priyanshi Katiyar, Alex Radunsky, Sunaina Anand, Shailesh Advani, Jil Kothari, Kasia Jakimowicz, Sheshank Shankar, Sethuraman T. V., Krutika Misra, Aishwarya Saxena, Sanskruti Landage, Richa Sonker, Parth Patwa, Aryan Mahindra, Mikhail Dmitrienko, Kanishka Vaish, Ashley Mehra, Srinidhi Murali, Rohan Iyer, Joseph Bae, Vivek Sharma, Abhishek Singh, Rachel Barbar , et al. (1 additional authors not shown)

Abstract: The COVID-19 Pandemic has left a devastating trail all over the world, in terms of loss of lives, economic decline, travel restrictions, trade deficit, and collapsing economy including real-estate, job loss, loss of health benefits, the decline in quality of access to care and services and overall quality of life. Immunization from the anticipated vaccines will not be the stand-alone guideline tha… ▽ More The COVID-19 Pandemic has left a devastating trail all over the world, in terms of loss of lives, economic decline, travel restrictions, trade deficit, and collapsing economy including real-estate, job loss, loss of health benefits, the decline in quality of access to care and services and overall quality of life. Immunization from the anticipated vaccines will not be the stand-alone guideline that will help surpass the pandemic and return to normalcy. Four pillars of effective public health intervention include diagnostic testing for both asymptomatic and symptomatic individuals, contact tracing, quarantine of individuals with symptoms or who are exposed to COVID-19, and maintaining strict hygiene standards at the individual and community level. Digital technology, currently being used for COVID-19 testing include certain mobile apps, web dashboards, and online self-assessment tools. Herein, we look into various digital solutions adapted by communities across universities, businesses, and other organizations. We summarize the challenges experienced using these tools in terms of quality of information, privacy, and user-centric issues. Despite numerous digital solutions available and being developed, many vary in terms of information being shared in terms of both quality and quantity, which can be overwhelming to the users. Understanding the testing landscape through a digital lens will give a clear insight into the multiple challenges that we face including data privacy, cost, and miscommunication. It is the destiny of digitalization to navigate testing for COVID-19. Block-chain based systems can be used for privacy preservation and ensuring ownership of the data to remain with the user. Another solution involves having digital health passports with relevant and correct information. In this early draft, we summarize the challenges and propose possible solutions to address the same. △ Less

Submitted 3 December, 2020; originally announced December 2020.

Comments: 28 pages, 4 figures

AutoMat: Accelerated Computational Electrochemical systems Discovery

Authors: Emil Annevelink, Rachel Kurchin, Eric Muckley, Lance Kavalsky, Vinay I. Hegde, Valentin Sulzer, Shang Zhu, Jiankun Pu, David Farina, Matthew Johnson, Dhairya Gandhi, Adarsh Dave, Hongyi Lin, Alan Edelman, Bharath Ramsundar, James Saal, Christopher Rackauckas, Viral Shah, Bryce Meredig, Venkatasubramanian Viswanathan

Abstract: Large-scale electrification is vital to addressing the climate crisis, but several scientific and technological challenges remain to fully electrify both the chemical industry and transportation. In both of these areas, new electrochemical materials will be critical, but their development currently relies heavily on human-time-intensive experimental trial and error and computationally expensive fi… ▽ More Large-scale electrification is vital to addressing the climate crisis, but several scientific and technological challenges remain to fully electrify both the chemical industry and transportation. In both of these areas, new electrochemical materials will be critical, but their development currently relies heavily on human-time-intensive experimental trial and error and computationally expensive first-principles, meso-scale and continuum simulations. We present an automated workflow, AutoMat, that accelerates these computational steps by introducing both automated input generation and management of simulations across scales from first principles to continuum device modeling. Furthermore, we show how to seamlessly integrate multi-fidelity predictions such as machine learning surrogates or automated robotic experiments "in-the-loop". The automated framework is implemented with design space search techniques to dramatically accelerate the overall materials discovery pipeline by implicitly learning design features that optimize device performance across several metrics. We discuss the benefits of AutoMat using examples in electrocatalysis and energy storage and highlight lessons learned. △ Less

Submitted 13 May, 2022; v1 submitted 3 November, 2020; originally announced November 2020.

Comments: v1-3:4 pages, 1 figure, accepted to NeurIPS Climate Change and AI Workshop 2020, updating acknowledgements and citations v4: substantially updated content and author list, accepted to MRS Bulletin

Clinical Landscape of COVID-19 Testing: Difficult Choices

Authors: Darshan Gandhi, Sanskruti Landage, Joseph Bae, Sheshank Shankar, Rohan Sukumaran, Parth Patwa, Sethuraman T V, Priyanshi Katiyar, Shailesh Advani, Rohan Iyer, Sunaina Anand, Aryan Mahindra, Rachel Barbar, Abhishek Singh, Ramesh Raskar

Abstract: The coronavirus disease 2019 (COVID-19) pandemic has spread rapidly across the world, leading to enormous amounts of human death and economic loss. Until definitive preventive or curative measures are developed, policies regarding testing, contact tracing, and quarantine remain the best public health tools for curbing viral spread. Testing is a crucial component of these efforts, enabling the iden… ▽ More The coronavirus disease 2019 (COVID-19) pandemic has spread rapidly across the world, leading to enormous amounts of human death and economic loss. Until definitive preventive or curative measures are developed, policies regarding testing, contact tracing, and quarantine remain the best public health tools for curbing viral spread. Testing is a crucial component of these efforts, enabling the identification and isolation of infected individuals. Differences in testing methodologies, time frames, and outcomes can have an impact on their overall efficiency, usability and efficacy. In this early draft, we draw a comparison between the various types of diagnostic tests including PCR, antigen, and home tests in relation to their relative advantages, disadvantages, and use cases. We also look into alternative and unconventional methods. Further, we analyze the short-term and long-term impacts of the virus and its testing on various verticals such as business, government laws, policies, and healthcare. △ Less

Submitted 15 November, 2020; v1 submitted 9 November, 2020; originally announced November 2020.

Comments: 9 pages, 12 figures

Visual Imitation Made Easy

Authors: Sarah Young, Dhiraj Gandhi, Shubham Tulsiani, Abhinav Gupta, Pieter Abbeel, Lerrel Pinto

Abstract: Visual imitation learning provides a framework for learning complex manipulation behaviors by leveraging human demonstrations. However, current interfaces for imitation such as kinesthetic teaching or teleoperation prohibitively restrict our ability to efficiently collect large-scale data in the wild. Obtaining such diverse demonstration data is paramount for the generalization of learned skills t… ▽ More Visual imitation learning provides a framework for learning complex manipulation behaviors by leveraging human demonstrations. However, current interfaces for imitation such as kinesthetic teaching or teleoperation prohibitively restrict our ability to efficiently collect large-scale data in the wild. Obtaining such diverse demonstration data is paramount for the generalization of learned skills to novel scenarios. In this work, we present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. To extract action information from these visual demonstrations, we use off-the-shelf Structure from Motion (SfM) techniques in addition to training a finger detection network. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task. For both tasks, we use standard behavior cloning to learn executable policies from the previously collected offline demonstrations. To improve learning performance, we employ a variety of data augmentations and provide an extensive analysis of its effects. Finally, we demonstrate the utility of our interface by evaluating on real robotic scenarios with previously unseen objects and achieve a 87% success rate on pushing and a 62% success rate on stacking. Robot videos are available at https://dhiraj100892.github.io/Visual-Imitation-Made-Easy. △ Less

Submitted 11 August, 2020; originally announced August 2020.

Swoosh! Rattle! Thump! -- Actions that Sound

Authors: Dhiraj Gandhi, Abhinav Gupta, Lerrel Pinto

Abstract: Truly intelligent agents need to capture the interplay of all their senses to build a rich physical understanding of their world. In robotics, we have seen tremendous progress in using visual and tactile perception; however, we have often ignored a key sense: sound. This is primarily due to the lack of data that captures the interplay of action and sound. In this work, we perform the first large-s… ▽ More Truly intelligent agents need to capture the interplay of all their senses to build a rich physical understanding of their world. In robotics, we have seen tremendous progress in using visual and tactile perception; however, we have often ignored a key sense: sound. This is primarily due to the lack of data that captures the interplay of action and sound. In this work, we perform the first large-scale study of the interactions between sound and robotic action. To do this, we create the largest available sound-action-vision dataset with 15,000 interactions on 60 objects using our robotic platform Tilt-Bot. By tilting objects and allowing them to crash into the walls of a robotic tray, we collect rich four-channel audio information. Using this data, we explore the synergies between sound and action and present three key insights. First, sound is indicative of fine-grained object class information, e.g., sound can differentiate a metal screwdriver from a metal wrench. Second, sound also contains information about the causal effects of an action, i.e. given the sound produced, we can predict what action was applied to the object. Finally, object representations derived from audio embeddings are indicative of implicit physical properties. We demonstrate that on previously unseen objects, audio embeddings generated through interactions can predict forward models 24% better than passive visual embeddings. Project videos and data are at https://dhiraj100892.github.io/swoosh/ △ Less

Submitted 3 July, 2020; originally announced July 2020.

Comments: To be presented at Robotics: Science and Systems 2020

Object Goal Navigation using Goal-Oriented Semantic Exploration

Authors: Devendra Singh Chaplot, Dhiraj Gandhi, Abhinav Gupta, Ruslan Salakhutdinov

Abstract: This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. End-to-end learning-based navigation methods struggle at this task as they are ineffective at exploration and long-term planning. We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses… ▽ More This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. End-to-end learning-based navigation methods struggle at this task as they are ineffective at exploration and long-term planning. We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category. Empirical results in visually realistic simulation environments show that the proposed model outperforms a wide range of baselines including end-to-end learning-based methods as well as modular map-based methods and led to the winning entry of the CVPR-2020 Habitat ObjectNav Challenge. Ablation analysis indicates that the proposed model learns semantic priors of the relative arrangement of objects in a scene, and uses them to explore efficiently. Domain-agnostic module design allow us to transfer our model to a mobile robot platform and achieve similar performance for object goal navigation in the real-world. △ Less

Submitted 1 July, 2020; v1 submitted 1 July, 2020; originally announced July 2020.

Comments: Winner of the CVPR 2020 AI-Habitat Object Goal Navigation Challenge. See the project webpage at https://devendrachaplot.github.io/projects/semantic-exploration.html

Learning to Explore using Active Neural SLAM

Authors: Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, Ruslan Salakhutdinov

Abstract: This work presents a modular and hierarchical approach to learn policies for exploring 3D environments, called `Active Neural SLAM'. Our approach leverages the strengths of both classical and learning-based methods, by using analytical path planners with learned SLAM module, and global and local policies. The use of learning provides flexibility with respect to input modalities (in the SLAM module… ▽ More This work presents a modular and hierarchical approach to learn policies for exploring 3D environments, called `Active Neural SLAM'. Our approach leverages the strengths of both classical and learning-based methods, by using analytical path planners with learned SLAM module, and global and local policies. The use of learning provides flexibility with respect to input modalities (in the SLAM module), leverages structural regularities of the world (in global policies), and provides robustness to errors in state estimation (in local policies). Such use of learning within each module retains its benefits, while at the same time, hierarchical decomposition and modular training allow us to sidestep the high sample complexities associated with training end-to-end policies. Our experiments in visually and physically realistic simulated 3D environments demonstrate the effectiveness of our approach over past learning and geometry-based approaches. The proposed model can also be easily transferred to the PointGoal task and was the winning entry of the CVPR 2019 Habitat PointGoal Navigation Challenge. △ Less

Submitted 10 April, 2020; originally announced April 2020.

Comments: Published in ICLR-2020. See the project webpage at https://devendrachaplot.github.io/projects/Neural-SLAM for supplementary videos. The code is available at https://github.com/devendrachaplot/Neural-SLAM

Object-centric Forward Modeling for Model Predictive Control

Authors: Yufei Ye, Dhiraj Gandhi, Abhinav Gupta, Shubham Tulsiani

Abstract: We present an approach to learn an object-centric forward model, and show that this allows us to plan for sequences of actions to achieve distant desired goals. We propose to model a scene as a collection of objects, each with an explicit spatial location and implicit visual feature, and learn to model the effects of actions using random interaction data. Our model allows capturing the robot-objec… ▽ More We present an approach to learn an object-centric forward model, and show that this allows us to plan for sequences of actions to achieve distant desired goals. We propose to model a scene as a collection of objects, each with an explicit spatial location and implicit visual feature, and learn to model the effects of actions using random interaction data. Our model allows capturing the robot-object and object-object interactions, and leads to more sample-efficient and accurate predictions. We show that this learned model can be leveraged to search for action sequences that lead to desired goal configurations, and that in conjunction with a learned correction module, this allows for robust closed loop execution. We present experiments both in simulation and the real world, and show that our approach improves over alternate implicit or pixel-space forward models. Please see our project page (https://judyye.github.io/ocmpc/) for result videos. △ Less

Submitted 8 October, 2019; originally announced October 2019.

Overview of Guidance, Navigation and Control System of the TeamIndus lunar lander

Authors: Vishesh Vatsal, C. Barath, J. Yogeshwaran, Deepana Gandhi, Chhavilata Sahu, Karthic Balasubramanian, Shyam Mohan, Midhun S. Menon, P. Natarajan, Vivek Raghavan

Abstract: TeamIndus' lunar logistics vision includes multiple lunar missions to meet requirements of science, commercial and efforts towards global exploration. The first mission is slated for launch in 2020. The prime objective is to demonstrate autonomous precision lunar landing, and Surface Exploration Rover to collect data on the vicinity of the landing site. TeamIndus has developed various technologies… ▽ More TeamIndus' lunar logistics vision includes multiple lunar missions to meet requirements of science, commercial and efforts towards global exploration. The first mission is slated for launch in 2020. The prime objective is to demonstrate autonomous precision lunar landing, and Surface Exploration Rover to collect data on the vicinity of the landing site. TeamIndus has developed various technologies towards lowering the access barrier to the lunar surface. This paper shall provide an overview of design of lander GNC system. The design of the GNC system has been described after concluding studies on sensor and actuator configurations. Frugal design approach is followed in the selection of GNC hardware. The paper describes the constraints for the orbital maneuvers and the lunar descent strategy. Various aspects of the GNC design of autonomous lunar descent maneuver: timeline of events, guidance, inertial and optical terrain-relative navigation schemes are described. The GNC software description focuses on system architecture, modes of operation, and core elements of the GNC software. The GNC algorithms have been tested using Monte-Carlo simulations and Processor-in-Loop runs. The paper concludes with a summary of key risk-mitigation studies for soft landing. △ Less

Submitted 25 July, 2019; originally announced July 2019.

PyRobot: An Open-source Robotics Framework for Research and Benchmarking

Authors: Adithyavairavan Murali, Tao Chen, Kalyan Vasudev Alwala, Dhiraj Gandhi, Lerrel Pinto, Saurabh Gupta, Abhinav Gupta

Abstract: This paper introduces PyRobot, an open-source robotics framework for research and benchmarking. PyRobot is a light-weight, high-level interface on top of ROS that provides a consistent set of hardware independent mid-level APIs to control different robots. PyRobot abstracts away details about low-level controllers and inter-process communication, and allows non-robotics researchers (ML, CV researc… ▽ More This paper introduces PyRobot, an open-source robotics framework for research and benchmarking. PyRobot is a light-weight, high-level interface on top of ROS that provides a consistent set of hardware independent mid-level APIs to control different robots. PyRobot abstracts away details about low-level controllers and inter-process communication, and allows non-robotics researchers (ML, CV researchers) to focus on building high-level AI applications. PyRobot aims to provide a research ecosystem with convenient access to robotics datasets, algorithm implementations and models that can be used to quickly create a state-of-the-art baseline. We believe PyRobot, when paired up with low-cost robot platforms such as LoCoBot, will reduce the entry barrier into robotics, and democratize robotics. PyRobot is open-source, and can be accessed via https://pyrobot.org. △ Less

Submitted 19 June, 2019; originally announced June 2019.

Self-Supervised Exploration via Disagreement

Authors: Deepak Pathak, Dhiraj Gandhi, Abhinav Gupta

Abstract: Efficient exploration is a long-standing problem in sensorimotor learning. Major advances have been demonstrated in noise-free, non-stochastic domains such as video games and simulation. However, most of these formulations either get stuck in environments with stochastic dynamics or are too inefficient to be scalable to real robotics setups. In this paper, we propose a formulation for exploration… ▽ More Efficient exploration is a long-standing problem in sensorimotor learning. Major advances have been demonstrated in noise-free, non-stochastic domains such as video games and simulation. However, most of these formulations either get stuck in environments with stochastic dynamics or are too inefficient to be scalable to real robotics setups. In this paper, we propose a formulation for exploration inspired by the work in active learning literature. Specifically, we train an ensemble of dynamics models and incentivize the agent to explore such that the disagreement of those ensembles is maximized. This allows the agent to learn skills by exploring in a self-supervised manner without any external reward. Notably, we further leverage the disagreement objective to optimize the agent's policy in a differentiable manner, without using reinforcement learning, which results in a sample-efficient exploration. We demonstrate the efficacy of this formulation across a variety of benchmark environments including stochastic-Atari, Mujoco and Unity. Finally, we implement our differentiable exploration on a real robot which learns to interact with objects completely from scratch. Project videos and code are at https://pathak22.github.io/exploration-by-disagreement/ △ Less

Submitted 10 June, 2019; originally announced June 2019.

Comments: Accepted at ICML 2019. Website at https://pathak22.github.io/exploration-by-disagreement/

Fashionable Modelling with Flux

Authors: Michael Innes, Elliot Saba, Keno Fischer, Dhairya Gandhi, Marco Concetto Rudilosso, Neethu Mariya Joy, Tejan Karmali, Avik Pal, Viral Shah

Abstract: Machine learning as a discipline has seen an incredible surge of interest in recent years due in large part to a perfect storm of new theory, superior tooling, renewed interest in its capabilities. We present in this paper a framework named Flux that shows how further refinement of the core ideas of machine learning, built upon the foundation of the Julia programming language, can yield an environ… ▽ More Machine learning as a discipline has seen an incredible surge of interest in recent years due in large part to a perfect storm of new theory, superior tooling, renewed interest in its capabilities. We present in this paper a framework named Flux that shows how further refinement of the core ideas of machine learning, built upon the foundation of the Julia programming language, can yield an environment that is simple, easily modifiable, and performant. We detail the fundamental principles of Flux as a framework for differentiable programming, give examples of models that are implemented within Flux to display many of the language and framework-level features that contribute to its ease of use and high productivity, display internal compiler techniques used to enable the acceleration and performance that lies at the heart of Flux, and finally give an overview of the larger ecosystem that Flux fits inside of. △ Less

Submitted 10 November, 2018; v1 submitted 31 October, 2018; originally announced November 2018.

Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias

Authors: Abhinav Gupta, Adithyavairavan Murali, Dhiraj Gandhi, Lerrel Pinto

Abstract: Data-driven approaches to solving robotic tasks have gained a lot of traction in recent years. However, most existing policies are trained on large-scale datasets collected in curated lab settings. If we aim to deploy these models in unstructured visual environments like people's homes, they will be unable to cope with the mismatch in data distribution. In such light, we present the first systemat… ▽ More Data-driven approaches to solving robotic tasks have gained a lot of traction in recent years. However, most existing policies are trained on large-scale datasets collected in curated lab settings. If we aim to deploy these models in unstructured visual environments like people's homes, they will be unable to cope with the mismatch in data distribution. In such light, we present the first systematic effort in collecting a large dataset for robotic grasping in homes. First, to scale and parallelize data collection, we built a low cost mobile manipulator assembled for under 3K USD. Second, data collected using low cost robots suffer from noisy labels due to imperfect execution and calibration errors. To handle this, we develop a framework which factors out the noise as a latent variable. Our model is trained on 28K grasps collected in several houses under an array of different environmental conditions. We evaluate our models by physically executing grasps on a collection of novel objects in multiple unseen homes. The models trained with our home dataset showed a marked improvement of 43.7% over a baseline model trained with data collected in lab. Our architecture which explicitly models the latent noise in the dataset also performed 10% better than one that did not factor out the noise. We hope this effort inspires the robotics community to look outside the lab and embrace learning based approaches to handle inaccurate cheap robots. △ Less

Submitted 18 July, 2018; originally announced July 2018.

Learning to Grasp Without Seeing

Authors: Adithyavairavan Murali, Yin Li, Dhiraj Gandhi, Abhinav Gupta

Abstract: Can a robot grasp an unknown object without seeing it? In this paper, we present a tactile-sensing based approach to this challenging problem of grasping novel objects without prior knowledge of their location or physical properties. Our key idea is to combine touch based object localization with tactile based re-grasping. To train our learning models, we created a large-scale grasping dataset, in… ▽ More Can a robot grasp an unknown object without seeing it? In this paper, we present a tactile-sensing based approach to this challenging problem of grasping novel objects without prior knowledge of their location or physical properties. Our key idea is to combine touch based object localization with tactile based re-grasping. To train our learning models, we created a large-scale grasping dataset, including more than 30 RGB frames and over 2.8 million tactile samples from 7800 grasp interactions of 52 objects. To learn a representation of tactile signals, we propose an unsupervised auto-encoding scheme, which shows a significant improvement of 4-9% over prior methods on a variety of tactile perception tasks. Our system consists of two steps. First, our touch localization model sequentially 'touch-scans' the workspace and uses a particle filter to aggregate beliefs from multiple hits of the target. It outputs an estimate of the object's location, from which an initial grasp is established. Next, our re-grasping model learns to progressively improve grasps with tactile feedback based on the learned features. This network learns to estimate grasp stability and predict adjustment for the next grasp. Re-grasping thus is performed iteratively until our model identifies a stable grasp. Finally, we demonstrate extensive experimental results on grasping a large set of novel objects using tactile sensing alone. Furthermore, when applied on top of a vision-based policy, our re-grasping model significantly boosts the overall accuracy by 10.6%. We believe this is the first attempt at learning to grasp with only tactile sensing and without any prior object knowledge. △ Less

Submitted 10 May, 2018; originally announced May 2018.

CASSL: Curriculum Accelerated Self-Supervised Learning

Authors: Adithyavairavan Murali, Lerrel Pinto, Dhiraj Gandhi, Abhinav Gupta

Abstract: Recent self-supervised learning approaches focus on using a few thousand data points to learn policies for high-level, low-dimensional action spaces. However, scaling this framework for high-dimensional control require either scaling up the data collection efforts or using a clever sampling strategy for training. We present a novel approach - Curriculum Accelerated Self-Supervised Learning (CASSL)… ▽ More Recent self-supervised learning approaches focus on using a few thousand data points to learn policies for high-level, low-dimensional action spaces. However, scaling this framework for high-dimensional control require either scaling up the data collection efforts or using a clever sampling strategy for training. We present a novel approach - Curriculum Accelerated Self-Supervised Learning (CASSL) - to train policies that map visual information to high-level, higher- dimensional action spaces. CASSL orders the sampling of training data based on control dimensions: the learning and sampling are focused on few control parameters before other parameters. The right curriculum for learning is suggested by variance-based global sensitivity analysis of the control space. We apply our CASSL framework to learning how to grasp using an adaptive, underactuated multi-fingered gripper, a challenging system to control. Our experimental results indicate that CASSL provides significant improvement and generalization compared to baseline methods such as staged curriculum learning (8% increase) and complete end-to-end learning with random exploration (14% improvement) tested on a set of novel objects. △ Less

Submitted 12 February, 2018; v1 submitted 3 August, 2017; originally announced August 2017.

Learning to Fly by Crashing

Authors: Dhiraj Gandhi, Lerrel Pinto, Abhinav Gupta

Abstract: How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? One approach is to use a small dataset collected by human experts: however, high capacity learning algorithms tend to overfit when trained with little data. An alternative is to use simulation. But the gap between simulation and real world remains large especially for perception problems. The reason most research av… ▽ More How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? One approach is to use a small dataset collected by human experts: however, high capacity learning algorithms tend to overfit when trained with little data. An alternative is to use simulation. But the gap between simulation and real world remains large especially for perception problems. The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone 11,500 times to create one of the biggest UAV crash dataset. This dataset captures the different ways in which a UAV can crash. We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation. We show that this simple self-supervised model is quite effective in navigating the UAV even in extremely cluttered environments with dynamic obstacles including humans. For supplementary video see: https://youtu.be/u151hJaGKUo △ Less

Submitted 26 April, 2017; v1 submitted 18 April, 2017; originally announced April 2017.

The Curious Robot: Learning Visual Representations via Physical Interactions

Authors: Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta

Abstract: What is the right supervisory signal to train visual representations? Current approaches in computer vision use category labels from datasets such as ImageNet to train ConvNets. However, in case of biological agents, visual representation learning does not require millions of semantic labels. We argue that biological agents use physical interactions with the world to learn visual representations u… ▽ More What is the right supervisory signal to train visual representations? Current approaches in computer vision use category labels from datasets such as ImageNet to train ConvNets. However, in case of biological agents, visual representation learning does not require millions of semantic labels. We argue that biological agents use physical interactions with the world to learn visual representations unlike current vision systems which just use passive observations (images and videos downloaded from web). For example, babies push objects, poke them, put them in their mouth and throw them to learn representations. Towards this goal, we build one of the first systems on a Baxter platform that pushes, pokes, grasps and observes objects in a tabletop environment. It uses four different types of physical interactions to collect more than 130K datapoints, with each datapoint providing supervision to a shared ConvNet architecture allowing us to learn visual representations. We show the quality of learned representations by observing neuron activations and performing nearest neighbor retrieval on this learned representation. Quantitatively, we evaluate our learned ConvNet on image classification tasks and show improvements compared to learning without external data. Finally, on the task of instance retrieval, our network outperforms the ImageNet network on recall@1 by 3% △ Less

Submitted 25 July, 2016; v1 submitted 5 April, 2016; originally announced April 2016.