Showing 1–50 of 66 results for author: Subramanian, G

Confidence Aware Inverse Constrained Reinforcement Learning

Authors: Sriram Ganapathi Subramanian, Guiliang Liu, Mohammed Elmahgiubi, Kasra Rezaee, Pascal Poupart

Abstract: In coming up with solutions to real-world problems, humans implicitly adhere to constraints that are too numerous and complex to be specified completely. However, reinforcement learning (RL) agents need these constraints to learn the correct optimal policy in these settings. The field of Inverse Constraint Reinforcement Learning (ICRL) deals with this problem and provides algorithms that aim to es… ▽ More In coming up with solutions to real-world problems, humans implicitly adhere to constraints that are too numerous and complex to be specified completely. However, reinforcement learning (RL) agents need these constraints to learn the correct optimal policy in these settings. The field of Inverse Constraint Reinforcement Learning (ICRL) deals with this problem and provides algorithms that aim to estimate the constraints from expert demonstrations collected offline. Practitioners prefer to know a measure of confidence in the estimated constraints, before deciding to use these constraints, which allows them to only use the constraints that satisfy a desired level of confidence. However, prior works do not allow users to provide the desired level of confidence for the inferred constraints. This work provides a principled ICRL method that can take a confidence level with a set of expert demonstrations and outputs a constraint that is at least as constraining as the true underlying constraint with the desired level of confidence. Further, unlike previous methods, this method allows a user to know if the number of expert trajectories is insufficient to learn a constraint with a desired level of confidence, and therefore collect more expert trajectories as required to simultaneously learn constraints with the desired level of confidence and a policy that achieves the desired level of performance. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: Paper to appear in ICML 2024

How Useful is Intermittent, Asynchronous Expert Feedback for Bayesian Optimization?

Authors: Agustinus Kristiadi, Felix Strieth-Kalthoff, Sriram Ganapathi Subramanian, Vincent Fortuin, Pascal Poupart, Geoff Pleiss

Abstract: Bayesian optimization (BO) is an integral part of automated scientific discovery -- the so-called self-driving lab -- where human inputs are ideally minimal or at least non-blocking. However, scientists often have strong intuition, and thus human feedback is still useful. Nevertheless, prior works in enhancing BO with expert feedback, such as by incorporating it in an offline or online but blockin… ▽ More Bayesian optimization (BO) is an integral part of automated scientific discovery -- the so-called self-driving lab -- where human inputs are ideally minimal or at least non-blocking. However, scientists often have strong intuition, and thus human feedback is still useful. Nevertheless, prior works in enhancing BO with expert feedback, such as by incorporating it in an offline or online but blocking (arrives at each BO iteration) manner, are incompatible with the spirit of self-driving labs. In this work, we study whether a small amount of randomly arriving expert feedback that is being incorporated in a non-blocking manner can improve a BO campaign. To this end, we run an additional, independent computing thread on top of the BO loop to handle the feedback-gathering process. The gathered feedback is used to learn a Bayesian preference model that can readily be incorporated into the BO thread, to steer its exploration-exploitation process. Experiments on toy and chemistry datasets suggest that even just a few intermittent, asynchronous expert feedback can be useful for improving or constraining BO. This can especially be useful for its implication in improving self-driving labs, e.g. making them more data-efficient and less costly. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: AABI 2024. Code: https://github.com/wiseodd/bo-async-feedback

A viscous drop in a planar linear flow -- the role of deformation on streamline topology

Authors: Sabarish V. Narayanan, Ganesh Subramanian

Abstract: Planar linear flows are a one-parameter family, with the parameter $\hatα\in [-1,1]$ being a measure of the relative magnitudes of extension and vorticity; $\hatα = -1$, $0$ and $1$ correspond to solid-body rotation, simple shear flow and planar extension, respectively. For a neutrally buoyant spherical drop in a hyperbolic planar linear flow with $\hatα\in(0,1]$, the near-field streamlines are cl… ▽ More Planar linear flows are a one-parameter family, with the parameter $\hatα\in [-1,1]$ being a measure of the relative magnitudes of extension and vorticity; $\hatα = -1$, $0$ and $1$ correspond to solid-body rotation, simple shear flow and planar extension, respectively. For a neutrally buoyant spherical drop in a hyperbolic planar linear flow with $\hatα\in(0,1]$, the near-field streamlines are closed for $0 \leq \hatα < 1$ and for $λ> λ_c = 2 \hatα / (1 - \hatα)$, $λ$ being the drop-to-medium viscosity ratio; all streamlines are closed for an ambient elliptic linear flow with $\hatα\in[-1,0)$. We use both analytical and numerical tools to show that drop deformation, as characterized by a non-zero capillary number ($Ca$), destroys the aforementioned closed-streamline topology. While inertia has previously been shown to transform closed Stokesian streamlines into open spiraling ones that run from upstream to downstream infinity, the streamline topology around a deformed drop, for small but finite $Ca$, is more complicated. Only a subset of the original closed streamlines transforms to open spiraling ones, while the remaining ones densely wind around a configuration of nested invariant tori. Our results contradict previous efforts pointing to the persistence of the closed streamline topology exterior to a deformed drop and have important implications for transport and mixing. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Scaling Data Plane Verification with Intent-based Slicing

Authors: Kuan-Yen Chou, Santhosh Prabhu, Giri Subramanian, Wenxuan Zhou, Aanand Nayyar, Brighten Godfrey, Matthew Caesar

Abstract: Data plane verification has grown into a powerful tool to ensure network correctness. However, existing monolithic data plane models have high memory requirements with large networks, and the existing method of scaling out is too limited in expressiveness to capture practical network features. In this paper, we describe Scylla, a general data plane verifier that provides fine-grained scale-out wit… ▽ More Data plane verification has grown into a powerful tool to ensure network correctness. However, existing monolithic data plane models have high memory requirements with large networks, and the existing method of scaling out is too limited in expressiveness to capture practical network features. In this paper, we describe Scylla, a general data plane verifier that provides fine-grained scale-out without the need for a monolithic network model. Scylla creates models for what we call intent-based slices, each of which is constructed at a fine (rule-level) granularity with just enough to verify a given set of intents. The sliced models are retained in memory across a cluster and are incrementally updated in a distributed compute cluster in response to network updates. Our experiments show that Scylla makes the scaling problem more granular -- tied to the size of the intent-based slices rather than that of the overall network. This enables Scylla to verify large, complex networks in minimum units of work that are significantly smaller (in both memory and time) than past techniques, enabling fast scale-out verification with minimal resource requirement. △ Less

Submitted 31 May, 2024; originally announced May 2024.

Information Compression in Dynamic Information Disclosure Games

Authors: Dengwang Tang, Vijay G. Subramanian

Abstract: We consider a two-player dynamic information design problem between a principal and a receiver -- a game is played between the two agents on top of a Markovian system controlled by the receiver's actions, where the principal obtains and strategically shares some information about the underlying system with the receiver in order to influence their actions. In our setting, both players have long-ter… ▽ More We consider a two-player dynamic information design problem between a principal and a receiver -- a game is played between the two agents on top of a Markovian system controlled by the receiver's actions, where the principal obtains and strategically shares some information about the underlying system with the receiver in order to influence their actions. In our setting, both players have long-term objectives, and the principal sequentially commits to their strategies instead of committing at the beginning. Further, the principal cannot directly observe the system state, but at every turn they can choose randomized experiments to observe the system partially. The principal can share details about the experiments to the receiver. For our analysis we impose the truthful disclosure rule: the principal is required to truthfully announce the details and the result of each experiment to the receiver immediately after the experiment result is revealed. Based on the received information, the receiver takes an action when its their turn, with the action influencing the state of the underlying system. We show that there exist Perfect Bayesian equilibria in this game where both agents play Canonical Belief Based (CBB) strategies using a compressed version of their information, rather than full information, to choose experiments (for the principal) or actions (for the receiver). We also provide a backward inductive procedure to solve for an equilibrium in CBB strategies. △ Less

Submitted 18 March, 2024; originally announced March 2024.

Comments: 14 pages, 5 figures

Supporting the next generation lithography roadmap using partial state-feedback reduced-order switching predictive models

Authors: Raaja Ganapathy Subramanian, Barry Moest, Bart Paarhuis

Abstract: To support the ever-increasing performance requirements of lithography systems in terms of throughput and accuracy, in this paper, we introduce a design framework for partial state-feedback reduced-order switching predictive models. By combining measurements and predictions, this method aims to: 1) improve overall system performance by reducing the placement errors in a die within and across the f… ▽ More To support the ever-increasing performance requirements of lithography systems in terms of throughput and accuracy, in this paper, we introduce a design framework for partial state-feedback reduced-order switching predictive models. By combining measurements and predictions, this method aims to: 1) improve overall system performance by reducing the placement errors in a die within and across the full-wafer and 2) eliminate redundant measurements by using predictions to improve system throughput. We primarily focus on well-known trade-off introduced by measurement time, which can correct errors at a cost of throughput, noise and not being robust to uncertainties. The proposed approach consists of a reduced-order model with a switching logic that acts a scheduler to deal with uncertain operating conditions. It also utilizes linear predictive models as a basis for the control design which is appealing to the ease and cost of implementation, enhancing the applicability. For the add-on part, the scheduler logic is adapted based on expected operating conditions of the system while guaranteeing global uniform ultimate bounded asymptotic stability. Lastly, to deal with measurement layouts, the predictor combines the measurement into model using partial state-feedback. Effectiveness of the proposed strategy is demonstrated in practice on a high-precision industrial scanner. △ Less

Submitted 20 March, 2024; v1 submitted 21 February, 2024; originally announced February 2024.

Comments: 9 pages, 9 figures, to be submitted in IEEE-TIE

Disposable face masks: a direct source for inhalation of microplastics

Authors: Andres F. Prada, Avram Distler, Shyuan Cheng, John W. Scott, Leonardo P. Chamorro, Ganesh Subramanian, Vishal Verma, Andrew Turner

Abstract: Surgical masks have played a crucial role in healthcare facilities to protect against respiratory and infectious diseases, particularly during the COVID-19 pandemic. However, the synthetic fibers, mainly made of polypropylene, used in their production may adversely affect the environment and human health. Recent studies have confirmed the presence of microplastics and fibers in human lungs and hav… ▽ More Surgical masks have played a crucial role in healthcare facilities to protect against respiratory and infectious diseases, particularly during the COVID-19 pandemic. However, the synthetic fibers, mainly made of polypropylene, used in their production may adversely affect the environment and human health. Recent studies have confirmed the presence of microplastics and fibers in human lungs and have related these synthetic particles with the occurrence of pulmonary ground glass nodules. Using a piston system to simulate human breathing, this study investigates the role of surgical masks as a direct source of inhalation of microplastics. Results reveal the release of particles of sizes ranging from nanometers (300 nm) to millimeters (~2 mm) during normal breathing conditions, raising concerns about the potential health risks. Notably, large visible particles (> 1 mm) were observed to be ejected from masks with limited wear after only a few breathing cycles. Given the widespread use of masks by healthcare workers and the potential future need for mask usage by the general population during seasonal infectious diseases or new pandemics, developing face masks using safe materials for both users and the environment is imperative. △ Less

Submitted 30 August, 2023; originally announced August 2023.

Comments: 11 pages, 3 figures

ChemGymRL: An Interactive Framework for Reinforcement Learning for Digital Chemistry

Authors: Chris Beeler, Sriram Ganapathi Subramanian, Kyle Sprague, Nouha Chatti, Colin Bellinger, Mitchell Shahen, Nicholas Paquin, Mark Baula, Amanuel Dawit, Zihan Yang, Xinkai Li, Mark Crowley, Isaac Tamblyn

Abstract: This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents `on-the-fly' by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to wor… ▽ More This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents `on-the-fly' by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to work in. We introduce a set of highly customizable and open-source RL environments, ChemGymRL, based on the standard Open AI Gym template. ChemGymRL supports a series of interconnected virtual chemical benches where RL agents can operate and train. The paper introduces and details each of these benches using well-known chemical reactions as illustrative examples, and trains a set of standard RL algorithms in each of these benches. Finally, discussion and comparison of the performances of several standard RL methods are provided in addition to a list of directions for future work as a vision for the further development and usage of ChemGymRL. △ Less

Submitted 23 May, 2023; originally announced May 2023.

Comments: 19 pages, 13 figures, 2 tables

Inertial migration of a neutrally buoyant spheroid in plane Poiseuille flow

Authors: Prateek Anand, Ganesh Subramanian

Abstract: We study the cross-stream inertial migration of a torque-free neutrally buoyant spheroid, of an arbitrary aspect ratio $κ$, in wall-bounded plane Poiseuille flow for small particle Reynolds numbers\,($Re_p\ll1$) and confinement ratios\,($λ\ll1$), with the channel Reynolds number, $Re_c = Re_p/λ^2$, assumed to be arbitrary; here, $λ=L/H$ where $L$ is the semi-major axis of the spheroid and $H$ deno… ▽ More We study the cross-stream inertial migration of a torque-free neutrally buoyant spheroid, of an arbitrary aspect ratio $κ$, in wall-bounded plane Poiseuille flow for small particle Reynolds numbers\,($Re_p\ll1$) and confinement ratios\,($λ\ll1$), with the channel Reynolds number, $Re_c = Re_p/λ^2$, assumed to be arbitrary; here, $λ=L/H$ where $L$ is the semi-major axis of the spheroid and $H$ denotes the separation between the channel walls. In the Stokes limit\,($Re_p =0)$ and for $λ\ll 1$, a spheroid rotates along any of an infinite number of Jeffery orbits parameterized by an orbit constant $C$, while translating with a time dependent speed along a given ambient streamline. Weak inertial effects stabilize either the spinning\,($C=0$) or the tumbling orbit\,($C=\infty$), or both, depending on $κ$. The separation of the Jeffery-rotation and orbital drift time scales, from that associated with cross-stream migration, implies that the latter occurs due to a Jeffery-averaged lift velocity. Although the magnitude of this averaged lift velocity depends on $κ$ and $C$, the shape of the lift profiles are identical to those for a sphere, regardless of $Re_c$. In particular, the equilibrium positions for a spheroid remain identical to the classical Segre-Silberberg ones for a sphere, starting off at a distance of about $0.6(H/2)$ from the channel centerline for small $Re_c$, and migrating wallward with increasing $Re_c$. For spheroids with $κ\sim O(1)$, the Jeffery-averaged analysis is valid for $Re_p\ll1$; for extreme aspect ratio spheroids, the regime of validity becomes more restrictive being given by $Re_p\,κ/\ln κ\ll 1$ and $Re_p/κ^2 \ll 1$ for $κ\rightarrow \infty$\,(slender fibers) and $κ\rightarrow 0$\,(flat disks), respectively. △ Less

Submitted 28 February, 2023; originally announced March 2023.

Learning from Multiple Independent Advisors in Multi-agent Reinforcement Learning

Authors: Sriram Ganapathi Subramanian, Matthew E. Taylor, Kate Larson, Mark Crowley

Abstract: Multi-agent reinforcement learning typically suffers from the problem of sample inefficiency, where learning suitable policies involves the use of many data samples. Learning from external demonstrators is a possible solution that mitigates this problem. However, most prior approaches in this area assume the presence of a single demonstrator. Leveraging multiple knowledge sources (i.e., advisors)… ▽ More Multi-agent reinforcement learning typically suffers from the problem of sample inefficiency, where learning suitable policies involves the use of many data samples. Learning from external demonstrators is a possible solution that mitigates this problem. However, most prior approaches in this area assume the presence of a single demonstrator. Leveraging multiple knowledge sources (i.e., advisors) with expertise in distinct aspects of the environment could substantially speed up learning in complex environments. This paper considers the problem of simultaneously learning from multiple independent advisors in multi-agent reinforcement learning. The approach leverages a two-level Q-learning architecture, and extends this framework from single-agent to multi-agent settings. We provide principled algorithms that incorporate a set of advisors by both evaluating the advisors at each state and subsequently using the advisors to guide action selection. We also provide theoretical convergence and sample complexity guarantees. Experimentally, we validate our approach in three different test-beds and show that our algorithms give better performances than baselines, can effectively integrate the combined expertise of different advisors, and learn to ignore bad advice. △ Less

Submitted 2 March, 2023; v1 submitted 26 January, 2023; originally announced January 2023.

Comments: Paper to appear in AAMAS 2023, London, UK

Scalar transport from deformed drops: the singular role of streamline topology

Authors: Pavan Kumar Singeetham, Sumesh P Thampi, Ganesh Subramanian

Abstract: We examine scalar transport from a neutrally buoyant drop, in an ambient planar extensional flow, in the limit of a dominant drop phase resistance. For this interior problem, we consider the effect of drop-deformation-induced change in streamline topology on the transport rate (the Nusselt number $Nu$). The importance of drop deformation is characterized by the Capillary number ($Ca$). For a spher… ▽ More We examine scalar transport from a neutrally buoyant drop, in an ambient planar extensional flow, in the limit of a dominant drop phase resistance. For this interior problem, we consider the effect of drop-deformation-induced change in streamline topology on the transport rate (the Nusselt number $Nu$). The importance of drop deformation is characterized by the Capillary number ($Ca$). For a spherical drop ($Ca = 0$), closed streamlines lead to the ratio $Nu/Nu_0$ increasing with the Peclet number($Pe$), from unity to a diffusion-limited plateau value ($\approx 4.1$); $Nu_0$ here denotes the purely diffusive rate of transport. For any finite $Ca$, the flow field consists of spiralling streamlines that densely wind around nested tori foliating the deformed drop interior. $Nu$ now increases beyond the aforementioned primary plateau, saturating in a secondary plateau that approaches $23.3$ for $Ca \rightarrow 0$, $Pe Ca \rightarrow \infty$, and appears independent of the drop-to-medium viscosity ratio. $Nu/Nu_0$ exhibits an analogous variation for other planar linear flows, although chaotically wandering streamlines in these cases are expected to lead to a tertiary enhancement regime. △ Less

Submitted 12 January, 2023; originally announced January 2023.

Inertial migration in pressure-driven channel flow: beyond the Segre-Silberberg pinch

Authors: Prateek Anand, Ganesh Subramanian

Abstract: We examine theoretically the inertial migration of a neutrally buoyant rigid sphere in pressure-driven channel flow, accounting for its finite size relative to the channel width (the confinement ratio). For sufficiently large channel Reynolds numbers\,($Re_c$), a small but finite confinement ratio qualitatively alters the inertial lift velocity profiles obtained using a point-particle formulation.… ▽ More We examine theoretically the inertial migration of a neutrally buoyant rigid sphere in pressure-driven channel flow, accounting for its finite size relative to the channel width (the confinement ratio). For sufficiently large channel Reynolds numbers\,($Re_c$), a small but finite confinement ratio qualitatively alters the inertial lift velocity profiles obtained using a point-particle formulation. Finite size effects are shown to lead to new equilibria, in addition to the well known Segre-Silberberg pinch locations. Consequently, a sphere can migrate to either the near-wall Segre-Silberberg equilibria, or the new stable equilibria located closer to the channel centerline, depending on $Re_c$ and its initial position. Our findings are in accord with recent experiments and simulations, and have implications for passive sorting of particles based on size, shape and other physical characteristics, in microfluidic applications. △ Less

Submitted 2 January, 2023; originally announced January 2023.

Learning-based Optimal Admission Control in a Single Server Queuing System

Authors: Asaf Cohen, Vijay G. Subramanian, Yili Zhang

Abstract: We consider a long-term average profit maximizing admission control problem in an M/M/1 queuing system with unknown service and arrival rates. With a fixed reward collected upon service completion and a cost per unit of time enforced on customers waiting in the queue, a dispatcher decides upon arrivals whether to admit the arriving customer or not based on the full history of observations of the q… ▽ More We consider a long-term average profit maximizing admission control problem in an M/M/1 queuing system with unknown service and arrival rates. With a fixed reward collected upon service completion and a cost per unit of time enforced on customers waiting in the queue, a dispatcher decides upon arrivals whether to admit the arriving customer or not based on the full history of observations of the queue-length of the system. (Naor 1969, Econometrica) showed that if all the parameters of the model are known, then it is optimal to use a static threshold policy -- admit if the queue-length is less than a predetermined threshold and otherwise not. We propose a learning-based dispatching algorithm and characterize its regret with respect to optimal dispatch policies for the full information model of Naor (1969). We show that the algorithm achieves an $O(1)$ regret when all optimal thresholds with full information are non-zero, and achieves an $O(\ln^{1+ε}(N))$ regret for any specified $ε>0$, in the case that an optimal threshold with full information is $0$ (i.e., an optimal policy is to reject all arrivals), where $N$ is the number of arrivals. △ Less

Submitted 23 November, 2023; v1 submitted 21 December, 2022; originally announced December 2022.

Anomalous dispersion of microswimmer populations

Authors: Laxminarsimharao Vennamneni, Piyush Garg, Ganesh Subramanian

Abstract: We examine the longitudinal dispersion of spheroidal microswimmers in pressure-driven channel flow. When time scales corresponding to swimmer orientation relaxation, and diffusion in the gradient and flow directions, are well separated, a multiple scales analysis leads to the shear-enhanced diffusivity governing the long-time spread of the swimmer population along the flow\,(longitudinal) directio… ▽ More We examine the longitudinal dispersion of spheroidal microswimmers in pressure-driven channel flow. When time scales corresponding to swimmer orientation relaxation, and diffusion in the gradient and flow directions, are well separated, a multiple scales analysis leads to the shear-enhanced diffusivity governing the long-time spread of the swimmer population along the flow\,(longitudinal) direction. For large $Pe_r$, $Pe_r$ being the rotary Peclet number, this diffusivity scales as $O(Pe_r^4D_t)$ for $1 \leq κ\lesssim 2$, and as $O(Pe_r^{\frac{10}{3}}D_t)$ for $κ= \infty$, $D_t$ being the (bare)\,swimmer translational diffusivity and $κ$ the swimmer aspect ratio. For $2 \lesssim κ< \infty$, swimmers collapse onto the centerline with increasing $Pe_r$, leading to an anomalously reduced diffusivity of $O(Pe_r^{5+C(κ)}D_t)$. Here, $C(κ)\!<\!-1$ characterizes the algebraic decay of swimmer concentration outside an $O(Pe_r^{-1})$ central core, with the anomalous exponent governed by large velocity variations sampled by the few swimmers outside this core. $C(κ)$ dips below $-5$ for $κ\gtrsim 10$, leading to a flow-independent bound of $O(κ^{10}D_t)$ for the dispersion of sufficiently slender swimmers. △ Less

Submitted 4 December, 2022; originally announced December 2022.

Comments: 6 pages of the main text and 6 pages of supplementary information

Inertial migration of a sphere in plane Couette flow

Authors: Prateek Anand, Ganesh Subramanian

Abstract: We study the inertial migration of a torque-free neutrally buoyant sphere in wall-bounded plane Couette flow over a wide range of channel Reynolds numbers, $Re_c$, in the limit of small particle Reynolds number\,($Re_p\ll1$) and confinement ratio\,($λ\ll1$). Here, $Re_c = V_\text{wall}H/ν$ where $H$ denotes the separation between the channel walls, $V_\text{wall}$ denotes the speed of the moving w… ▽ More We study the inertial migration of a torque-free neutrally buoyant sphere in wall-bounded plane Couette flow over a wide range of channel Reynolds numbers, $Re_c$, in the limit of small particle Reynolds number\,($Re_p\ll1$) and confinement ratio\,($λ\ll1$). Here, $Re_c = V_\text{wall}H/ν$ where $H$ denotes the separation between the channel walls, $V_\text{wall}$ denotes the speed of the moving wall, and $ν$ is the kinematic viscosity of the Newtonian suspending fluid; $λ= a/H$, $a$ being the sphere radius, with $Re_p=λ^2 Re_c$. The channel centerline is found to be the only (stable)\,equilibrium below a critical $Re_c\,(\approx 148)$, consistent with the predictions of earlier small-$Re_c$ analyses. A supercritical pitchfork bifurcation at the critical $Re_c$ creates a pair of stable off-center equilibria, symmetrically located with respect to the centerline, with the original centerline equilibrium simultaneously becoming unstable. The new equilibria migrate wallward with increasing $Re_c$. In contrast to the inference based on recent computations, the aforementioned bifurcation occurs for arbitrarily small $Re_p$ provided $λ$ is sufficiently small. An analogous bifurcation occurs in the two-dimensional scenario, that is, for a circular cylinder suspended freely in plane Couette flow, with the critical $Re_c$ being approximately $110$. △ Less

Submitted 2 November, 2022; originally announced November 2022.

ABB-BERT: A BERT model for disambiguating abbreviations and contractions

Authors: Prateek Kacker, Andi Cupallari, Aswin Gridhar Subramanian, Nimit Jain

Abstract: Abbreviations and contractions are commonly found in text across different domains. For example, doctors' notes contain many contractions that can be personalized based on their choices. Existing spelling correction models are not suitable to handle expansions because of many reductions of characters in words. In this work, we propose ABB-BERT, a BERT-based model, which deals with an ambiguous lan… ▽ More Abbreviations and contractions are commonly found in text across different domains. For example, doctors' notes contain many contractions that can be personalized based on their choices. Existing spelling correction models are not suitable to handle expansions because of many reductions of characters in words. In this work, we propose ABB-BERT, a BERT-based model, which deals with an ambiguous language containing abbreviations and contractions. ABB-BERT can rank them from thousands of options and is designed for scale. It is trained on Wikipedia text, and the algorithm allows it to be fine-tuned with little compute to get better performance for a domain or person. We are publicly releasing the training dataset for abbreviations and contractions derived from Wikipedia. △ Less

Submitted 8 July, 2022; originally announced July 2022.

Journal ref: Proceedings of the 18th International Conference on Natural Language Processing, pages 289 297 Silchar, India, 2021

The geometry of planar linear flows

Authors: Sabarish V Narayanan, Ganesh Subramanian

Abstract: We identify incompressible planar linear flows that are generalizations of the well known one-parameter family characterized by the ratio of in-plane extension to (out-of-plane) vorticity. The latter `canonical' family is classified into elliptic and hyperbolic linear flows with closed and open streamlines, respectively, corresponding to the extension-to-vorticity ratio being less or greater than… ▽ More We identify incompressible planar linear flows that are generalizations of the well known one-parameter family characterized by the ratio of in-plane extension to (out-of-plane) vorticity. The latter `canonical' family is classified into elliptic and hyperbolic linear flows with closed and open streamlines, respectively, corresponding to the extension-to-vorticity ratio being less or greater than unity; unity being the marginal case of simple shear flow. The novel flows possess an out-of-plane extension, but the streamlines may nevertheless be closed or open, allowing for an organization, in a three-dimensional parameter space, into regions of `eccentric' elliptic and hyperbolic flows, separated by a surface of degenerate linear flows with parabolic streamlines that are generalizations of simple shear. We discuss implications for various fluid mechanical scenarios. △ Less

Submitted 15 June, 2022; originally announced June 2022.

Comments: 6 pages, 3 figures in main manuscript, Others included in Supplemental Material

Understanding viscoelastic flow instabilities: Oldroyd-B and beyond

Authors: Hugo A. Castillo Sanchez, Mihailo R. Jovanovic, Satish Kumar, Alexander Morozov, V. Shankar, Ganesh Subramanian, Helen J. Wilson

Abstract: The Oldroyd-B model has been used extensively to predict a host of instabilities in shearing flows of viscoelastic fluids, often realized experimentally using polymer solutions. The present review, written on the occasion of the birth centenary of James Oldroyd, provides an overview of instabilities found across major classes of shearing flows. These comprise (i) the canonical rectilinear shearing… ▽ More The Oldroyd-B model has been used extensively to predict a host of instabilities in shearing flows of viscoelastic fluids, often realized experimentally using polymer solutions. The present review, written on the occasion of the birth centenary of James Oldroyd, provides an overview of instabilities found across major classes of shearing flows. These comprise (i) the canonical rectilinear shearing flows including plane Couette, plane and pipe Poiseuille flows; (ii) viscometric shearing flows with curved streamlines such as those in the Taylor-Couette, cone-and-plate and parallel-plate geometries; (iii) non-viscometric shearing flows with an underlying extensional flow topology such as the flow in a cross-slot device; and (iv) multilayer shearing flows. While the underlying focus in all these cases is on results obtained using the Oldroyd-B model, we also discuss their relation to the actual instability, and as to how the shortcomings of the Oldroyd-B model may be overcome by the use of more realistic constitutive models. All the three commonly used tools of stability analysis, viz., modal linear stability, nonmodal stability, and weakly nonlinear stability analyses are discussed, with supporting evidence from experiments and numerical simulations as appropriate. Despite only accounting for a shear-rate-independent viscosity and first normal stress coefficient, the Oldroyd-B model is able to qualitatively predict the majority of instabilities in the aforementioned shearing flows. The review also highlights, where appropriate, open questions in the area of viscoelastic stability. △ Less

Submitted 16 February, 2022; originally announced February 2022.

Comments: Invited review article for the special issue of JNNFM commemorating the birth centenary of James Oldroyd

Journal ref: Journal of non-Newtonian Fluid Mechanics, Vol 302, 104742, 2022

Order of Commitments in Bayesian Persuasion with Partial-informed Senders

Authors: Shih-Tang Su, Vijay G. Subramanian

Abstract: The commitment power of senders distinguishes Bayesian persuasion problems from other games with (strategic) communication. Persuasion games with multiple senders have largely studied simultaneous commitment and signalling settings. However, many real-world instances with multiple senders have sequential signalling. In such contexts, commitments can also be made sequentially, and then the order of… ▽ More The commitment power of senders distinguishes Bayesian persuasion problems from other games with (strategic) communication. Persuasion games with multiple senders have largely studied simultaneous commitment and signalling settings. However, many real-world instances with multiple senders have sequential signalling. In such contexts, commitments can also be made sequentially, and then the order of commitment by the senders -- the sender signalling last committing first or last -- could significantly impact the equilibrium payoffs and strategies. For a two-sender persuasion game where the senders are partially aware of the state of the world, we find necessary and sufficient conditions to determine when different commitment orders yield different payoff profiles. In particular, for the two-sender setting, we show that different payoff profiles arise if two properties hold: 1) the two senders are willing to collaborate in persuading the receiver in some state(s); and 2) the sender signalling second can carry out a credible threat when committing first such that the other sender's room to design signals gets constrained. △ Less

Submitted 13 February, 2022; originally announced February 2022.

Comments: 27 pages, submitted to EC22

Two-fluid kinetic theory for dilute polymer solutions

Authors: Shiwani Singh, Ganesh Subramanian, Santosh Ansumali

Abstract: We provide a Boltzmann-type kinetic description for dilute polymer solutions based on two-fluid theory. This Boltzmann-type description uses a quasi-equilibrium based relaxation mechanism to model collisions between a polymer dumbbell and a solvent molecule. The model reproduces the desired macroscopic equations for the polymer-solvent mixture. The proposed kinetic scheme leads to a numerical algo… ▽ More We provide a Boltzmann-type kinetic description for dilute polymer solutions based on two-fluid theory. This Boltzmann-type description uses a quasi-equilibrium based relaxation mechanism to model collisions between a polymer dumbbell and a solvent molecule. The model reproduces the desired macroscopic equations for the polymer-solvent mixture. The proposed kinetic scheme leads to a numerical algorithm which is along the lines of the lattice Boltzmann method. Finally, the algorithm is applied to describe the evolution of a perturbed Kolmogorov flow profile, whereby we recover the major elastic effect exhibited by a polymer solution, specifically, the suppression of the original inertial instability. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Decentralized Mean Field Games

Authors: Sriram Ganapathi Subramanian, Matthew E. Taylor, Mark Crowley, Pascal Poupart

Abstract: Multiagent reinforcement learning algorithms have not been widely adopted in large scale environments with many agents as they often scale poorly with the number of agents. Using mean field theory to aggregate agents has been proposed as a solution to this problem. However, almost all previous methods in this area make a strong assumption of a centralized system where all the agents in the environ… ▽ More Multiagent reinforcement learning algorithms have not been widely adopted in large scale environments with many agents as they often scale poorly with the number of agents. Using mean field theory to aggregate agents has been proposed as a solution to this problem. However, almost all previous methods in this area make a strong assumption of a centralized system where all the agents in the environment learn the same policy and are effectively indistinguishable from each other. In this paper, we relax this assumption about indistinguishable agents and propose a new mean field system known as Decentralized Mean Field Games, where each agent can be quite different from others. All agents learn independent policies in a decentralized fashion, based on their local observations. We define a theoretical solution concept for this system and provide a fixed point guarantee for a Q-learning based algorithm in this system. A practical consequence of our approach is that we can address a `chicken-and-egg' problem in empirical mean field reinforcement learning algorithms. Further, we provide Q-learning and actor-critic algorithms that use the decentralized mean field learning approach and give stronger performances compared to common baselines in this area. In our setting, agents do not need to be clones of each other and learn in a fully decentralized fashion. Hence, for the first time, we show the application of mean field learning methods in fully competitive environments, large-scale continuous action space environments, and other environments with heterogeneous agents. Importantly, we also apply the mean field method in a ride-sharing problem using a real-world dataset. We propose a decentralized solution to this problem, which is more practical than existing centralized training methods. △ Less

Submitted 13 April, 2022; v1 submitted 16 December, 2021; originally announced December 2021.

Comments: This work is to appear in AAAI-22. Recent version has minor formatting changes and some typos corrected

Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environments

Authors: Ken Ming Lee, Sriram Ganapathi Subramanian, Mark Crowley

Abstract: Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this pa… ▽ More Independent reinforcement learning algorithms have no theoretical guarantees for finding the best policy in multi-agent settings. However, in practice, prior works have reported good performance with independent algorithms in some domains and bad performance in others. Moreover, a comprehensive study of the strengths and weaknesses of independent algorithms is lacking in the literature. In this paper, we carry out an empirical comparison of the performance of independent algorithms on four PettingZoo environments that span the three main categories of multi-agent environments, i.e., cooperative, competitive, and mixed. We show that in fully-observable environments, independent algorithms can perform on par with multi-agent algorithms in cooperative and competitive settings. For the mixed environments, we show that agents trained via independent algorithms learn to perform well individually, but fail to learn to cooperate with allies and compete with enemies. We also show that adding recurrence improves the learning of independent algorithms in cooperative partially observable environments. △ Less

Submitted 1 November, 2021; originally announced November 2021.

Comments: 15 pages, 7 figures, Accepted for NeurIPS 2021 Deep Reinforcement Learning Workshop

Multi-Agent Advisor Q-Learning

Authors: Sriram Ganapathi Subramanian, Matthew E. Taylor, Kate Larson, Mark Crowley

Abstract: In the last decade, there have been significant advances in multi-agent reinforcement learning (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possible. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating… ▽ More In the last decade, there have been significant advances in multi-agent reinforcement learning (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possible. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online sub-optimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow us to improve learning by appropriately incorporating advice from an advisor (ADMIRAL-DM), and evaluate the effectiveness of an advisor (ADMIRAL-AE). We analyze the algorithms theoretically and provide fixed-point guarantees regarding their learning in general-sum stochastic games. Furthermore, extensive experiments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors. △ Less

Submitted 1 March, 2023; v1 submitted 25 October, 2021; originally announced November 2021.

Comments: Paper has been accepted to Journal of Artificial Intelligence Research (JAIR). Please refer to https://jair.org/index.php/jair/article/view/13445 for JAIR version. The most recent version includes two illustrative figures that pictorially describes the settings of the two algorithms (i.e., ADMIRAL-DM and ADMIRAL-AE)

On the benefits of being constrained when receiving signals

Authors: Shih-Tang Su, David Kempe, Vijay G. Subramanian

Abstract: We study a Bayesian persuasion setting in which the receiver is trying to match the (binary) state of the world. The sender's utility is partially aligned with the receiver's, in that conditioned on the receiver's action, the sender derives higher utility when the state of the world matches the action. Our focus is on whether, in such a setting, being constrained helps a receiver. Intuitively, i… ▽ More We study a Bayesian persuasion setting in which the receiver is trying to match the (binary) state of the world. The sender's utility is partially aligned with the receiver's, in that conditioned on the receiver's action, the sender derives higher utility when the state of the world matches the action. Our focus is on whether, in such a setting, being constrained helps a receiver. Intuitively, if the receiver can only take the sender's preferred action with a smaller probability, the sender might have to reveal more information, so that the receiver can take the action more specifically when the sender prefers it. We show that with a binary state of the world, this intuition indeed carries through: under very mild non-degeneracy conditions, a more constrained receiver will always obtain (weakly) higher utility than a less constrained one. Unfortunately, without additional assumptions, the result does not hold when there are more than two states in the world, which we show with an explicit example. △ Less

Submitted 25 October, 2021; v1 submitted 21 October, 2021; originally announced October 2021.

Bayesian Persuasion in Sequential Trials

Authors: Shih-Tang Su, Vijay G. Subramanian, Grant Schoenebeck

Abstract: We consider a Bayesian persuasion or information design problem where the sender tries to persuade the receiver to take a particular action via a sequence of signals. This we model by considering multi-phase trials with different experiments conducted based on the outcomes of prior experiments. In contrast to most of the literature, we consider the problem with constraints on signals imposed on th… ▽ More We consider a Bayesian persuasion or information design problem where the sender tries to persuade the receiver to take a particular action via a sequence of signals. This we model by considering multi-phase trials with different experiments conducted based on the outcomes of prior experiments. In contrast to most of the literature, we consider the problem with constraints on signals imposed on the sender. This we achieve by fixing some of the experiments in an exogenous manner; these are called determined experiments. This modeling helps us understand real-world situations where this occurs: e.g., multi-phase drug trials where the FDA determines some of the experiments, funding of a startup by a venture capital firm, start-up acquisition by big firms where late-stage assessments are determined by the potential acquirer, multi-round job interviews where the candidates signal initially by presenting their qualifications but the rest of the screening procedures are determined by the interviewer. The non-determined experiments (signals) in the multi-phase trial are to be chosen by the sender in order to persuade the receiver best. With a binary state of the world, we start by deriving the optimal signaling policy in the only non-trivial configuration of a two-phase trial with binary-outcome experiments. We then generalize to multi-phase trials with binary-outcome experiments where the determined experiments can be placed at any chosen node in the trial tree. Here we present a dynamic programming algorithm to derive the optimal signaling policy that uses the two-phase trial solution's structural insights. We also contrast the optimal signaling policy structure with classical Bayesian persuasion strategies to highlight the impact of the signaling constraints on the sender. △ Less

Submitted 22 November, 2021; v1 submitted 18 October, 2021; originally announced October 2021.

Comments: This is a camera-ready version of the 17th conference on web and internet economics (WINE 2021)

Perspectives on viscoelastic flow instabilities and elastic turbulence

Authors: Sujit S. Datta, Arezoo M. Ardekani, Paulo E. Arratia, Antony N. Beris, Irmgard Bischofberger, Jens G. Eggers, J. Esteban López-Aguilar, Suzanne M. Fielding, Anna Frishman, Michael D. Graham, Jeffrey S. Guasto, Simon J. Haward, Sarah Hormozi, Gareth H. McKinley, Robert J. Poole, Alexander Morozov, V. Shankar, Eric S. G. Shaqfeh, Amy Q. Shen, Holger Stark, Victor Steinberg, Ganesh Subramanian, Howard A. Stone

Abstract: Viscoelastic fluids are a common subclass of rheologically complex materials that are encountered in diverse fields from biology to polymer processing. Often the flows of viscoelastic fluids are unstable in situations where ordinary Newtonian fluids are stable, owing to the nonlinear coupling of the elastic and viscous stresses. Perhaps more surprisingly, the instabilities produce flows with the h… ▽ More Viscoelastic fluids are a common subclass of rheologically complex materials that are encountered in diverse fields from biology to polymer processing. Often the flows of viscoelastic fluids are unstable in situations where ordinary Newtonian fluids are stable, owing to the nonlinear coupling of the elastic and viscous stresses. Perhaps more surprisingly, the instabilities produce flows with the hallmarks of turbulence -- even though the effective Reynolds numbers may be $O(1)$ or smaller. We provide perspectives on viscoelastic flow instabilities by integrating the input from speakers at a recent international workshop: historical remarks, characterization of fluids and flows, discussion of experimental and simulation tools, and modern questions and puzzles that motivate further studies of this fascinating subject. The materials here will be useful for researchers and educators alike, especially as the subject continues to evolve in both fundamental understanding and applications in engineering and the sciences. △ Less

Submitted 22 August, 2021; originally announced August 2021.

Comments: A perspective article based on a virtual workshop of the Princeton Center for Theoretical Sciences that took place in January 2021

Motion of a sphere in a viscous density stratified fluid

Authors: Arun Kumar Varanasi, Ganesh Subramanian

Abstract: We examine the translation of a sphere in a stably stratified ambient in the limit of small Reynolds ($Re \ll 1$) and viscous Richardson numbers ($Ri_v \ll 1$); here, $Re = \frac{ρUa}μ$ and $Ri_v = \frac{γa^3 g}{μU}$ with $a$ being the sphere radius, $U$ the translation speed, $ρ$ and $μ$ the density and viscosity of the stratified ambient, $g$ the acceleration due to gravity, and $γ$ the density… ▽ More We examine the translation of a sphere in a stably stratified ambient in the limit of small Reynolds ($Re \ll 1$) and viscous Richardson numbers ($Ri_v \ll 1$); here, $Re = \frac{ρUa}μ$ and $Ri_v = \frac{γa^3 g}{μU}$ with $a$ being the sphere radius, $U$ the translation speed, $ρ$ and $μ$ the density and viscosity of the stratified ambient, $g$ the acceleration due to gravity, and $γ$ the density gradient (assumed constant) characterizing the ambient stratification. In contrast to most earlier efforts, our study considers the convection dominant limit corresponding to $Pe = \frac{Ua}{D} \gg 1$, $D$ being the diffusivity of the stratifying agent. We characterize in detail the velocity and density fields around the particle in what we term the Stokes stratification regime, defined by $Re \ll Ri_v^{\frac{1}{3}} \ll 1$, and corresponding to the dominance of buoyancy over inertial forces. Buoyancy forces associated with the perturbed stratification fundamentally alter the viscously dominated fluid motion at large distances. At distances of order the stratification screening length, that scales as $aRi_v^{-\frac{1}{3}}$, the motion transforms from the familiar fore-aft symmetric Stokesian form to a fore-aft asymmetric pattern of recirculating cells with primarily horizontal motion within; except in the vicinity of the rear stagnation streamline. At larger distances, the motion is vanishingly small except within (a) an axisymmetric horizontal wake whose vertical extent grows as $O(r_t^{\frac{2}{5}})$, $r_t$ being the distance in the plane perpendicular to translation and (b) a buoyant reverse jet behind the particle that narrows as the inverse square root of distance downstream. △ Less

Submitted 29 July, 2021; originally announced July 2021.

Comments: 36 pages, 14 figures

MSC Class: 76D50 ACM Class: J.2

Linear stability of a rotating liquid column revisited

Authors: Pulkit Dubey, Anubhab Roy, Ganesh Subramanian

Abstract: We revisit the somewhat classical problem of the linear stability of a rigidly rotating liquid column in this communication. Although literature pertaining to this problem dates back to 1959, the relation between inviscid and viscous stability criteria has not yet been clarified. While the viscous criterion for stability, given by $We = n^2+k^2-1$, is both necessary and sufficient, this relation h… ▽ More We revisit the somewhat classical problem of the linear stability of a rigidly rotating liquid column in this communication. Although literature pertaining to this problem dates back to 1959, the relation between inviscid and viscous stability criteria has not yet been clarified. While the viscous criterion for stability, given by $We = n^2+k^2-1$, is both necessary and sufficient, this relation has only been shown to be sufficient in the inviscid case. Here, $We = ρΩ^2 a^3/γ$ is the Weber number and measures the relative magnitudes of the centrifugal and surface tension forces, with $Ω$ being the angular velocity of the rigidly rotating column, $a$ the column radius, $ρ$ the density of the fluid, and $γ$ the surface tension coefficient; $k$ and $n$ denote the axial and azimuthal wavenumbers of the imposed perturbation. We show that the subtle difference between the inviscid and viscous criteria arises from the surprisingly complicated picture of inviscid stability in the $We-k$ plane. For all $n >1$, the viscously unstable region, corresponding to $We > n^2+k^2-1$, contains an infinite hierarchy of inviscidly stable islands ending in cusps, with a dominant leading island. Only the dominant island, now infinite in extent along the $We$ axis, persists for $n= 1$. This picture may be understood, based on the underlying eigenspectrum, as arising from the cascade of coalescences between a retrograde mode, that is the continuation of the cograde surface-tension-driven mode across the zero Doppler frequency point, and successive retrograde Coriolis modes constituting an infinite hierarchy. △ Less

Submitted 14 July, 2021; originally announced July 2021.

Comments: 22 pages, 13 figures

MSC Class: 76B45; 76B47 ACM Class: J.2

Journal ref: J. Fluid Mech. 933 (2022) A55

A continuous pathway between the elasto-inertial and elastic turbulent states in viscoelastic channel flow

Authors: Mohammad Khalid, V. Shankar, Ganesh Subramanian

Abstract: We show that viscoelastic plane Poiseuille flow becomes linearly unstable in the absence of inertia, in the limit of high elasticities, for ultra-dilute polymer solutions. While inertialess elastic instabilities have been predicted for curvilinear shear flows, this is the first ever report of a purely elastic linear instability in a rectilinear shear flow. The novel instability continues upto a Re… ▽ More We show that viscoelastic plane Poiseuille flow becomes linearly unstable in the absence of inertia, in the limit of high elasticities, for ultra-dilute polymer solutions. While inertialess elastic instabilities have been predicted for curvilinear shear flows, this is the first ever report of a purely elastic linear instability in a rectilinear shear flow. The novel instability continues upto a Reynolds number ($Re$) of $O(1000)$, corresponding to the recently identified elasto-inertial turbulent state believed to underlie the maximum-drag-reduced regime. Thus, for highly elastic ultra-dilute polymer solutions, a single linearly unstable modal branch may underlie transition to elastic turbulence at zero $Re$, and to elasto-inertial turbulence at moderate $Re$, implying the existence of continuous pathways connecting the turbulent states to each other, and to the laminar base state. △ Less

Submitted 11 March, 2021; originally announced March 2021.

Comments: 5 pages, 4 figures

MSC Class: 76E05

Journal ref: Phys. Rev. Lett. 127, 134502 (2021)

The Effect of Q-function Reuse on the Total Regret of Tabular, Model-Free, Reinforcement Learning

Authors: Volodymyr Tkachuk, Sriram Ganapathi Subramanian, Matthew E. Taylor

Abstract: Some reinforcement learning methods suffer from high sample complexity causing them to not be practical in real-world situations. $Q$-function reuse, a transfer learning method, is one way to reduce the sample complexity of learning, potentially improving usefulness of existing algorithms. Prior work has shown the empirical effectiveness of $Q$-function reuse for various environments when applied… ▽ More Some reinforcement learning methods suffer from high sample complexity causing them to not be practical in real-world situations. $Q$-function reuse, a transfer learning method, is one way to reduce the sample complexity of learning, potentially improving usefulness of existing algorithms. Prior work has shown the empirical effectiveness of $Q$-function reuse for various environments when applied to model-free algorithms. To the best of our knowledge, there has been no theoretical work showing the regret of $Q$-function reuse when applied to the tabular, model-free setting. We aim to bridge the gap between theoretical and empirical work in $Q$-function reuse by providing some theoretical insights on the effectiveness of $Q$-function reuse when applied to the $Q$-learning with UCB-Hoeffding algorithm. Our main contribution is showing that in a specific case if $Q$-function reuse is applied to the $Q$-learning with UCB-Hoeffding algorithm it has a regret that is independent of the state or action space. We also provide empirical results supporting our theoretical findings. △ Less

Submitted 7 March, 2021; originally announced March 2021.

Comments: 7 pages, 2 figures, submitted to ALA 2021

The rotation of a sedimenting anisotropic particle in a linearly stratified ambient

Authors: Arun Kumar Varanasi, Navaneeth K. Marath, Ganesh Subramanian

Abstract: We derive the torque on a spheroid of an arbitrary aspect ratio $κ$ sedimenting in a linearly stratified ambient. The analysis demarcates regions in parameter space corresponding to broadside-on and edgewise (longside-on) settling in the limit $Re, Ri_v \ll 1$, where $Re = ρ_0UL/μ$ and $Ri_v =γL^3g/μU$, the Reynolds and viscous Richardson numbers, respectively, are dimensionless measures of the im… ▽ More We derive the torque on a spheroid of an arbitrary aspect ratio $κ$ sedimenting in a linearly stratified ambient. The analysis demarcates regions in parameter space corresponding to broadside-on and edgewise (longside-on) settling in the limit $Re, Ri_v \ll 1$, where $Re = ρ_0UL/μ$ and $Ri_v =γL^3g/μU$, the Reynolds and viscous Richardson numbers, respectively, are dimensionless measures of the importance of inertial and buoyancy forces relative to viscous ones. Here, $L$ is the spheroid semi-major axis, $U$ an appropriate settling velocity scale, $μ$ the fluid viscosity, and $γ\,(>0)$ the (constant)\,density gradient characterizing the stably stratified ambient, with $ρ_0$ being the fluid density taken to be a constant within the Boussinesq framework. A reciprocal theorem formulation identifies three contributions to the torque: (1) an $O(Re)$ inertial contribution that already exists in a homogeneous ambient, and orients the spheroid broadside-on; (2) an $O(Ri_v)$ hydrostatic contribution due to the ambient linear stratification that also orients the spheroid broadside-on; and (3) a hydrodynamic contribution arising from the perturbation of the ambient stratification by the spheroid whose nature depends on $Pe$; $Pe = UL/D$ being the Peclet number with $D$ the diffusivity of the stratifying agent. For $Pe \gg 1$, the hydrodynamic contribution is $O(Ri_v^{\frac{2}{3}}$) in the Stokes stratification regime characterized by $Re \ll Ri_v^{\frac{1}{3}}$, and orients the spheroid edgewise regardless of $κ$. The differing orientation dependencies of the inertial and large-$Pe$ hydrodynamic stratification torques imply that the broadside-on and edgewise settling regimes are separated by two distinct $κ$-dependent critical curves in the $Ri_v/Re^{\frac{3}{2}}-κ$ plane. The predictions are consistent with recent experimental observations. △ Less

Submitted 17 February, 2021; originally announced February 2021.

Comments: 34 pages, 15 figures

MSC Class: 76T20

Inertio-elastic instability of a vortex column

Authors: Anubhab Roy, Piyush Garg, Jhumpal Shashikiran Reddy, Ganesh Subramanian

Abstract: We analyze the instability of a vortex column in a dilute polymer solution at large $\textit{Re}$ and $\textit{De}$ with $\textit{El} = \textit{De}/\textit{Re}$, the elasticity number, being finite. Here, $\textit{Re} = Ω_0 a^2/ν_s$ and $\textit{De} = Ω_0 τ$ are, respectively, the Reynolds and Deborah numbers based on the core angular velocity ($Ω_0$), the radius of the column ($a$), the solvent-b… ▽ More We analyze the instability of a vortex column in a dilute polymer solution at large $\textit{Re}$ and $\textit{De}$ with $\textit{El} = \textit{De}/\textit{Re}$, the elasticity number, being finite. Here, $\textit{Re} = Ω_0 a^2/ν_s$ and $\textit{De} = Ω_0 τ$ are, respectively, the Reynolds and Deborah numbers based on the core angular velocity ($Ω_0$), the radius of the column ($a$), the solvent-based kinematic viscosity ($ν_s = μ_s/ρ$), and the polymeric relaxation time ($τ$). The stability of small-amplitude perturbations in this distinguished limit is governed by the elastic Rayleigh equation whose spectrum is parameterized by $ \textrm{E} = \textit{El}(1-β)$, $β$ being the ratio of the solvent to the solution viscosity. The neglect of the relaxation terms, in the said limit, implies that the polymer solution supports undamped elastic shear waves propagating relative to the base-state flow. The existence of these shear waves leads to multiple (three) continuous spectra associated with the elastic Rayleigh equation in contrast to just one for the original Rayleigh equation. Further, unlike the neutrally stable inviscid case, an instability of the vortex column arises for finite E due to a pair of elastic shear waves being driven into a resonant interaction under the differential convection by the irrotational shearing flow outside the core. An asymptotic analysis for the Rankine profile shows the absence of an elastic threshold; although, for small E, the growth rate of the unstable discrete mode is transcendentally small, being O$(\textrm{E}^2e^{-1/\textrm{E}^{\frac{1}{2}}})$. An accompanying numerical investigation shows that the instability persists for smooth vorticity profiles, provided the radial extent of the transition region (from the rotational core to the irrotational exterior) is less than a certain $\textrm{E}$-dependent threshold. △ Less

Submitted 4 January, 2021; originally announced January 2021.

Partially Observable Mean Field Reinforcement Learning

Authors: Sriram Ganapathi Subramanian, Matthew E. Taylor, Mark Crowley, Pascal Poupart

Abstract: Traditional multi-agent reinforcement learning algorithms are not scalable to environments with more than a few agents, since these algorithms are exponential in the number of agents. Recent research has introduced successful methods to scale multi-agent reinforcement learning algorithms to many agent scenarios using mean field theory. Previous work in this field assumes that an agent has access t… ▽ More Traditional multi-agent reinforcement learning algorithms are not scalable to environments with more than a few agents, since these algorithms are exponential in the number of agents. Recent research has introduced successful methods to scale multi-agent reinforcement learning algorithms to many agent scenarios using mean field theory. Previous work in this field assumes that an agent has access to exact cumulative metrics regarding the mean field behaviour of the system, which it can then use to take its actions. In this paper, we relax this assumption and maintain a distribution to model the uncertainty regarding the mean field of the system. We consider two different settings for this problem. In the first setting, only agents in a fixed neighbourhood are visible, while in the second setting, the visibility of agents is determined at random based on distances. For each of these settings, we introduce a Q-learning based algorithm that can learn effectively. We prove that this Q-learning estimate stays very close to the Nash Q-value (under a common set of assumptions) for the first setting. We also empirically show our algorithms outperform multiple baselines in three different games in the MAgents framework, which supports large environments with many agents learning simultaneously to achieve possibly distinct goals. △ Less

Submitted 24 January, 2021; v1 submitted 31 December, 2020; originally announced December 2020.

Comments: Paper to be published in International Conference on Autonomous Agents and Multiagent Systems (AAMAS) - 2021. New version has some typos corrected

Maximum Reward Formulation In Reinforcement Learning

Authors: Sai Krishna Gottipati, Yashaswi Pathak, Rohan Nuttall, Sahir, Raviteja Chunduru, Ahmed Touati, Sriram Ganapathi Subramanian, Matthew E. Taylor, Sarath Chandar

Abstract: Reinforcement learning (RL) algorithms typically deal with maximizing the expected cumulative return (discounted or undiscounted, finite or infinite horizon). However, several crucial applications in the real world, such as drug discovery, do not fit within this framework because an RL agent only needs to identify states (molecules) that achieve the highest reward within a trajectory and does not… ▽ More Reinforcement learning (RL) algorithms typically deal with maximizing the expected cumulative return (discounted or undiscounted, finite or infinite horizon). However, several crucial applications in the real world, such as drug discovery, do not fit within this framework because an RL agent only needs to identify states (molecules) that achieve the highest reward within a trajectory and does not need to optimize for the expected cumulative return. In this work, we formulate an objective function to maximize the expected maximum reward along a trajectory, derive a novel functional form of the Bellman equation, introduce the corresponding Bellman operators, and provide a proof of convergence. Using this formulation, we achieve state-of-the-art results on the task of molecule generation that mimics a real-world drug discovery pipeline. △ Less

Submitted 18 December, 2023; v1 submitted 7 October, 2020; originally announced October 2020.

Comments: 14 pages, 5 figures Update based on reviewer feedback

The center-mode instability of viscoelastic plane Poiseuille flow

Authors: Mohammad Khalid, Indresh Chaudhary, Piyush Garg, V. Shankar, Ganesh Subramanian

Abstract: A modal stability analysis shows that plane Poiseuille flow of an Oldroyd-B fluid becomes unstable to a `center mode' with phase speed close to the maximum base-flow velocity, $U_{max}$. The governing dimensionless groups are the Reynolds number $Re = ρU_{max} H/η$, the elasticity number $E = λη/(H^2ρ)$, and the ratio of solvent to solution viscosity $η_s/η$; here, $λ$ is the polymer relaxation ti… ▽ More A modal stability analysis shows that plane Poiseuille flow of an Oldroyd-B fluid becomes unstable to a `center mode' with phase speed close to the maximum base-flow velocity, $U_{max}$. The governing dimensionless groups are the Reynolds number $Re = ρU_{max} H/η$, the elasticity number $E = λη/(H^2ρ)$, and the ratio of solvent to solution viscosity $η_s/η$; here, $λ$ is the polymer relaxation time, $H$ is the channel half-width, and $ρ$ is the fluid density. For experimentally relevant values (e.g., $E \sim 0.1$ and $β\sim 0.9$), the predicted critical Reynolds number, $Re_c$, for the center-mode instability is around $200$, with the associated eigenmodes being spread out across the channel. In the asymptotic limit of $E(1 -β) \ll 1$, with $E$ fixed, corresponding to strongly elastic dilute polymer solutions, $Re_c \propto (E(1-β))^{-\frac{3}{2}}$ and the critical wavenumber $k_c \propto (E(1-β))^{-\frac{1}{2}}$. The unstable eigenmode in this limit is confined in a thin layer near the channel centerline. The above features are largely analogous to the center-mode instability in viscoelastic pipe flow (Garg et al., Phys. Rev. Lett., 121, 024502 (2018)), and suggest a universal linear mechanism underlying the onset of turbulence in both channel and pipe flows of suffciently elastic dilute polymer solutions. △ Less

Submitted 1 August, 2020; originally announced August 2020.

MSC Class: 76E05 ACM Class: J.2

An anisotropic particle in a simple shear flow: an instance of chaotic scattering

Authors: Mahan Raj Banerjee, Ganesh Subramanian

Abstract: In the Stokesian limit, the streamline topology around a single neutrally buoyant sphere is identical to the topology of pair-sphere pathlines, both in an ambient simple shear flow. In both cases there are fore-aft symmetric open and closed trajectories spatially demarcated by an axisymmetric separatrix surface. This topology has crucial implications for both scalar transport from a single sphere,… ▽ More In the Stokesian limit, the streamline topology around a single neutrally buoyant sphere is identical to the topology of pair-sphere pathlines, both in an ambient simple shear flow. In both cases there are fore-aft symmetric open and closed trajectories spatially demarcated by an axisymmetric separatrix surface. This topology has crucial implications for both scalar transport from a single sphere, and for the rheology of a dilute suspension of spheres. We show that the topology of the fluid pathlines around a neutrally buoyant freely rotating spheroid, in simple shear flow, is profoundly different, and will have a crucial bearing on transport from such particles in shearing flows. To the extent that fluid pathlines in the single-spheroid problem and pair-trajectories in the two-spheroid problem, are expected to bear a qualitative resemblance to each other, the non-trivial trajectory topology identified here will also have significant consequences for the rheology of dilute suspensions of anisotropic particles. △ Less

Submitted 21 May, 2020; originally announced May 2020.

Comments: 46 pages, 34 figures

Linear instability of viscoelastic pipe flow

Authors: Indresh Chaudhary, Piyush Garg, Ganesh Subramanian, Viswanathan Shankar

Abstract: A modal stability analysis shows that pressure-driven pipe flow of an Oldroyd-B fluid is linearly unstable to axisymmetric perturbations, in stark contrast to its Newtonian counterpart which is linearly stable at all Reynolds numbers. The dimensionless groups that govern stability are the Reynolds number, the elasticity number, and the ratio of solvent to solution viscosity. The unstable mode has… ▽ More A modal stability analysis shows that pressure-driven pipe flow of an Oldroyd-B fluid is linearly unstable to axisymmetric perturbations, in stark contrast to its Newtonian counterpart which is linearly stable at all Reynolds numbers. The dimensionless groups that govern stability are the Reynolds number, the elasticity number, and the ratio of solvent to solution viscosity. The unstable mode has a phase speed close to the base-state maximum over the entire unstable region in the relevant parameter space, implying that the unstable mode belongs to a class of viscoelastic center modes. Unlike the Newtonian transition which is dominated by nonlinear processes, the linear instability discussed here could be very relevant to the onset of turbulence in viscoelastic pipe flows. The prediction of an instability is, in fact, consistent with several experimental studies on pipe flow of polymer solutions, ranging from previous reports of early turbulence to the more recent discovery of elasto-inertial turbulence. The instability identified in this study comprehensively dispels the prevailing notion of pipe flow of viscoelastic fluids being linearly stable in the Reynolds-Weissenberg plane, marking a possible paradigm shift in our understanding of transition in rectilinear viscoelastic shearing flows. △ Less

Submitted 20 March, 2020; originally announced March 2020.

Comments: 51 pages, 30 figures

Journal ref: J. Fluid Mech. 908 (2021) A11

A review of machine learning applications in wildfire science and management

Authors: Piyush Jain, Sean C P Coogan, Sriram Ganapathi Subramanian, Mark Crowley, Steve Taylor, Mike D Flannigan

Abstract: Artificial intelligence has been applied in wildfire science and management since the 1990s, with early applications including neural networks and expert systems. Since then the field has rapidly progressed congruently with the wide adoption of machine learning (ML) in the environmental sciences. Here, we present a scoping review of ML in wildfire science and management. Our objective is to improv… ▽ More Artificial intelligence has been applied in wildfire science and management since the 1990s, with early applications including neural networks and expert systems. Since then the field has rapidly progressed congruently with the wide adoption of machine learning (ML) in the environmental sciences. Here, we present a scoping review of ML in wildfire science and management. Our objective is to improve awareness of ML among wildfire scientists and managers, as well as illustrate the challenging range of problems in wildfire science available to data scientists. We first present an overview of popular ML approaches used in wildfire science to date, and then review their use in wildfire science within six problem domains: 1) fuels characterization, fire detection, and mapping; 2) fire weather and climate change; 3) fire occurrence, susceptibility, and risk; 4) fire behavior prediction; 5) fire effects; and 6) fire management. We also discuss the advantages and limitations of various ML approaches and identify opportunities for future advances in wildfire science and management within a data science context. We identified 298 relevant publications, where the most frequently used ML methods included random forests, MaxEnt, artificial neural networks, decision trees, support vector machines, and genetic algorithms. There exists opportunities to apply more current ML methods (e.g., deep learning and agent based learning) in wildfire science. However, despite the ability of ML models to learn on their own, expertise in wildfire science is necessary to ensure realistic modelling of fire processes across multiple scales, while the complexity of some ML methods requires sophisticated knowledge for their application. Finally, we stress that the wildfire research and management community plays an active role in providing relevant, high quality data for use by practitioners of ML methods. △ Less

Submitted 19 August, 2020; v1 submitted 1 March, 2020; originally announced March 2020.

Comments: 83 pages, 4 figures, 3 tables

Journal ref: Environmental Reviews. 28(4): 478-505, 2020

Towards Label-Free 3D Segmentation of Optical Coherence Tomography Images of the Optic Nerve Head Using Deep Learning

Authors: Sripad Krishna Devalla, Tan Hung Pham, Satish Kumar Panda, Liang Zhang, Giridhar Subramanian, Anirudh Swaminathan, Chin Zhi Yun, Mohan Rajan, Sujatha Mohan, Ramaswami Krishnadas, Vijayalakshmi Senthil, John Mark S. de Leon, Tin A. Tun, Ching-Yu Cheng, Leopold Schmetterer, Shamira Perera, Tin Aung, Alexandre H. Thiery, Michael J. A. Girard

Abstract: Since the introduction of optical coherence tomography (OCT), it has been possible to study the complex 3D morphological changes of the optic nerve head (ONH) tissues that occur along with the progression of glaucoma. Although several deep learning (DL) techniques have been recently proposed for the automated extraction (segmentation) and quantification of these morphological changes, the device s… ▽ More Since the introduction of optical coherence tomography (OCT), it has been possible to study the complex 3D morphological changes of the optic nerve head (ONH) tissues that occur along with the progression of glaucoma. Although several deep learning (DL) techniques have been recently proposed for the automated extraction (segmentation) and quantification of these morphological changes, the device specific nature and the difficulty in preparing manual segmentations (training data) limit their clinical adoption. With several new manufacturers and next-generation OCT devices entering the market, the complexity in deploying DL algorithms clinically is only increasing. To address this, we propose a DL based 3D segmentation framework that is easily translatable across OCT devices in a label-free manner (i.e. without the need to manually re-segment data for each device). Specifically, we developed 2 sets of DL networks. The first (referred to as the enhancer) was able to enhance OCT image quality from 3 OCT devices, and harmonized image-characteristics across these devices. The second performed 3D segmentation of 6 important ONH tissue layers. We found that the use of the enhancer was critical for our segmentation network to achieve device independency. In other words, our 3D segmentation network trained on any of 3 devices successfully segmented ONH tissue layers from the other two devices with high performance (Dice coefficients > 0.92). With such an approach, we could automatically segment images from new OCT devices without ever needing manual segmentation data from such devices. △ Less

Submitted 22 February, 2020; originally announced February 2020.

Multi Type Mean Field Reinforcement Learning

Authors: Sriram Ganapathi Subramanian, Pascal Poupart, Matthew E. Taylor, Nidhi Hegde

Abstract: Mean field theory provides an effective way of scaling multiagent reinforcement learning algorithms to environments with many agents that can be abstracted by a virtual mean agent. In this paper, we extend mean field multiagent algorithms to multiple types. The types enable the relaxation of a core assumption in mean field reinforcement learning, which is that all agents in the environment are pla… ▽ More Mean field theory provides an effective way of scaling multiagent reinforcement learning algorithms to environments with many agents that can be abstracted by a virtual mean agent. In this paper, we extend mean field multiagent algorithms to multiple types. The types enable the relaxation of a core assumption in mean field reinforcement learning, which is that all agents in the environment are playing almost similar strategies and have the same goal. We conduct experiments on three different testbeds for the field of many agent reinforcement learning, based on the standard MAgents framework. We consider two different kinds of mean field environments: a) Games where agents belong to predefined types that are known a priori and b) Games where the type of each agent is unknown and therefore must be learned based on observations. We introduce new algorithms for each type of game and demonstrate their superior performance over state of the art algorithms that assume that all agents belong to the same type and other baseline algorithms in the MAgent framework. △ Less

Submitted 21 June, 2022; v1 submitted 6 February, 2020; originally announced February 2020.

Comments: The paper appears in the proceedings of International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS) 2020. Revised version has some typos corrected

Orientation Dynamics of Sedimenting Anisotropic Particles in Turbulence

Authors: Prateek Anand, Samriddhi Sankar Ray, Ganesh Subramanian

Abstract: We examine the dynamics of small anisotropic particles (spheroids) sedimenting through homogeneous isotropic turbulence using direct numerical simulations and theory. The gravity-induced inertial torque acting on sub-Kolmogorov spheroids leads to pronouncedly non-Gaussian orientation distributions localized about the broadside-on(to gravity) orientation. Orientation distributions and average settl… ▽ More We examine the dynamics of small anisotropic particles (spheroids) sedimenting through homogeneous isotropic turbulence using direct numerical simulations and theory. The gravity-induced inertial torque acting on sub-Kolmogorov spheroids leads to pronouncedly non-Gaussian orientation distributions localized about the broadside-on(to gravity) orientation. Orientation distributions and average settling velocities are obtained over a wide range of spheroid aspect ratios, Stokes and Froude numbers. Orientational moments from the simulations compare well with analytical predictions in the inertialess rapid-settling limit, with both exhibiting a non-monotonic dependence on spheroid aspect ratio. Deviations arise at Stokes numbers of order unity due to a spatially inhomogeneous particle concentration field resulting from a preferential sweeping effect; as a consequence, the time-averaged particle settling velocities exceed the orientationally averaged estimates. △ Less

Submitted 18 July, 2020; v1 submitted 29 June, 2019; originally announced July 2019.

Comments: 18 pages, 5 figures, includes Supplemental material

Journal ref: Phys. Rev. Lett. 125, 034501 (2020)

Shear induced migration of microswimmers in pressure-driven channel flow

Authors: LaxminarsimhaRao V., Sankalp Nambiar, Ganesh Subramanian

Abstract: We study the shear induced migration of microswimmers (primarily, active Brownian particles or ABP's) in a plane Poiseuille flow. For wide channels characterized by $U_b/HD_r \ll 1$, the separation between time scales characterizing the swimmer orientation dynamics (of O($D^{-1}_r$)) and those that characterize migration across the channel (of O($H^{2}D_r/U^{2}_b$)), allows for use of the method o… ▽ More We study the shear induced migration of microswimmers (primarily, active Brownian particles or ABP's) in a plane Poiseuille flow. For wide channels characterized by $U_b/HD_r \ll 1$, the separation between time scales characterizing the swimmer orientation dynamics (of O($D^{-1}_r$)) and those that characterize migration across the channel (of O($H^{2}D_r/U^{2}_b$)), allows for use of the method of multiple scales to derive a drift-diffusion equation for the swimmer concentration profile; here, $U_b$ is the swimming speed, $H$ is the channel half-width, and $D_r$ is the swimmer rotary diffusivity. The steady state concentration profile is a function of the Péclet number, $Pe = U_{f}/(D_r H)$ ($U_f$ being the channel centerline velocity), and the swimmer aspect ratio $κ$. Swimmers with $ κ\gg 1$ (with $ κ\sim$ O(1)), in the regime $1 \ll \textit{Pe} \ll κ^3$ ($Pe\sim$ O(1)), migrate towards the channel walls, corresponding to a high-shear trapping behavior. For $Pe \gg κ^3 $ ($Pe \gg $ 1 for $κ\sim$ O(1)), however, swimmers migrate towards the centerline, corresponding to a low-shear trapping behavior. Interestingly, within the low-shear trapping regime, swimmers with $κ< 2$ asymptote to a $Pe$-independent concentration profile for large $Pe$, while those with $κ\geq 2$ exhibit a `centerline-collapse' for $Pe \to \infty$. The prediction of low-shear-trapping, validated by Langevin simulations, is the first explanation of recent experimental observations [Barry $\textit{et al}$. (2015)]. We organize the high-shear and low-shear trapping regimes on a $Pe-κ$ plane, thereby highlighting the singular behavior of infinite-aspect-ratio swimmers. △ Less

Submitted 26 June, 2019; v1 submitted 3 June, 2019; originally announced June 2019.

Enhanced velocity fluctuations in interacting swimmer suspensions

Authors: Sankalp Nambiar, Piyush Garg, Ganesh Subramanian

Abstract: A dilute non-interacting suspension of micro-swimmers exhibits a finite velocity variance and short-ranged correlations that decay over a swimmer length. For a suspension of interacting straight swimmers, however, pair-interactions leads to a non-decaying velocity covariance, and a variance that diverges logarithmically with system size. The divergence is arrested on inclusion of orientation decor… ▽ More A dilute non-interacting suspension of micro-swimmers exhibits a finite velocity variance and short-ranged correlations that decay over a swimmer length. For a suspension of interacting straight swimmers, however, pair-interactions leads to a non-decaying velocity covariance, and a variance that diverges logarithmically with system size. The divergence is arrested on inclusion of orientation decorrelation mechanisms. Results for suspensions of run-and-tumble particles (RTPs) are presented, where the underlying straight-swimmer divergence leads to a broad cross-over between the ballistic and diffusive regimes, of immersed passive tracers, in the limit of long run lengths. Our analysis explains long-standing experimental observations of a volume-fraction dependent crossover time for passive tracer dynamics. △ Less

Submitted 24 December, 2019; v1 submitted 14 February, 2019; originally announced February 2019.

Comments: 36 pages, 14 figures

Journal ref: J. Fluid Mech. 907 (2021) A26

Derandomized Load Balancing using Random Walks on Expander Graphs

Authors: Dengwang Tang, Vijay G. Subramanian

Abstract: In a computing center with a huge amount of machines, when a job arrives, a dispatcher need to decide which machine to route this job to based on limited information. A classical method, called the power-of-$d$ choices algorithm is to pick $d$ servers independently at random and dispatch the job to the least loaded server among the $d$ servers. In this paper, we analyze a low-randomness variant of… ▽ More In a computing center with a huge amount of machines, when a job arrives, a dispatcher need to decide which machine to route this job to based on limited information. A classical method, called the power-of-$d$ choices algorithm is to pick $d$ servers independently at random and dispatch the job to the least loaded server among the $d$ servers. In this paper, we analyze a low-randomness variant of this dispatching scheme, where $d$ queues are sampled through $d$ independent non-backtracking random walks on a $k$-regular graph $G$. Under certain assumptions of the graph $G$ we show that under this scheme, the dynamics of the queuing system converges to the same deterministic ordinary differential equation (ODE) for the power-of-$d$ choices scheme. We also show that the system is stable under the proposed scheme, and the stationary distribution of the system converges to the fixed point of the ODE. △ Less

Submitted 21 April, 2019; v1 submitted 17 January, 2019; originally announced January 2019.

Comments: 20 pages

Balanced Allocation with Random Walk Based Sampling

Authors: Dengwang Tang, Vijay G. Subramanian

Abstract: In the standard ball-in-bins experiment, a well-known scheme is to sample $d$ bins independently and uniformly at random and put the ball into the least loaded bin. It can be shown that this scheme yields a maximum load of $\log\log n/\log d+O(1)$ with high probability. Subsequent work analyzed the model when at each time, $d$ bins are sampled through some correlated or non-uniform way. However,… ▽ More In the standard ball-in-bins experiment, a well-known scheme is to sample $d$ bins independently and uniformly at random and put the ball into the least loaded bin. It can be shown that this scheme yields a maximum load of $\log\log n/\log d+O(1)$ with high probability. Subsequent work analyzed the model when at each time, $d$ bins are sampled through some correlated or non-uniform way. However, the case when the sampling for different balls are correlated are rarely investigated. In this paper we propose three schemes for the ball-in-bins allocation problem. We assume that there is an underlying $k$-regular graph connecting the bins. The three schemes are variants of power-of-$d$ choices, except that the sampling of $d$ bins at each time are based on the locations of $d$ independently moving non-backtracking random walkers, with the positions of the random walkers being reset when certain events occurs. We show that under some conditions for the underlying graph that can be summarized as the graph having large enough girth, all three schemes can perform as well as power-of-$d$, so that the maximum load is bounded by $\log\log n/\log d+O(1)$ with high probability. △ Less

Submitted 11 October, 2018; v1 submitted 5 October, 2018; originally announced October 2018.

Comments: 37 pages

A Deep Learning Approach to Denoise Optical Coherence Tomography Images of the Optic Nerve Head

Authors: Sripad Krishna Devalla, Giridhar Subramanian, Tan Hung Pham, Xiaofei Wang, Shamira Perera, Tin A. Tun, Tin Aung, Leopold Schmetterer, Alexandre H. Thiery, Michael J. A. Girard

Abstract: Purpose: To develop a deep learning approach to de-noise optical coherence tomography (OCT) B-scans of the optic nerve head (ONH). Methods: Volume scans consisting of 97 horizontal B-scans were acquired through the center of the ONH using a commercial OCT device (Spectralis) for both eyes of 20 subjects. For each eye, single-frame (without signal averaging), and multi-frame (75x signal averaging… ▽ More Purpose: To develop a deep learning approach to de-noise optical coherence tomography (OCT) B-scans of the optic nerve head (ONH). Methods: Volume scans consisting of 97 horizontal B-scans were acquired through the center of the ONH using a commercial OCT device (Spectralis) for both eyes of 20 subjects. For each eye, single-frame (without signal averaging), and multi-frame (75x signal averaging) volume scans were obtained. A custom deep learning network was then designed and trained with 2,328 "clean B-scans" (multi-frame B-scans), and their corresponding "noisy B-scans" (clean B-scans + gaussian noise) to de-noise the single-frame B-scans. The performance of the de-noising algorithm was assessed qualitatively, and quantitatively on 1,552 B-scans using the signal to noise ratio (SNR), contrast to noise ratio (CNR), and mean structural similarity index metrics (MSSIM). Results: The proposed algorithm successfully denoised unseen single-frame OCT B-scans. The denoised B-scans were qualitatively similar to their corresponding multi-frame B-scans, with enhanced visibility of the ONH tissues. The mean SNR increased from $4.02 \pm 0.68$ dB (single-frame) to $8.14 \pm 1.03$ dB (denoised). For all the ONH tissues, the mean CNR increased from $3.50 \pm 0.56$ (single-frame) to $7.63 \pm 1.81$ (denoised). The MSSIM increased from $0.13 \pm 0.02$ (single frame) to $0.65 \pm 0.03$ (denoised) when compared with the corresponding multi-frame B-scans. Conclusions: Our deep learning algorithm can denoise a single-frame OCT B-scan of the ONH in under 20 ms, thus offering a framework to obtain superior quality OCT B-scans with reduced scanning times and minimal patient discomfort. △ Less

Submitted 27 September, 2018; originally announced September 2018.

Stress relaxation in a dilute bacterial suspension: The active-passive transition

Authors: Sankalp Nambiar, Phanikanth S, Nott P. R., Ganesh Subramanian

Abstract: We analyse the time dependent non-linear rheology of a dilute bacterial suspension (e.g. E. coli) for a pair of impulsively started linear flows - simple shear and uniaxial extension. The rheology is governed by the bacterium orientation distribution which satisfies a kinetic equation that includes rotation by the imposed flow, and relaxation to isotropy via rotary diffusion and tumbling. The rele… ▽ More We analyse the time dependent non-linear rheology of a dilute bacterial suspension (e.g. E. coli) for a pair of impulsively started linear flows - simple shear and uniaxial extension. The rheology is governed by the bacterium orientation distribution which satisfies a kinetic equation that includes rotation by the imposed flow, and relaxation to isotropy via rotary diffusion and tumbling. The relevant dimensionless parameters are the Peclet number $Pe\equiv \dotγτ$, which dictates the importance of flow-induced orientation anisotropy, and $τD_r$, which quantifies the relative importance of the two intrinsic orientation decorrelation mechanisms (tumbling and rotary diffusion). Here, $τ$ is the mean run duration of a bacterium that exhibits a run-and-tumble dynamics, $D_r$ is the intrinsic rotary diffusivity of the bacterium and $\dotγ$ is the characteristic magnitude of the imposed velocity gradient. The solution of the kinetic equation is obtained numerically using a spectral Galerkin method, and yields the relevant rheological properties over the entire range of $Pe$. For simple shear, the stress relaxation predicted by our analysis at small $Pe$ is in good agreement with the experimental observations of Lopez et al. (2015). However, the analysis at large $Pe$ yields relaxations that are qualitatively different. The rheological response in the experiments corresponds to a transition from a nearly isotropic suspension of active swimmers at small $Pe$, to an apparently (nearly) isotropic suspension of passive rods at large $Pe$. In contrast, the computations yield the expected transition to a nearly flow-aligned suspension of passive rigid rods at high $Pe$. We probe this active-passive transition systematically, complementing the numerical solution with analytical solutions obtained from perturbation expansions about appropriate base states. △ Less

Submitted 6 September, 2018; originally announced September 2018.

Concentration banding instability of a sheared bacterial suspension

Authors: Laxminarsimharao V, Piyush Garg, Ganesh Subramanian

Abstract: We demonstrate a novel shear-induced mechanism for growth of concentration fluctuations in a bacterial suspension. Using a linear stability analysis, a homogeneously sheared suspension is shown to support exponentially growing layering perturbations in the shear-rate and bacterial concentration. Non-linear simulations show that the instability eventually leads to gradient-banded velocity profiles,… ▽ More We demonstrate a novel shear-induced mechanism for growth of concentration fluctuations in a bacterial suspension. Using a linear stability analysis, a homogeneously sheared suspension is shown to support exponentially growing layering perturbations in the shear-rate and bacterial concentration. Non-linear simulations show that the instability eventually leads to gradient-banded velocity profiles, with a local depletion of bacteria at the interface between the bands. Our results show that long-ranged hydrodynamic interactions are sufficient to explain recent observations of shear-bands in bacterial suspensions. △ Less

Submitted 23 August, 2018; originally announced August 2018.

Comments: 6 pages of main text and 9 pages of supplementary information

Water under the Ridge: Evaporation, Translation, Crumpling and Encapsulation of a Water Droplet atop a Liquid Polymeric Film

Authors: Sri Ganesh Subramanian, Sachin Nair, Sunando DasGupta

Abstract: The intriguing dynamics of a compound liquid-system comprising of water and (uncured) poly-dimethylsiloxane is explored in the present work. The viscoelastic nature of the film, coupled with the dynamics of evaporation, triggered a self-propulsion in the droplet, which gradually segued into the crumpling of the film, and finally culminated in the encapsulation of the water droplet by the polymer.… ▽ More The intriguing dynamics of a compound liquid-system comprising of water and (uncured) poly-dimethylsiloxane is explored in the present work. The viscoelastic nature of the film, coupled with the dynamics of evaporation, triggered a self-propulsion in the droplet, which gradually segued into the crumpling of the film, and finally culminated in the encapsulation of the water droplet by the polymer. The physics of the hitherto unreported phenomena has been explained via the development of an analytical model, by taking into account all the germane forces. It is conjectured that this symbiotic and self-sustained dynamics, aided with the non-requirement of any complex fabrication procedures, would pave the path for the development of precision drug-delivery, unmediated flow-focusing, self-mixed microreactors, the study of micro-swimmers, surface encapsulation, and photonics, to name a few. △ Less

Submitted 8 March, 2018; originally announced March 2018.

Comments: For Supplementary Information kindly contact the corresponding author

Viscoelastic pipe flow is linearly unstable

Authors: Piyush Garg, Indresh Chaudhary, Mohammad Khalid, V Shankar, Ganesh Subramanian

Abstract: Newtonian pipe flow is known to be linearly stable at all Reynolds numbers. We report, for the first time, a linear instability of pressure driven pipe flow of a viscoelastic fluid, obeying the Oldroyd-B constitutive equation commonly used to model dilute polymer solutions. The instability is shown to exist at Reynolds numbers significantly lower than those at which transition to turbulence is typ… ▽ More Newtonian pipe flow is known to be linearly stable at all Reynolds numbers. We report, for the first time, a linear instability of pressure driven pipe flow of a viscoelastic fluid, obeying the Oldroyd-B constitutive equation commonly used to model dilute polymer solutions. The instability is shown to exist at Reynolds numbers significantly lower than those at which transition to turbulence is typically observed for Newtonian pipe flow. Our results qualitatively explain experimental observations of transition to turbulence in pipe flow of dilute polymer solutions at flow rates where Newtonian turbulence is absent. The instability discussed here should form the first stage in a hitherto unexplored dynamical pathway to turbulence in polymer solutions. An analogous instability exists for plane Poiseuille flow. △ Less

Submitted 4 December, 2017; v1 submitted 21 November, 2017; originally announced November 2017.

Journal ref: Phys. Rev. Lett. 121, 024502 (2018)