Showing 1–9 of 9 results for author: Schofield, D

Developing Healthcare Language Model Embedding Spaces

Authors: Niall Taylor, Dan Schofield, Andrey Kormilitzin, Dan W Joyce, Alejo Nevado-Holgado

Abstract: Pre-trained Large Language Models (LLMs) often struggle on out-of-domain datasets like healthcare focused text. We explore specialized pre-training to adapt smaller LLMs to different healthcare datasets. Three methods are assessed: traditional masked language modeling, Deep Contrastive Learning for Unsupervised Textual Representations (DeCLUTR), and a novel pre-training objective utilizing metadat… ▽ More Pre-trained Large Language Models (LLMs) often struggle on out-of-domain datasets like healthcare focused text. We explore specialized pre-training to adapt smaller LLMs to different healthcare datasets. Three methods are assessed: traditional masked language modeling, Deep Contrastive Learning for Unsupervised Textual Representations (DeCLUTR), and a novel pre-training objective utilizing metadata categories from the healthcare settings. These schemes are evaluated on downstream document classification tasks for each dataset, with additional analysis of the resultant embedding spaces. Contrastively trained models outperform other approaches on the classification tasks, delivering strong performance from limited labeled data and with fewer model parameter updates required. While metadata-based pre-training does not further improve classifications across the datasets, it yields interesting embedding cluster separability. All domain adapted LLMs outperform their publicly available general base LLM, validating the importance of domain-specialization. This research illustrates efficient approaches to instill healthcare competency in compact LLMs even under tight computational budgets, an essential capability for responsible and sustainable deployment in local healthcare settings. We provide pre-training guidelines for specialized healthcare LLMs, motivate continued inquiry into contrastive objectives, and demonstrates adaptation techniques to align small LLMs with privacy-sensitive medical tasks. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Count, Crop and Recognise: Fine-Grained Recognition in the Wild

Authors: Max Bain, Arsha Nagrani, Daniel Schofield, Andrew Zisserman

Abstract: The goal of this paper is to label all the animal individuals present in every frame of a video. Unlike previous methods that have principally concentrated on labelling face tracks, we aim to label individuals even when their faces are not visible. We make the following contributions: (i) we introduce a 'Count, Crop and Recognise' (CCR) multistage recognition process for frame level labelling. The… ▽ More The goal of this paper is to label all the animal individuals present in every frame of a video. Unlike previous methods that have principally concentrated on labelling face tracks, we aim to label individuals even when their faces are not visible. We make the following contributions: (i) we introduce a 'Count, Crop and Recognise' (CCR) multistage recognition process for frame level labelling. The Count and Recognise stages involve specialised CNNs for the task, and we show that this simple staging gives a substantial boost in performance; (ii) we compare the recall using frame based labelling to both face and body track based labelling, and demonstrate the advantage of frame based with CCR for the specified goal; (iii) we introduce a new dataset for chimpanzee recognition in the wild; and (iv) we apply a high-granularity visualisation technique to further understand the learned CNN features for the recognition of chimpanzee individuals. △ Less

Submitted 9 October, 2019; v1 submitted 19 September, 2019; originally announced September 2019.

Diversification, Volatility, and Surprising Alpha

Authors: Adrian Banner, Robert Fernholz, Vassilios Papathanakos, Johannes Ruf, David Schofield

Abstract: It has been widely observed that capitalization-weighted indexes can be beaten by surprisingly simple, systematic investment strategies. Indeed, in the U.S. stock market, equal-weighted portfolios, random-weighted portfolios, and other naive, non- optimized portfolios tend to outperform a capitalization-weighted index over the long term. This outperformance is generally attributed to beneficial fa… ▽ More It has been widely observed that capitalization-weighted indexes can be beaten by surprisingly simple, systematic investment strategies. Indeed, in the U.S. stock market, equal-weighted portfolios, random-weighted portfolios, and other naive, non- optimized portfolios tend to outperform a capitalization-weighted index over the long term. This outperformance is generally attributed to beneficial factor exposures. Here, we provide a deeper, more general explanation of this phenomenon by decomposing portfolio log-returns into an average growth and an excess growth component. Using a rank-based empirical study we argue that the excess growth component plays the major role in explaining the outperformance of naive portfolios. In particular, individual stock growth rates are not as critical as is traditionally assumed. △ Less

Submitted 11 September, 2018; originally announced September 2018.

Compactifications of the Klebanov-Witten CFT and new $AdS_3$ backgrounds

Authors: Yago Bea, Jose D. Edelstein, Georgios Itsios, Karta S. Kooner, Carlos Nunez, Daniel Schofield, J. Anibal Sierra-Garcia

Abstract: In this paper we find various new backgrounds in Type IIB, IIA and M-theory with an $AdS_3$-factor. The solutions are smooth and preserve small amounts of SUSY. These new backgrounds are found by application of non-Abelian T-duality (sometimes combined with T-duality) on the supergravity solution dual to the Klebanov-Witten CFT compactified to two dimensions. The field theory aspects encoded by th… ▽ More In this paper we find various new backgrounds in Type IIB, IIA and M-theory with an $AdS_3$-factor. The solutions are smooth and preserve small amounts of SUSY. These new backgrounds are found by application of non-Abelian T-duality (sometimes combined with T-duality) on the supergravity solution dual to the Klebanov-Witten CFT compactified to two dimensions. The field theory aspects encoded by these backgrounds are studied. We give a detailed account of conserved charges, central charges, entanglement entropy and Wilson loops. Further, we present a possible field theory interpretation for our backgrounds. △ Less

Submitted 25 March, 2015; originally announced March 2015.

Comments: 38 pages plus appendices, 6 figures

Confinement, Phase Transitions and non-Locality in the Entanglement Entropy

Authors: Uri Kol, Carlos Nunez, Daniel Schofield, Jacob Sonnenschein, Michael Warschawski

Abstract: In this paper we study the conjectural relation between confinement in a quantum field theory and the presence of a phase transition in its corresponding entanglement entropy. We determine the sufficient conditions for the latter and compare to the conditions for having a confining Wilson line. We demonstrate the relation in several examples. Superficially, it may seem that certain confining field… ▽ More In this paper we study the conjectural relation between confinement in a quantum field theory and the presence of a phase transition in its corresponding entanglement entropy. We determine the sufficient conditions for the latter and compare to the conditions for having a confining Wilson line. We demonstrate the relation in several examples. Superficially, it may seem that certain confining field theories with a non-local high energy behaviour, like the dual of D5 branes wrapping a two-cycle, do not admit the corresponding phase transition. However, upon closer inspection we find that, through the introduction of a regulating UV-cutoff, new eight-surface configurations appear, that satisfy the correct concavity condition and recover the phase transition in the entanglement entropy. We show that a local-UV-completion to the confining non-local theories has a similar effect to that of the aforementioned cutoff. △ Less

Submitted 11 March, 2014; originally announced March 2014.

Comments: 64 pages. Lots of figures

Gauge/gravity dualities and bulk phase transitions

Authors: Anton F. Faedo, Maurizio Piai, Daniel Schofield

Abstract: We consider D7-branes probing several classes of Type IIB supergravity backgrounds, and study the classical problem of finding equilibrium configurations for the embedding functions. This is a method employed to model chiral symmetry breaking in the gravity dual of a strongly-coupled, confining gauge theory. We unveil and discuss a new type of phase transition appearing in the gravity systems, whi… ▽ More We consider D7-branes probing several classes of Type IIB supergravity backgrounds, and study the classical problem of finding equilibrium configurations for the embedding functions. This is a method employed to model chiral symmetry breaking in the gravity dual of a strongly-coupled, confining gauge theory. We unveil and discuss a new type of phase transition appearing in the gravity systems, which is similar in nature and meaning to bulk phase transitions on the lattice. The existence of this genre of phase transition puts a new, intrinsic limit on the region of parameter space which can be used to study the physics of the dual field theory. We complete the analysis of D7 embeddings in wrapped-D5 supergravity backgrounds, and explain in what cases chiral-symmetry breaking is sensibly modelled by the gravity construction. △ Less

Submitted 25 April, 2014; v1 submitted 17 February, 2014; originally announced February 2014.

Comments: 15 pages, 9 figures; v2 minor corrections, results and analysis unaffected; version to appear in Phys.Rev.D

Report number: ICCUB-14-043

Journal ref: Phys. Rev. D 89, 106001 (2014)

On the stability of multi-scale models of dynamical symmetry breaking from holography

Authors: Anton F. Faedo, Maurizio Piai, Daniel Schofield

Abstract: We consider two classes of backgrounds of Type IIB supergravity obtained by wrapping D5-branes on a two-cycle inside the conifold. The field theory dual exhibits confinement and, in addition, a region in which the dynamics is walking, at least in the weak sense that the running of the coupling is anomalously slow. We introduce quenched matter in the fundamental, modelled by probe D7-branes which w… ▽ More We consider two classes of backgrounds of Type IIB supergravity obtained by wrapping D5-branes on a two-cycle inside the conifold. The field theory dual exhibits confinement and, in addition, a region in which the dynamics is walking, at least in the weak sense that the running of the coupling is anomalously slow. We introduce quenched matter in the fundamental, modelled by probe D7-branes which wrap an internal three-dimensional manifold and lie at the equator of the transverse two-sphere. In the space spanned by the remaining internal angle and the radial coordinate the branes admit two embeddings. The first one is U-shaped: the branes merge at some finite value of the radius. The second one is disconnected and extends along the entire radial direction at fixed angular separation. We interpret these two configurations as corresponding to chiral-symmetry breaking and preserving phases, respectively. We present a simple diagnostic tool to examine the classical stability of the embedding, based on the concavity/convexity conditions for the relevant thermodynamic potentials. We use this criterion to show that U-shaped probes that explore the walking region are unstable, hence providing a dynamical origin for the tachyonic mode found in the literature. Whenever this occurs, the disconnected solution becomes favored energetically. We find that in one of the two classes of backgrounds the U-shaped embedding is always unstable, and thus never realised dynamically. Consequently, these models cannot be used to describe chiral-symmetry breaking. In the second category of solutions, our analysis reveals the presence of a first-order phase transition between chiral-symmetry broken and restored phases. Interestingly, this is in the same class that contains a parametrically light scalar in the spectrum of glueballs of the dual field theory. △ Less

Submitted 24 February, 2014; v1 submitted 10 December, 2013; originally announced December 2013.

Comments: 17 pages, 8 figures; v2 minor corrections, results and analysis unaffected, references amended

Report number: ICCUB-13-246

The Structure of the Non-SUSY Baryonic Branch of Klebanov-Strassler

Authors: Stephen Bennett, Daniel Schofield

Abstract: We study the two-dimensional space of supergravity solutions corresponding to non-supersymmetric deformations of the baryonic branch of Klebanov-Strassler. By combining analytical methods with a numerical survey of the parameter space, we find that this solution space includes as limits the softly-broken N=1 solutions of Gubser et al. and those of Dymarsky and Kuperstein. We also identify a one-di… ▽ More We study the two-dimensional space of supergravity solutions corresponding to non-supersymmetric deformations of the baryonic branch of Klebanov-Strassler. By combining analytical methods with a numerical survey of the parameter space, we find that this solution space includes as limits the softly-broken N=1 solutions of Gubser et al. and those of Dymarsky and Kuperstein. We also identify a one-dimensional family of solutions corresponding to a natural non-supersymmetric generalisation of Klebanov-Strassler, and one corresponding to the limit in which supersymmetry is completely absent, even in the far UV. For almost all of the parameter space we find indications that much of the structure of the supersymmetric baryonic branch survives. △ Less

Submitted 12 April, 2012; originally announced April 2012.

Comments: 29 pages plus appendices, 11 figures

The Non-SUSY Baryonic Branch: Soft Supersymmetry Breaking of N=1 Gauge Theories

Authors: Stephen Bennett, Elena Caceres, Carlos Nunez, Daniel Schofield, Steve Young

Abstract: We study a non-supersymmetric deformation of the field theory dual to the baryonic branch of Klebanov-Strassler. Using a combination of analytical (series expansions) and numerical methods we construct non-supersymmetric backgrounds that smoothly interpolate between the desired UV and IR behaviors. We calculate various observables of the field theory and propose a picture of soft breaking by gaugi… ▽ More We study a non-supersymmetric deformation of the field theory dual to the baryonic branch of Klebanov-Strassler. Using a combination of analytical (series expansions) and numerical methods we construct non-supersymmetric backgrounds that smoothly interpolate between the desired UV and IR behaviors. We calculate various observables of the field theory and propose a picture of soft breaking by gaugino masses that is consistent with the various calculations on the string side. △ Less

Submitted 7 November, 2011; originally announced November 2011.

Comments: 32 pages plus many appendixes. One figure