Gitta Kutyniok

🚀 Exciting News! 🚀 We are thrilled to announce that Kumo AI's position paper "Relational Deep Learning: Graph Representation Learning on Relational Databases," has been accepted at #ICML 2024! 🎉 Join us in Vienna from July 21-27, where our very own Matthias Fey will be presenting this exciting new research area. 🔗 https://icml.cc/ In collaboration with Stanford University, we introduce Relational Deep Learning (#RDL), a new research area for applying deep learning on relational databases by viewing them as temporal, heterogeneous graphs. This method eliminates the need for manual feature engineering, leading to more accurate models built faster. 🚀 We’ve also developed #RelBench, a benchmark suite to facilitate research in this new area, using diverse datasets from Stack Exchange discussions to Amazon book reviews. 📚 Stay tuned for more updates on our presentation format, and check out our paper here: https://lnkd.in/dk24vbSg #ICML2024 #MachineLearning #AI #GraphNeuralNetworks #RelationalDeepLearning #DataScience #KumoAI #StanfordCollaboration

36

Interesting paper on benchmarking: "Towards Robust Benchmarking of Quantum Optimization Algorithms" by David Bucher, Nico Kraus, Jonas Blenninger, Michael Lachner, Jonas Stein and Claudia Linnhoff-Popien Abstract: Benchmarking the performance of quantum optimization algorithms is crucial for identifying utility for industry-relevant use cases. Benchmarking processes vary between optimization applications and depend on user-specified goals. The heuristic nature of quantum algorithms poses challenges, especially when comparing to classical counterparts. A key problem in existing benchmarking frameworks is the lack of equal effort in optimizing for the best quantum and, respectively, classical approaches. This paper presents a comprehensive set of guidelines comprising universal steps towards fair benchmarks. We discuss (1) application-specific algorithm choice, ensuring every solver is provided with the most fitting mathematical formulation of a problem; (2) the selection of benchmark data, including hard instances and real-world samples; (3) the choice of a suitable holistic figure of merit, like time-to-solution or solution quality within time constraints; and (4) equitable hyperparameter training to eliminate bias towards a particular method. The proposed guidelines are tested across three benchmarking scenarios, utilizing the Max-Cut (MC) and Travelling Salesperson Problem (TSP). The benchmarks employ classical mathematical algorithms, such as Branch-and-Cut (BNC) solvers, classical heuristics, Quantum Annealing (QA), and the Quantum Approximate Optimization Algorithm (QAOA). Link: https://lnkd.in/eh826qZi #quantumcomputing #quantummachinelearning #quantumoptimization #researchpaper #research

139

Congratulations to my student Yunlong Song for successfully defending his Ph.D. in “Learning Robot Control: From #ReinforcementLearning to #DifferentiableSimulation”! Many thanks to the external reviewers Yuke Zhu from the U. of Texas at Austin, Marco Hutter from ETH Zurich, and Martin Riedmiller from Google DeepMind! Yunlong's contributions are: - To show that Reinforcement Learning (RL) outperforms Optimal Control in autonomous racing because it directly optimizes a non-differentiable task-level objective. - To propose a policy-search-for-model-predictive-control (MPC) framework, combining RL's ability to optimize high-level task objectives with MPC's precise actuation and constraint handling. - To introduce a differentiable simulation framework to leverage robot dynamics for more stable and efficient policy training. - To develop a high-performance drone racing system that outperforms optimal control methods and professional pilots. - To develop Flightmare, a flexible modular quadrotor simulator for reinforcement learning and vision-based flight. - Video Recording of the PhD defense: https://lnkd.in/d53QGNck - Yunlong's webpage (publications, source code, slides): https://lnkd.in/dUS68yUH - Google Scholar: https://lnkd.in/d-UMw7bT Congratulations, Yunlong; it has been an honor to work with you! University of Zurich, NCCR Robotics, European Research Council (ERC) UZH Innovation Hub #deeplearning #robotics #machinelearning

819

22 Kommentare

Heiko Rölke will present our paper on irrational number computations at the International Symposium on Parallel and Distributed Computing (ISPDC)... Abstract Irrational numbers are difficult to compute due to their non-rational nature, i.e. they cannot just be expressed as a ratio consisting of two integer numbers. Various examples of these irrational constants exist, like the circle’s circumference to its diameter π, Euler’s number e, the Golden ratio φ, or the square roots of 2, 3, or 5, and the like. Irrational numbers can be expressed as a sequence of decimal digits just like other real numbers, however, they do not terminate nor do they contain some kind of repeating number pattern. The goal of our work is to efficiently compute and store a very large irrational number by computing as many digits as possible. This test is designed to improve our ability to compute and process extremely large datasets in order to prepare ourselves for future real-world applications based on computational-intensive data analytics concepts, hence high-performance computing is required to solve such data-related problems. To reach these goals we make use of two AMD processors with one terabyte of RAM and 64 cores in total, and an aggregated 510 terabytes of disk space. To run such a computation, we employ 38 hard disk drives running at 7,200 rpm. 34 of these disks are used as swap space and 4 disks will hold the final digits. We illustrate our approach by applying it to the irrational number π and compute 62.8 trillion decimal places of it. Finally, we discuss scalability issues, limitations, challenges, and future directions of our methodology.

7

"FAIR-USE4OS: Guidelines for creating impactful open-source software" just published in PLOS Computational Biology. Thanks to all my co-authors Hugo Gruson Leo Wolansky Agnes Kiragga Daniel S. Katz. Just adding an open-source licence to some code does not mean it's truly accessible and is no guarantee of long-term impact. In this paper we extend the FAIR guidelines to evaluate the usability and accessibility of open-source software by adding USE: User-Centered, Sustainable, Equitable. We hope this is useful to developers of open-source software and in particular to researchers and funders to effectively plan and evaluate open-source projects. https://lnkd.in/et9Bx2hT

15

2 Kommentare

From Cleveland State University A First Look at Kolmogorov-Arnold Networks in Surrogate-assisted Evolutionary Algorithms https://lnkd.in/eu9PstMT An interesting paper on the applications of Kolmogorov Arnold Network to evolutionary algorithms for optimisation shows strong potential of KANs and significant superior data fitting capabilities compared to MLPs. abs: Surrogate-assisted Evolutionary Algorithm (SAEA) is an essential method for solving expensive expensive problems. Utilizing surrogate models to substitute the optimization function can significantly reduce reliance on the function evaluations during the search process, thereby lowering the optimization costs. The construction of surrogate models is a critical component in SAEAs, with numerous machine learning algorithms playing a pivotal role in the model-building phase. This paper introduces Kolmogorov-Arnold Networks (KANs) as surrogate models within SAEAs, examining their application and effectiveness. We employ KANs for regression and classification tasks, focusing on the selection of promising solutions during the search process, which consequently reduces the number of expensive function evaluations. Experimental results indicate that KANs demonstrate commendable performance within SAEAs, effectively decreasing the number of function calls and enhancing the optimization efficiency. The relevant code is publicly accessible and can be found in the GitHub repository. https://lnkd.in/e5aqF4zY #ai #NeuralComputing #EvolutionaryComputing

17

Instead of throwing "everything and the kitchen sink" into the ML model, when you break the features down into theoretically-related groups, multifractal descriptors do better than many other feature groups at classifying five different kinds of anomalous diffusion. Thanks to Henrik Seckler, Ralf Metzler, and Madhur Mangalam for letting me join this exciting project "Multifractal spectral features enhance classification of anomalous diffusion" just out now: https://lnkd.in/e2MNtKBJ

25

#Learning #QuantumComputing #qubitstensors #PauliMatrices Wolfang Pauli introduced three quantum gates represented as X, Y and Z. Collectively they are called Pauli Matrices. X is also known as NOT gate. All three are single qubit gates.

3

Our simple #stride-ai methodology for modeling threats to #ai-ml pipelines is gaining traction! Interested in securing machine learning applications? Take a look at our paper (it reached 600 reads in a few weeks) and check out the #github site with the complete document kit. Lara Mauri Dr Shreekant (Ticky) Thakkar Chan Yeob Yeun

16

Q&A with Sepp Hochreiter at the Cyber Valley Days hosted by the new ELLIS Institute in Tübingen : - Q: companies have made billions with your invention, LSTM. Have you gotten anything back. - A: No. Also, my advisor overlooked the part about LSTM in my PhD dissertation, so it wasn't published until years later. But more recently, we published the "ELU" activation function and I got a mojito out of it. - Q: what is the future of LLMs? - A: We need to get rid of this transformer nonsense. It's very inefficient. People do not pay attention to a large window of words, they remember abstract representations of past text. We've proposed the xLSTM architecture for that. But it costs a lot of money to test it at a large scale. So we created a startup.

1.854

86 Kommentare

I have a blog with a lot of mathematical models (mostly puzzle problems) and each 5th days a new is uploaded. See https://matmod.ch/blog/

44

1 Kommentar

This morning, I read an interesting article: "Benchmarking Optimizers for Qumode State Preparation with Variational Quantum Algorithms" by Shuwen Kan et al. Abstract: —Quantum state preparation involves preparing a target state from an initial system, a process integral to applications such as quantum machine learning and solving systems of linear equations. Recently, there has been a growing interest in qumodes due to advancements in the field and their potential applications. However there is a notable gap in the literature specifically addressing this area. This paper aims to bridge this gap by providing performance benchmarks of various optimizers used in state preparation with Variational Quantum Algorithms. We conducted extensive testing across multiple scenarios, including different target states, both ideal and sampling simulations, and varying numbers of basis gate layers. Our evaluations offer insights into the complexity of learning each type of target state and demonstrate that some optimizers perform better than others in this context. Notably, the Powell optimizer was found to be exceptionally robust against sampling errors, making it a preferred choice in scenarios prone to such inaccuracies. Additionally, the Simultaneous Perturbation Stochastic Approximation optimizer was distinguished for its efficiency and ability to handle increased parameter dimensionality effectively. Link: https://lnkd.in/dkB8hBHC #quantumalgorithms #quantummachinelearning #quantumcomputing #quantumoptimizers

30

Workflows for Causal ML in Medicine? This new Nature Medicine paper offers a perspective.

4

We are thrilled to share our groundbreaking paper published today in Nature: "Low Latency Automotive Vision with Event Cameras." Paper: https://lnkd.in/dFEtFsxU Video: https://lnkd.in/dAr4CX5g Code & Dataset: https://lnkd.in/djYW6mKH Frame-based sensors such as the RGB cameras used in the automotive industry face a bandwidth–latency trade-off: higher frame rates reduce perceptual latency but increase bandwidth demands, whereas lower frame rates save bandwidth at the cost of missing vital scene dynamics due to increased perceptual latency (see Fig. 1a of the paper). Event cameras have emerged as alternative vision sensors to address this trade-off. Event cameras measure the changes in intensity asynchronously, offering high temporal resolution and sparsity, markedly reducing bandwidth and latency requirements. Despite these advantages, event-camera-based algorithms are either highly efficient but lag behind image-based ones in accuracy or sacrifice the sparsity and efficiency of events to achieve comparable results. To overcome this, we propose a hybrid event- and frame-based object detector based on Deep Asynchronous GNNs, which preserves the advantages of each modality and thus does not suffer from this trade-off. Our method exploits the high temporal resolution and sparsity of events and the rich but low temporal resolution information in standard images to generate efficient, high-rate object detections, reducing perceptual and computational latency. In doing so, it emulates the slow-fast pathways in biological neural networks and uses them to its advantage. We show that using a 20-Hz RGB camera plus an event camera achieves the same latency as a 5,000-Hz camera with the bandwidth of a 50-Hz camera, i.e. an over 100-fold bandwidth reduction, without compromising accuracy. Our approach paves the way for efficient and robust perception in edge-case scenarios by uncovering the potential of event cameras. We release the code and the dataset (DSEC-Detection) to the public. Kudos to Daniel Gehrig, who, with this work, also received the UZH Annual Award for the Best PhD thesis! **Reference** Daniel Gehrig, Davide Scaramuzza Low Latency Automotive Vision with Event Cameras Nature, May 29, 2024. DOI: 10.1038/s41586-024-07409-w PDF (Open Access): https://lnkd.in/dFEtFsxU Video (Narrated): https://lnkd.in/dAr4CX5g Code & Datasets: https://lnkd.in/djYW6mKH   University of Zurich UZH Innovation Hub European Research Council (ERC) NCCR Robotics Switzerland Innovation Park Zurich University of Zurich Faculty of Science

2.726

55 Kommentare

A new lecture in the Course: "Introduction to Quantum Computing", has been published: https://lnkd.in/d4AgycWv Lecture 17: The Deutsch-Jozsa Problem In this lecture we generalize the Deutsch's problem. Now the input query function is a function over binary strings of length n, and the goal is to determine again whether the function is constant or balanced, given it is not otherwise as a promise. The quantum solution has an advantage over the classical solution whether deterministic or probabilistic. In addition, the classical probabilistic solution has an advantage over the classical deterministic if we can accept a small possibility of erroneous output. The instruction language is Arabic, however, all material, slides, concepts, definitions, examples, illustrations, etc., are all given in English.

10

🚀 Thrilled to announce that our paper "Domain-Aware Fine-Tuning of Foundation Models" based on my thesis work at the University of Tuebingen and Bosch Center for Artificial Intelligence (BCAI) is published at the ICML 2024 Workshop on Foundation Models in the Wild! 🎉 We propose Domino (Domain Adaptive Normalization) to make foundation models more robust and adaptable to unseen domains by introducing domain embeddings during fine-tuning to enhance domain awareness. Our experiments show that: 1️⃣ The Stable Diffusion-based backbones achieves superior generalization performance for semantic segmentation 2️⃣Domino significantly enhances model generalization by leveraging domain information effectively 📈 Joint work with Margret Keuper, Anna Khoreva, Dan Zhang, and Yumeng Li at Bosch Center for AI (BCAI) 🙌 Read more about our findings and explore the potential of foundation models adapting to new domains: https://lnkd.in/eyDsjRFp #AI #MachineLearning #ComputerVision #DomainAdaptation #GenAI #FoundationModels #Research #ICML2024 #BoschAI #UniversityofTübingen

73

4 Kommentare

Dynamically available total-order broadcast (TOB) protocols tolerate fluctuating participation, e.g., as high as 99% of their participants going offline, which is especially useful in permissionless blockchain environments. However, dynamically available TOB protocols are synchronous protocols, and they lose their safety guarantees during periods of asynchrony. This is a major issue in practice. In this paper, we propose a simple but effective mechanism for tolerating bounded periods of asynchrony in dynamically available TOB protocols that ensure safety deterministically. We propose to trade off assumptions limiting the online/offline churn rate in exchange for tolerating bounded asynchronous periods through the use of a configurable message-expiration period. In practice, this allows picking a small synchrony bound 𝛿, and therefore obtain a fast protocol in the common case, knowing that the protocol tolerates occasional periods of duration at most 𝜋 > 𝛿 during which the bound does not hold.

16

Happy to share that our next AI seminar at Technology Innovation Institute will be given by Prof. Lenka Zdeborova from EPFL. Prof. Zdeborova will walk us through phase transitions between positional and semantic learning in the attention mechanism using a mathematically tractable dot-product attention model. 📆 Date & time: 25th June, 2024, 3:00pm (GST) 👉🏻 Register at https://lnkd.in/dwJVZibg 👩‍🏫 Lenka Zdeborová is a Professor of Physics and Computer Science at École Polytechnique Fédérale de Lausanne, where she leads the Statistical Physics of Computation Laboratory. She received a PhD in physics from the University of Paris-Sud and Charles University in Prague in 2008. She spent two years in the Los Alamos National Laboratory as the Director's Postdoctoral Fellow. Between 2010 and 2020, she was a researcher at CNRS, working in the Institute of Theoretical Physics in CEA Saclay, France. In 2014, she was awarded the CNRS bronze medal, in 2016 Philippe Meyer prize in theoretical physics and an ERC Starting Grant, in 2018 the Irène Joliot-Curie prize, in 2021 the Gibbs lectureship of AMS and the Neuron Fund award. She was an editorial board member for the Journal of Physics A, Physical Review E, Physical Review X, SIMODS, Machine Learning: Science and Technology, and Information and Inference. Lenka's expertise is in the application of concepts from statistical physics, such as advanced mean field methods, the replica method, and related message-passing algorithms, to problems in machine learning, signal processing, inference, and optimization. She enjoys erasing the boundaries between theoretical physics, mathematics and computer science. #attention #transformers #LLMs #AI #seminar #theory

69