-
The power of prediction: spatiotemporal Gaussian process modeling for predictive control in slope-based wavefront sensing
Authors:
Jalo Nousiainen,
Juha-Pekka Puska,
Tapio Helin,
Nuutti Hyvönen,
Markus Kasper
Abstract:
Time-delay error is a significant error source in adaptive optics (AO) systems. It arises from the latency between sensing the wavefront and applying the correction. Predictive control algorithms reduce the time-delay error, providing significant performance gains, especially for high-contrast imaging. However, the predictive controller's performance depends on factors such as the WFS type, the me…
▽ More
Time-delay error is a significant error source in adaptive optics (AO) systems. It arises from the latency between sensing the wavefront and applying the correction. Predictive control algorithms reduce the time-delay error, providing significant performance gains, especially for high-contrast imaging. However, the predictive controller's performance depends on factors such as the WFS type, the measurement noise, the AO system's geometry, and the atmospheric conditions.
This work studies the limits of prediction under different imaging conditions through spatiotemporal Gaussian process models. The method provides a predictive reconstructor that is optimal in the least-squares sense, conditioned on the fixed times series of WFS data and our knowledge of the atmosphere. We demonstrate that knowledge is power in predictive AO control. With an SHS-based extreme AO instrument, perfect knowledge of Frozen Flow evolution (wind and Cn2 profile) leads to a reduction of the residual wavefront phase variance up to a factor of 3.5 compared to a non-predictive approach. If there is uncertainty in the profile or evolution models, the gain is more modest. Still, assuming that only effective wind speed is available (without direction) led to reductions in variance by a factor of 2.3.
We also study the value of data for predictive filters by computing the experimental utility for different scenarios to answer questions such as: How many past data frames should the prediction filter consider, and is it always most advantageous to use the most recent data? We show that within the scenarios considered, more data consistently increases prediction accuracy. Further, we demonstrate that given a computational limitation on how many past frames we can use, an optimized selection of $n$ past frames leads to a 10-15% additional improvement in RMS over using the n latest consecutive frames of data.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
Laboratory Experiments of Model-based Reinforcement Learning for Adaptive Optics Control
Authors:
Jalo Nousiainen,
Byron Engler,
Markus Kasper,
Chang Rajani,
Tapio Helin,
Cédric T. Heritier,
Sascha P. Quanz,
Adrian M. Glauser
Abstract:
Direct imaging of Earth-like exoplanets is one of the most prominent scientific drivers of the next generation of ground-based telescopes. Typically, Earth-like exoplanets are located at small angular separations from their host stars, making their detection difficult. Consequently, the adaptive optics (AO) system's control algorithm must be carefully designed to distinguish the exoplanet from the…
▽ More
Direct imaging of Earth-like exoplanets is one of the most prominent scientific drivers of the next generation of ground-based telescopes. Typically, Earth-like exoplanets are located at small angular separations from their host stars, making their detection difficult. Consequently, the adaptive optics (AO) system's control algorithm must be carefully designed to distinguish the exoplanet from the residual light produced by the host star.
A new promising avenue of research to improve AO control builds on data-driven control methods such as Reinforcement Learning (RL). RL is an active branch of the machine learning research field, where control of a system is learned through interaction with the environment. Thus, RL can be seen as an automated approach to AO control, where its usage is entirely a turnkey operation. In particular, model-based reinforcement learning (MBRL) has been shown to cope with both temporal and misregistration errors. Similarly, it has been demonstrated to adapt to non-linear wavefront sensing while being efficient in training and execution.
In this work, we implement and adapt an RL method called Policy Optimization for AO (PO4AO) to the GHOST test bench at ESO headquarters, where we demonstrate a strong performance of the method in a laboratory environment. Our implementation allows the training to be performed parallel to inference, which is crucial for on-sky operation. In particular, we study the predictive and self-calibrating aspects of the method. The new implementation on GHOST running PyTorch introduces only around 700 microseconds in addition to hardware, pipeline, and Python interface latency. We open-source well-documented code for the implementation and specify the requirements for the RTC pipeline. We also discuss the important hyperparameters of the method, the source of the latency, and the possible paths for a lower latency implementation.
△ Less
Submitted 30 December, 2023;
originally announced January 2024.
-
Towards on-sky adaptive optics control using reinforcement learning
Authors:
J. Nousiainen,
C. Rajani,
M. Kasper,
T. Helin,
S. Y. Haffert,
C. Vérinaud,
J. R. Males,
K. Van Gorkom,
L. M. Close,
J. D. Long,
A. D. Hedglen,
O. Guyon,
L. Schatz,
M. Kautz,
J. Lumbres,
A. Rodack,
J. M. Knight,
K. Miller
Abstract:
The direct imaging of potentially habitable Exoplanets is one prime science case for the next generation of high contrast imaging instruments on ground-based extremely large telescopes. To reach this demanding science goal, the instruments are equipped with eXtreme Adaptive Optics (XAO) systems which will control thousands of actuators at a framerate of kilohertz to several kilohertz. Most of the…
▽ More
The direct imaging of potentially habitable Exoplanets is one prime science case for the next generation of high contrast imaging instruments on ground-based extremely large telescopes. To reach this demanding science goal, the instruments are equipped with eXtreme Adaptive Optics (XAO) systems which will control thousands of actuators at a framerate of kilohertz to several kilohertz. Most of the habitable exoplanets are located at small angular separations from their host stars, where the current XAO systems' control laws leave strong residuals.Current AO control strategies like static matrix-based wavefront reconstruction and integrator control suffer from temporal delay error and are sensitive to mis-registration, i.e., to dynamic variations of the control system geometry. We aim to produce control methods that cope with these limitations, provide a significantly improved AO correction and, therefore, reduce the residual flux in the coronagraphic point spread function.
We extend previous work in Reinforcement Learning for AO. The improved method, called PO4AO, learns a dynamics model and optimizes a control neural network, called a policy. We introduce the method and study it through numerical simulations of XAO with Pyramid wavefront sensing for the 8-m and 40-m telescope aperture cases. We further implemented PO4AO and carried out experiments in a laboratory environment using MagAO-X at the Steward laboratory. PO4AO provides the desired performance by improving the coronagraphic contrast in numerical simulations by factors 3-5 within the control region of DM and Pyramid WFS, in simulation and in the laboratory. The presented method is also quick to train, i.e., on timescales of typically 5-10 seconds, and the inference time is sufficiently small (< ms) to be used in real-time control for XAO with currently available hardware even for extremely large telescopes.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
Adaptive Optics control using Model-Based Reinforcement Learning
Authors:
Jalo Nousiainen,
Chang Rajani,
Markus Kasper,
Tapio Helin
Abstract:
Reinforcement Learning (RL) presents a new approach for controlling Adaptive Optics (AO) systems for Astronomy. It promises to effectively cope with some aspects often hampering AO performance such as temporal delay or calibration errors. We formulate the AO control loop as a model-based RL problem (MBRL) and apply it in numerical simulations to a simple Shack-Hartmann Sensor (SHS) based AO system…
▽ More
Reinforcement Learning (RL) presents a new approach for controlling Adaptive Optics (AO) systems for Astronomy. It promises to effectively cope with some aspects often hampering AO performance such as temporal delay or calibration errors. We formulate the AO control loop as a model-based RL problem (MBRL) and apply it in numerical simulations to a simple Shack-Hartmann Sensor (SHS) based AO system with 24 resolution elements across the aperture. The simulations show that MBRL controlled AO predicts the temporal evolution of turbulence and adjusts to mis-registration between deformable mirror and SHS which is a typical calibration issue in AO. The method learns continuously on timescales of some seconds and is therefore capable of automatically adjusting to changing conditions.
△ Less
Submitted 28 April, 2021;
originally announced April 2021.
-
PCS -- A Roadmap for Exoearth Imaging with the ELT
Authors:
Markus Kasper,
Nelly Cerpa Urra,
Prashant Pathak,
Markus Bonse,
Jalo Nousiainen,
Byron Engler,
Cédric Taïssir Heritier,
Jens Kammerer,
Serban Leveratto,
Chang Rajani,
Paul Bristow,
Miska Le Louarn,
Pierre-Yves Madec,
Stefan Ströbele,
Christophe Verinaud,
Adrian Glauser,
Sascha P. Quanz,
Tapio Helin,
Christoph Keller,
Frans Snik,
Anthony Boccaletti,
Gaël Chauvin,
David Mouillet,
Caroline Kulcsár,
Henri-François Raynaud
Abstract:
The Planetary Camera and Spectrograph (PCS) for the Extremely Large Telescope (ELT) will be dedicated to detecting and characterising nearby exoplanets with sizes from sub-Neptune to Earth-size in the neighbourhood of the Sun. This goal is achieved by a combination of eXtreme Adaptive Optics (XAO), coronagraphy and spectroscopy. PCS will allow us not only to take images, but also to look for biosi…
▽ More
The Planetary Camera and Spectrograph (PCS) for the Extremely Large Telescope (ELT) will be dedicated to detecting and characterising nearby exoplanets with sizes from sub-Neptune to Earth-size in the neighbourhood of the Sun. This goal is achieved by a combination of eXtreme Adaptive Optics (XAO), coronagraphy and spectroscopy. PCS will allow us not only to take images, but also to look for biosignatures such as molecular oxygen in the exoplanets' atmospheres. This article describes the PCS primary science goals, the instrument concept and the research and development activities that will be carried out over the coming years.
△ Less
Submitted 20 March, 2021;
originally announced March 2021.