Ignacio Carlucho

2023

1. Enhancing AUV Autonomy With Model Predictive Path Integral Control

   Pierre Nicolay, Yvan Petillot, Mykhaylo Marfeychuk, and 2 more authors

   In OCEANS 2023, 2023

   Abs arXiv

   Autonomous underwater vehicles (AUVs) play a crucial role in surveying marine environments, carrying out underwater inspection tasks, and ocean exploration. However, in order to ensure that the AUV is able to carry out its mission successfully, a control system capable of adapting to changing environmental conditions is required. Furthermore, to ensure the robotic platform’s safe operation, the onboard controller should be able to operate under certain constraints. In this work, we investigate the feasibility of Model Predictive Path Integral Control (MPPI) for the control of an AUV. We utilise a non-linear model of the AUV to propagate the samples of the MPPI, which allow us to compute the control action in real time. We provide a detailed evaluation of the effect of the main hyperparameters on the performance of the MPPI controller. Furthermore, we compared the performance of the proposed method with a classical PID and Cascade PID approach, demonstrating the superiority of our proposed controller. Finally, we present results where environmental constraints are added and show how MPPI can handle them by simply incorporating those constraints in the cost function.

2. 

   Semi-Parametric Control Architecture for Autonomous Underwater Vehicles Subject to Time Delays

   Ignacio Carlucho, Dylan Stephens, William Ard, and 1 more author

   IEEE Access, 2023

   Abs

   This paper presents a data-driven model-based control system for autonomous underwater vehicles (or AUVs) subject to input delays. This work is motivated by the input time delays that can arise in underwater robotics due to communication restrictions and sensor malfunctions. Such delays can highly degrade the performance of classical control structures resulting in unpredictable system behaviours. The proposed control architecture addresses such limitations. The approach incorporates a linear dynamic representation of the system obtained using the Koopman operator in an observer/state prediction formulation. The proposed control architecture is designed based on discrepancies between the data-driven estimation of the system’s behaviour and the actual AUV performance using chain predictors. The capabilities of the proposed approach are shown through experiments performed with a 4 degrees-of-freedom autonomous underwater vehicle. The results demonstrate stable behaviours without steady-state errors, even in the presence of long delay.

3. 

   A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learning

   Arrasy Rahman, Ignacio Carlucho, Niklas Höpner, and 1 more author

   Journal of Machine Learning Research, 2023

   Abs arXiv Code

   Open ad hoc teamwork is the problem of training a single agent to efficiently collaborate with an unknown group of teammates whose composition may change over time. A variable team composition creates challenges for the agent, such as the requirement to adapt to new team dynamics and dealing with changing state vector sizes. These challenges are aggravated in real-world applications where the controlled agent has no access to the full state of the environment. In this work, we develop a class of solutions for open ad hoc teamwork under full and partial observability. We start by developing a solution for the fully observable case that leverages graph neural network architectures to obtain an optimal policy based on reinforcement learning. We then extend this solution to partially observable scenarios by proposing different methodologies that maintain belief estimates over the latent environment states and team composition. These belief estimates are combined with our solution for the fully observable case to compute an agent’s optimal policy under partial observability in open ad hoc teamwork. Empirical results demonstrate that our approach can learn efficient policies in open ad hoc teamwork in full and partially observable cases. Further analysis demonstrates that our methods’ success is a result of effectively learning the effects of teammates’ actions while also inferring the inherent state of the environment under partial observability.

4. 

   Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversity

   Arrasy Rahman, Elliot Fosong, Ignacio Carlucho, and 1 more author

   Transactions on Machine Learning Research, 2023

   Abs arXiv Bib Code

   Ad hoc teamwork (AHT) is the challenge of designing a robust learner agent that effectively collaborates with unknown teammates without prior coordination mechanisms. Early approaches address the AHT challenge by training the learner with a diverse set of handcrafted teammate policies, usually designed based on an expert’s domain knowledge about the policies the learner may encounter. However, implementing teammate policies for training based on domain knowledge is not always feasible. In such cases, recent approaches attempted to improve the robustness of the learner by training it with teammate policies generated by optimising information-theoretic diversity metrics. The problem with optimising existing information-theoretic diversity metrics for teammate policy generation is the emergence of superficially different teammates. When used for AHT training, superficially different teammate behaviours may not improve a learner’s robustness during collaboration with unknown teammates. In this paper, we present an automated teammate policy generation method optimising the Best-Response Diversity (BRDiv) metric, which measures diversity based on the compatibility of teammate policies in terms of returns. We evaluate our approach in environments with multiple valid coordination strategies, comparing against methods optimising information-theoretic diversity metrics and an ablation not optimising any diversity metric. Our experiments indicate that optimising BRDiv yields a diverse set of training teammate policies that improve the learner’s performance relative to previous teammate generation approaches when collaborating with near-optimal previously unseen teammate policies.

   @article{rahman2023generating,
     title = {Generating Teammates for Training Robust Ad Hoc Teamwork Agents via Best-Response Diversity},
     author = {Rahman, Arrasy and Fosong, Elliot and Carlucho, Ignacio and Albrecht, Stefano V},
     journal = {Transactions on Machine Learning Research},
     issn = {2835-8856},
     year = {2023},
     url = {https://openreview.net/forum?id=l5BzfQhROl},
     note = {},
   }