Explainable Reinforcement Learning via Dynamic Mixture Policies

authored by

Maximilian Schier, Frederik Schubert, Bodo Rosenhahn

Abstract

Learning control policies using deep reinforcement learning has shown great success for a variety of applications, including robotics and automated driving. A key area limiting the adaptation of RL in the real world is the lack of trust in the decision-making process of such policies. Therefore, explainability is a requirement of any RL agent operating in the real world. In this work, we propose a family of control policies that are explainable-by-design regarding individual observation components on object-based scene representations. By estimating diagonal squashed Gaussian and categorical mixture distributions on sub-spaces of the decomposed observations, we develop stochastic policies with easy-to-read explanations of the decision-making process. Our design is generally applicable to any RL algorithm using stochastic policies. We showcase the explainability on an extensive suite of single-and multi-agent simulations, set-and sequence-based high-level scenes, and discrete and continuous action spaces, with performance at least on-par or better compared to standard policy architectures. In additional experiments, we analyze the robustness of our approach to its single additional hyper-parameter and examine its potential for very low computational requirements with tiny policies.

Organisation(s)

L3S Research Centre
Institute of Information Processing

Type

Conference contribution

Pages

13530-13536

No. of pages

7

Publication date

19.05.2025

Publication status

Published

Peer reviewed

Yes

ASJC Scopus subject areas

Software, Control and Systems Engineering, Artificial Intelligence, Electrical and Electronic Engineering

Electronic version(s)

https://doi.org/10.1109/ICRA55743.2025.11127782 (Access: Closed)