Nicolas Heess

Orcid: 0000-0001-7876-9256

Affiliations:
  • University College London, Centre for Computational Statistics and Machine Learning
  • University of Edinburgh, Institute for Adaptive and Neural Computation


According to our database1, Nicolas Heess authored at least 167 papers between 2009 and 2024.

Collaborative distances:

Timeline

Legend:

Book 
In proceedings 
Article 
PhD thesis 
Dataset
Other 

Links

Online presence:

On csauthors.net:

Bibliography

2024
RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation.
Trans. Mach. Learn. Res., 2024

Learning agile soccer skills for a bipedal robot with deep reinforcement learning.
Sci. Robotics, 2024

Preference Optimization as Probabilistic Inference.
CoRR, 2024

DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots.
CoRR, 2024

A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning.
CoRR, 2024

Deep Dive into Model-free Reinforcement Learning for Biological and Robotic Systems: Theory and Practice.
CoRR, 2024

Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning.
CoRR, 2024

Learning to Learn Faster from Human Feedback with Language Model Predictive Control.
CoRR, 2024

Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collaboration.
, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Proceedings of the IEEE International Conference on Robotics and Automation, 2024

Mastering Stacking of Diverse Shapes with Large-Scale Iterative Reinforcement Learning on Real Robots.
Proceedings of the IEEE International Conference on Robotics and Automation, 2024

Offline Actor-Critic Reinforcement Learning Scales to Large Models.
Proceedings of the Forty-first International Conference on Machine Learning, 2024

PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs.
Proceedings of the Forty-first International Conference on Machine Learning, 2024


Replay across Experiments: A Natural Extension of Off-Policy RL.
Proceedings of the Twelfth International Conference on Learning Representations, 2024

NfgTransformer: Equivariant Representation Learning for Normal-form Games.
Proceedings of the Twelfth International Conference on Learning Representations, 2024

Neural Population Learning beyond Symmetric Zero-Sum Games.
Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2024

The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models.
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024

2023
SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Exploration.
Trans. Mach. Learn. Res., 2023

Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities.
CoRR, 2023

TacticAI: an AI assistant for football tactics.
CoRR, 2023

Policy composition in reinforcement learning via multi-objective policy optimization.
CoRR, 2023

Towards A Unified Agent with Foundation Models.
CoRR, 2023

RoboCat: A Self-Improving Foundation Agent for Robotic Manipulation.
CoRR, 2023

Barkour: Benchmarking Animal-level Agility with Quadruped Robots.
CoRR, 2023

A Generalist Dynamics Model for Control.
CoRR, 2023

Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domains.
CoRR, 2023

Coherent Soft Imitation Learning.
Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fields.
Proceedings of the IEEE International Conference on Robotics and Automation, 2023

Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation.
Proceedings of the Eleventh International Conference on Learning Representations, 2023

Stateful Active Facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning.
Proceedings of the Eleventh International Conference on Learning Representations, 2023


Representation Learning in Deep RL via Discrete Information Bottleneck.
Proceedings of the International Conference on Artificial Intelligence and Statistics, 2023

2022

A Generalist Agent.
Trans. Mach. Learn. Res., 2022

From motor control to team play in simulated humanoid football.
Sci. Robotics, 2022

Behavior Priors for Efficient Reinforcement Learning.
J. Mach. Learn. Res., 2022

Coordinating Policies Among Multiple Agents via an Intelligent Communication Channel.
CoRR, 2022

Revisiting Gaussian mixture critics in off-policy reinforcement learning: a sample-based approach.
CoRR, 2022

Offline Distillation for Robot Lifelong Learning with Imbalanced Experience.
CoRR, 2022

Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors.
CoRR, 2022

Retrieval-Augmented Reinforcement Learning.
CoRR, 2022

Data augmentation for efficient learning from parametric experts.
Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner.
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022

Offline Meta-Reinforcement Learning for Industrial Insertion.
Proceedings of the 2022 International Conference on Robotics and Automation, 2022

Simplex Neural Population Learning: Any-Mixture Bayes-Optimality in Symmetric Zero-sum Games.
Proceedings of the International Conference on Machine Learning, 2022


Learning transferable motor skills with hierarchical latent mixture policies.
Proceedings of the Tenth International Conference on Learning Representations, 2022

NeuPL: Neural Population Learning.
Proceedings of the Tenth International Conference on Learning Representations, 2022

Evaluating Model-Based Planning and Planner Amortization for Continuous Control.
Proceedings of the Tenth International Conference on Learning Representations, 2022

COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimation.
Proceedings of the Tenth International Conference on Learning Representations, 2022

Forgetting and Imbalance in Robot Lifelong Learning with Off-policy Data.
Proceedings of the Conference on Lifelong Learning Agents, 2022

MO2: Model-Based Offline Options.
Proceedings of the Conference on Lifelong Learning Agents, 2022

2021
Game Plan: What AI can do for Football, and What Football can do for AI.
J. Artif. Intell. Res., 2021

Learning Dynamics Models for Model Predictive Agents.
CoRR, 2021

Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration.
CoRR, 2021

On Multi-objective Policy Optimization as a Tool for Reinforcement Learning.
CoRR, 2021

Entropic Desired Dynamics for Intrinsic Control.
Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Neural Production Systems.
Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Data-efficient Hindsight Off-policy Option Learning.
Proceedings of the 38th International Conference on Machine Learning, 2021

Counterfactual Credit Assignment in Model-Free Reinforcement Learning.
Proceedings of the 38th International Conference on Machine Learning, 2021

Collect & Infer - a fresh look at data-efficient Reinforcement Learning.
Proceedings of the Conference on Robot Learning, 8-11 November 2021, London, UK., 2021

A Constrained Multi-Objective Reinforcement Learning Framework.
Proceedings of the Conference on Robot Learning, 8-11 November 2021, London, UK., 2021

Towards Real Robot Learning in the Wild: A Case Study in Bipedal Locomotion.
Proceedings of the Conference on Robot Learning, 8-11 November 2021, London, UK., 2021

2020
Catch & Carry: reusable neural controllers for vision-guided whole-body tasks.
ACM Trans. Graph., 2020

dm_control: Software and tasks for continuous control.
Softw. Impacts, 2020

Counterfactual Credit Assignment in Model-Free Reinforcement Learning.
CoRR, 2020

Robust Constrained Reinforcement Learning for Continuous Control with Model Misspecification.
CoRR, 2020

Learning Dexterous Manipulation from Suboptimal Experts.
CoRR, 2020

Local Search for Policy Iteration in Continuous Control.
CoRR, 2020

Temporal Difference Uncertainties as a Signal for Exploration.
CoRR, 2020

Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban.
CoRR, 2020

Learning to swim in potential flow.
CoRR, 2020

Physically Embedded Planning Problems: New Challenges for Reinforcement Learning.
CoRR, 2020

Importance Weighted Policy Learning and Adaption.
CoRR, 2020

Action and Perception as Divergence Minimization.
CoRR, 2020

RL Unplugged: Benchmarks for Offline Reinforcement Learning.
CoRR, 2020

Simple Sensor Intentions for Exploration.
CoRR, 2020

Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning.
CoRR, 2020

Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning.
CoRR, 2020

Compositional Transfer in Hierarchical Reinforcement Learning.
Proceedings of the Robotics: Science and Systems XVI, 2020

Direct Policy Gradients: Direct Optimization of Policies in Discrete Action Spaces.
Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning.
Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Value-driven Hindsight Modelling.
Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Critic Regularized Regression.
Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Stabilizing Transformers for Reinforcement Learning.
Proceedings of the 37th International Conference on Machine Learning, 2020

CoMic: Complementary Task Learning & Mimicry for Reusable Skills.
Proceedings of the 37th International Conference on Machine Learning, 2020

A distributional view on multi-objective policy optimization.
Proceedings of the 37th International Conference on Machine Learning, 2020

V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Control.
Proceedings of the 8th International Conference on Learning Representations, 2020

Keep Doing What Worked: Behavior Modelling Priors for Offline Reinforcement Learning.
Proceedings of the 8th International Conference on Learning Representations, 2020

A Generalized Training Approach for Multiagent Learning.
Proceedings of the 8th International Conference on Learning Representations, 2020

Learning Dexterous Manipulation from Suboptimal Experts.
Proceedings of the 4th Conference on Robot Learning, 2020

Towards General and Autonomous Learning of Core Skills: A Case Study in Locomotion.
Proceedings of the 4th Conference on Robot Learning, 2020

Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions.
Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, 2020

2019
Reusable neural skill embeddings for vision-guided whole body movement and object manipulation.
CoRR, 2019

Quinoa: a Q-function You Infer Normalized Over Actions.
CoRR, 2019

Imagined Value Gradients: Model-Based Policy Optimization with Transferable Latent Dynamics Models.
CoRR, 2019

Regularized Hierarchical Policies for Compositional Transfer in Robotics.
CoRR, 2019

Meta reinforcement learning as task inference.
CoRR, 2019

Meta-learning of Sequential Strategies.
CoRR, 2019

Exploiting Hierarchy for Learning and Transfer in KL-regularized RL.
CoRR, 2019

Value constrained model-free continuous control.
CoRR, 2019

Self-supervised Learning of Image Embedding for Continuous Control.
CoRR, 2019

Hindsight Credit Assignment.
Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Reinforcement Learning Agents acquire Flocking and Symbiotic Behaviour in Simulated Ecosystems.
Proceedings of the 2019 Conference on Artificial Life, 2019

Composing Entropic Policies using Divergence Correction.
Proceedings of the 36th International Conference on Machine Learning, 2019

Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures.
Proceedings of the 7th International Conference on Learning Representations, 2019

Neural Probabilistic Motor Primitives for Humanoid Control.
Proceedings of the 7th International Conference on Learning Representations, 2019

Hierarchical Visuomotor Control of Humanoids.
Proceedings of the 7th International Conference on Learning Representations, 2019

Emergent Coordination Through Competition.
Proceedings of the 7th International Conference on Learning Representations, 2019

Information asymmetry in KL-regularized RL.
Proceedings of the 7th International Conference on Learning Representations, 2019

Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search.
Proceedings of the 7th International Conference on Learning Representations, 2019

Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics.
Proceedings of the 3rd Annual Conference on Robot Learning, 2019

Imagined Value Gradients: Model-Based Policy Optimization with Tranferable Latent Dynamics Models.
Proceedings of the 3rd Annual Conference on Robot Learning, 2019

Observational Learning by Reinforcement Learning.
Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 2019

The Body is Not a Given: Joint Agent Policy Learning and Morphology Evolution.
Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 2019

Credit Assignment Techniques in Stochastic Computation Graphs.
Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

The Termination Critic.
Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2019

2018
Relative Entropy Regularized Policy Iteration.
CoRR, 2018

Entropic Policy Composition with Generalized Policy Improvement and Divergence Correction.
CoRR, 2018

Relational inductive biases, deep learning, and graph networks.
CoRR, 2018

Reinforcement and Imitation Learning for Diverse Visuomotor Skills.
Proceedings of the Robotics: Science and Systems XIV, 2018

Graph Networks as Learnable Physics Engines for Inference and Control.
Proceedings of the 35th International Conference on Machine Learning, 2018

Learning by Playing Solving Sparse Reward Tasks from Scratch.
Proceedings of the 35th International Conference on Machine Learning, 2018

Mix & Match Agent Curricula for Reinforcement Learning.
Proceedings of the 35th International Conference on Machine Learning, 2018

Learning an Embedding Space for Transferable Robot Skills.
Proceedings of the 6th International Conference on Learning Representations, 2018

Distributed Distributional Deterministic Policy Gradients.
Proceedings of the 6th International Conference on Learning Representations, 2018

Maximum a Posteriori Policy Optimisation.
Proceedings of the 6th International Conference on Learning Representations, 2018

2017
Imagination-Augmented Agents for Deep Reinforcement Learning.
CoRR, 2017

Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards.
CoRR, 2017

Data-efficient Deep Reinforcement Learning for Dexterous Manipulation.
CoRR, 2017

Learning model-based planning from scratch.
CoRR, 2017

Learning human behaviors from motion capture by adversarial imitation.
CoRR, 2017

Emergence of Locomotion Behaviours in Rich Environments.
CoRR, 2017

Distral: Robust multitask reinforcement learning.
Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Imagination-Augmented Agents for Deep Reinforcement Learning.
Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Filtering Variational Objectives.
Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Learning Hierarchical Information Flow with Recurrent Neural Modules.
Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

Robust Imitation of Diverse Behaviors.
Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 2017

FeUdal Networks for Hierarchical Reinforcement Learning.
Proceedings of the 34th International Conference on Machine Learning, 2017

Particle Value Functions.
Proceedings of the 5th International Conference on Learning Representations, 2017

Metacontrol for Adaptive Imagination-Based Optimization.
Proceedings of the 5th International Conference on Learning Representations, 2017

Sample Efficient Actor-Critic with Experience Replay.
Proceedings of the 5th International Conference on Learning Representations, 2017

Sim-to-Real Robot Learning from Pixels with Progressive Nets.
Proceedings of the 1st Annual Conference on Robot Learning, CoRL 2017, Mountain View, 2017

2016
Continuous control with deep reinforcement learning.
Proceedings of the 4th International Conference on Learning Representations, 2016

Learning and Transfer of Modulated Locomotor Controllers.
CoRR, 2016

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models.
CoRR, 2016

Unsupervised Learning of 3D Structure from Images.
Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models.
Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, 2016

2015
Passing Expectation Propagation Messages with Kernel Methods.
CoRR, 2015

Memory-based control with recurrent neural networks.
CoRR, 2015

Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages.
Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, 2015

Gradient Estimation Using Stochastic Computation Graphs.
Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

Learning Continuous Control Policies by Stochastic Value Gradients.
Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

2014
The Shape Boltzmann Machine: A Strong Model of Object Shape.
Int. J. Comput. Vis., 2014

Recurrent Models of Visual Attention.
Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 2014

Bayes-Adaptive Simulation-based Search with Value Function Approximation.
Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, 2014

Deterministic Policy Gradient Algorithms.
Proceedings of the 31th International Conference on Machine Learning, 2014

Visual Boundary Prediction: A Deep Neural Prediction Network and Quality Dissection.
Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, 2014

2013
Learning to Pass Expectation Propagation Messages.
Proceedings of the Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013

2012
Learning generative models of mid-level structure in natural images.
PhD thesis, 2012

Searching for objects driven by context.
Proceedings of the Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3-6, 2012

Actor-Critic Reinforcement Learning with Energy-Based Policies.
Proceedings of the Tenth European Workshop on Reinforcement Learning, 2012

The Shape Boltzmann Machine: A strong model of object shape.
Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012

2011
Learning a Generative Model of Images by Factoring Appearance and Shape.
Neural Comput., 2011

Multimodal Nonlinear Filtering Using Gauss-Hermite Quadrature.
Proceedings of the Machine Learning and Knowledge Discovery in Databases, 2011

Weakly Supervised Learning of Foreground-Background Segmentation Using Masked RBMs.
Proceedings of the Artificial Neural Networks and Machine Learning - ICANN 2011, 2011

2009
Learning Generative Texture Models with extended Fields-of-Experts.
Proceedings of the British Machine Vision Conference, 2009


  Loading...