Description
The maml_rl repository hosts the codebase accompanying the influential paper "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks" by Finn et al. (ICML 2017). This project focuses specifically on implementing and running reinforcement learning (RL) experiments that demonstrate the efficacy of Model-Agnostic Meta-Learning (MAML) for fast adaptation in deep neural networks.
Read More
MAML is a meta-learning algorithm designed to train models that can quickly adapt to new tasks with only a few gradient steps. This codebase allows users to replicate the RL experiments presented in the paper, providing a practical tool for understanding and applying MAML in various RL scenarios. The repository is built upon the rllab code framework, a robust platform for developing and evaluating RL algorithms. Users will need to install TensorFlow v1.0+ as a dependency, as the MAML code specifically utilizes the TensorFlow version of rllab.
The structure of the repository includes directories for examples, documentation, and core MAML components. Scripts for running the experiments are located within the `maml_examples/` directory. The project supports both pointmass and MuJoCo environments, with the latter being accessible through `rllab/envs/mujoco/`. A key aspect highlighted by the developers is the current slowness of the code, with an open invitation for contributions to improve performance, particularly through enhanced parallelization of sampling and meta-learning graph computation.
For those interested in the supervised learning aspects of MAML, a separate codebase is referenced. The project emphasizes its foundation in rllab, which offers a wide array of continuous control tasks and implementations of various RL algorithms such as REINFORCE, Truncated Natural Policy Gradient, and Deep Deterministic Policy Gradient. rllab is compatible with OpenAI Gym and officially supports Python 3.5+. The maml_rl repository serves as a valuable resource for researchers and practitioners in the fields of meta-learning, reinforcement learning, and deep learning, enabling them to experiment with and extend state-of-the-art adaptation techniques.
- CategoryAI Models & LLMs
- Pricing ModelOpen source
- DeploymentCloud, Self-hosted
- PlatformsLinux, macOS, Windows
maml_rl's Core Features
Codebase for Model-Agnostic Meta-Learning (MAML) in RL experiments
Implements fast adaptation of deep networks for new tasks
Built upon the rllab reinforcement learning framework
Includes scripts for running experiments from the ICML 2017 paper
Supports pointmass and MuJoCo environments
Requires TensorFlow v1.0+
Facilitates research in meta-learning and reinforcement learning
Open invitation for contributions to improve code speed and parallelization
Getting Started with maml_rl
Set up environment: Install TensorFlow v1.0+ and clone the repository.
Install dependencies: Follow rllab installation guidelines.
Navigate to examples: Locate experiment scripts in `maml_examples/`.
Configure experiments: Adjust parameters as needed for specific tasks.
Run RL experiments: Execute scripts to train and test MAML agents.
Analyze results: Evaluate adaptation speed and performance on new tasks.
maml_rl's Use Cases
- Meta-learning RL agents
- Fast adaptation research
- Few-shot learning in RL
- Robotics adaptation
- Game AI development
