Unity(ML-agents) for Imitaion Learning.

Posted 2023-04-05 思考实践

ML-agent supports two types of learning:

1.RL:Learns by getting rewards.

2.Imitation:Learns by imitatating what the player does.

Imitation  learning is how you teach your ai directly how to behave in order to  achieve  a certain goal.

Firstly,set up the scene with some randomness,so that the ai doesn't know how to solve just one specific set of positions,so we randomize 

Two types of imitation learning that you can use [GAIL,BC]

BC - Behavior Cloning//行为克隆是最简单的模仿学习。

GAIL-Generative Adversarial Imitation Learning,the goal of the discriminator is to figure out if a certain action  came from the agent or from the demo.so essentially over time our agent will learn how to behave like the demo in order to trick the discriminator.

GAIL works by trying to trick a discriminator  into pretending  that the  actions came from the demo,whereas BC simply  tries to copy  exactly what you did ,the limitation of the BC is that it can never get better than the demos.So in order to get the best results,we need to  combine  all three.

First use BC ,it learns to act exactly like you,then when  combined with GAIL,it learns to act similarily to you while achieving the same goal,and when combined  with extrinsic rewards,it continues improving  upon those two,that's how we get superhuman learning .

Visualization in tensorboard 

 

 

Personally Speaking:

I have watched some basic project completed by ml-agents.Personally speaking,it feels like 

Reference

【Unity 教学】【中字】教你的人工智能！使用 Unity ML-Agents进行模仿学习！_哔哩哔哩_bilibili

(269条消息) 模仿学习笔记：行为克隆_UQI-LIUWJ的博客-CSDN博客

ML-Agents安装和配置

原文链接：https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md

安装和设置

要安装和使用ML-Agents，您需要安装Unity，克隆这个存储库并安装Python以及其他依赖项。除了Docker设置之外，下面的每个小节概述了每个步骤。

 

安装Unity 2017.1或更高版本

下载并安装Unity。如果您想使用我们的Docker设置（稍后介绍），请确保在安装Unity时选择Linux Build Support组件。

 

克隆ml-agents储存库

安装完成后，您需要克隆ML-Agents GitHub存储库。

git clone [e-mail]:Unity-Technologies/ml-agents.git

此存储库中的unity-environment目录包含要添加到项目中的Unity资源。 python目录包含训练代码。这两个目录都位于存储库的根目录。

 

安装Python（以及依赖项）

为了使用ML-Agents，需要Python 3以及需求文件中列出的依赖项。一些主要的依赖包括：

• TensorFlow
• Jupyter

Windows用户

如果您是一位刚接触Python和TensorFlow的Windows用户，请按照本指南设置您的Python环境。

 Mac和Unix用户

 如果您尚未拥有它，请下载并安装Python 3。

 如果您的Python环境不包含pip，请参阅有关安装它的说明。

 要安装依赖项，请进入存储库的python子目录，然后从命令行运行：

pip3 install .

 

基于Docker的安装

 如果您想使用Docker for ML-Agents，请按照本指南进行操作。

 

Unity软件包

 您可以在这里下载TensorFlowSharp插件作为Unity软件包。

 

帮帮我

 如果您在安装ML-Agents时遇到任何问题，请提交问题并确保包含操作系统，Python版本和确切错误消息的相关信息（有可能的话）。