Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

重构Unity Wrapper #25

Open
StepNeverStop opened this issue Dec 28, 2020 · 0 comments
Open

重构Unity Wrapper #25

StepNeverStop opened this issue Dec 28, 2020 · 0 comments
Assignees
@StepNeverStop
Labels
enhancement New feature or request optimization Better performance or solution

Comments

@StepNeverStop
Copy link
Owner

  1. 由python端在连接Unity时自动发送诸如“环境并行数量、智能体雷达检测密度、是否强制reset环境”等变量
  2. 由python端在初始化训练环境时指定是否需要stack状态输入，无需另写StackWapper
  3. ...
@StepNeverStop StepNeverStop self-assigned this Dec 28, 2020
@StepNeverStop StepNeverStop added the enhancement New feature or request label Dec 28, 2020
StepNeverStop added a commit that referenced this issue Dec 31, 2020
@StepNeverStop
refactor(unity): optimized unity training (#25)
080f381 
- add `initialize_config` in config.yaml
- add `reset_config` and `step_config` for unity training
- remove `GrayVisualWrapper`. `ResizeVisualWrapper`. `StackVisualWrapper`
StepNeverStop added a commit that referenced this issue Dec 31, 2020
@StepNeverStop
refactor(unity): using env_copys in yaml instead of --n_agents in…
efa6f65 
… command line when using unity training agents (#25)
StepNeverStop added a commit that referenced this issue Dec 31, 2020
@StepNeverStop
setting(unity): change default value of env_copys (#25)
b92ed95
StepNeverStop added a commit that referenced this issue Jan 1, 2021
@StepNeverStop
refactor(unity): optimize unity wrapper (#25)
747b3fa 
- add `UnitySingleBehaviorInfo` in indexs.py
- remove BasicActionWrapper
- remove redundant identifiers
StepNeverStop added a commit that referenced this issue Jan 5, 2021
@StepNeverStop
refactor(data): TAG 2.0.0 (#25, #33, open #31)
dc5391d 
- support multi-vector and multi-visual input
- optimize `gym` and `unity` wrapper
- fix `ActorCriticValueCts`
- tag 2.0.0
- add `ObsSpec`
- refactor `SingleAgentEnvArgs` and `MultiAgentEnvArgs`
- remove `self.s_dim`, use `self.concat_vector_dim` instead
- stop using vector input normalization temporarily
@StepNeverStop StepNeverStop added the optimization Better performance or solution label Jan 6, 2021
StepNeverStop added a commit that referenced this issue Jan 6, 2021
@StepNeverStop
style: remove redundant code (#25)
fb9e5eb
StepNeverStop added a commit that referenced this issue Jan 6, 2021
@StepNeverStop
fix&refactor(unity): optimize unity wrapper, fix ddpg (#25, #34)
7dad01d
StepNeverStop added a commit that referenced this issue Jan 6, 2021
@StepNeverStop
refactor(unity): optimize unity wrapper (#25)
719ccf4
StepNeverStop added a commit that referenced this issue Jul 2, 2021
@StepNeverStop
fix(multi-agents): implemented n-step replay buffer for multi-agents …
dbc6136 
…training. (#41,#25,#31)

1. change variable name from "is_lg_batch_size" to "can_sample"
2. optimized unity wrapper
3. optimized multi-agents replay buffers
StepNeverStop added a commit that referenced this issue Jul 4, 2021
@StepNeverStop
fix&feat: optimized training framework. (#41, #34, #33, #31, #25)
0a8d247 
1. fixed n-step replay buffer
2. reconstruct representation net
3. remove 'use_stack'
4. implement multi-agent algorithms with shared parameters
5. optimized agent network
StepNeverStop added a commit that referenced this issue Jul 28, 2021
@StepNeverStop
perf: optimized code-style. (#45, #41, #39, #35, #25)
a3c989a 
1. removed sarl off-policy algorithm pd_ddpg, 'cause it's not in main stream
2. updated README
3. removed `iql` and added script `IndependentMA.py` instead to implement independent multi-agent algorithms
4. optimized summary writing
5. move NamedDict from 'rls.common.config' to 'rls.common.specs'
6. updated example config
7. updated `.gitignore`
8. added property `is_multi` to identify whether training task is for sarl or marl for both unity and gym
9. reconstructed inheritance relationships between algorithms and their's superclass
10. removed `1.e+18` in yaml files and use a large integer number instead, 'cause we want a large integer rather than float
StepNeverStop added a commit that referenced this issue Jul 29, 2021
@StepNeverStop
v4.1.0 perf: implemented unified env interface.(#34, #31, #26, #25, #40)
941e14d 
1. added `test.yaml` for quickly verify RLs
2. change folder name from `algos` to `algorithms` for better reading
3. removed single agent recoder, all algorithms(sarl&marl) using  `SimpleMovingAverageRecoder`
4. removed `GymVectorizedType` in `common/specs.py`
5. removed `common/train/*`, and implement unified training interface in `rls/train`
6. reconstructed `make_env` function in `rls/envs/make_env`
7. optimized function `load_config`
8. moved `off_policy_buffer.yaml` to `rls/configs/buffer`
9. removed configurations like `eval_while_train`, `add_noise2buffer` etc.
10. optimized environments' configuration files
11. optimized environment wrappers and implemented unified env interface for `gym` and `unity`, see `env_base.py`
12. updated dockerfiles
13. updated README
StepNeverStop added a commit that referenced this issue Jul 29, 2021
@StepNeverStop
v4.1.2 fix(unity): fixed bugs in continuous training and unity traini…
f95a510 
…ng. (#34, #25)

1. updated `setup.py`
2. removed redundant packages
3. fixed bugs in unity wrapper
4. fixed bugs in agent models that occurred in continuous-action training tasks
5. fixed bugs in class `MLP`
StepNeverStop added a commit that referenced this issue Aug 25, 2021
@StepNeverStop
perf: reconstruct repo(#47, #25, #46, #34, #31, #33, #39, #41, #45, #26)
67b8979
StepNeverStop added a commit that referenced this issue Aug 27, 2021
@StepNeverStop
fix(unity): fixed visual input training using Unity3D (#34, #25)
39d60c3
StepNeverStop added a commit that referenced this issue Sep 4, 2021
@StepNeverStop
v0.0.1 perf(env): optimized env wrapper and data storing. (#31,#34,#25)
7d5e00f 
*. redefine version to v0.0.1
1. removed package `supersuit`
2. implemented class `MPIEnv`
3. implemented class `VECEnv`
4. optimized env wrappers, implemented `render` method to `gyms` environment.
5. reconstructed some of returns of `env.step` from `obs` to `obs_fa` and `obs_fs`.
  - `obs_fa` is used to choose action based by agent/policy. For the cross point of episode i and i+1, `obs_fa` represents $observation_{i+1}^{0}$, otherwise it keeps same with `obs_fs` which represents $observation_{i}^{t}$.
  - `obs_fs` is used to be stored in buffer. For the cross point of episode i and i+1, `obs_fs` represents $observation_{i}^{T}$, otherwise it keeps same with `obs_fa`.
6. optimzed `rssm` related based on mentioned `obs_fs`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@StepNeverStop StepNeverStop

Labels
enhancement New feature or request optimization Better performance or solution
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@StepNeverStop