학술대회

강연자

이기민 (UC Berkeley 박사)

제목

PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training

학회

2021, ICML (International Conference on Machine Learning), Oral (3% Acceptance Rate)

요약

Conveying complex objectives to reinforcement learning (RL) agents can often be difficult, involving meticulous design of reward functions that are sufficiently informative yet easy enough to provide. Human-in-the-loop RL methods allow practitioners to instead interactively teach agents through tailored feedback; however, such approaches have been challenging to scale since human feedback is very expensive. In this work, we aim to make this process more sample- and feedback-efficient. We present an off-policy, interactive RL algorithm that capitalizes on the strengths of both feedback and off-policy learning. Specifically, we learn a reward model by actively querying a teacher's preferences between two clips of behavior and use it to train an agent. To enable off-policy learning, we relabel all the agent's past experience when its reward model changes. We additionally show that pre-training our agents with unsupervised exploration substantially increases the mileage of its queries. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods, including a variety of locomotion and robotic manipulation skills. We also show that our method is able to utilize real-time human feedback to effectively prevent reward exploitation and learn new behaviors that are difficult to specify with standard reward functions.

강연자

홍승우 (KAIST 박사과정)

제목

Real-time constrained nonlinear model predictive control on SO(3) for dynamic legged locomotion

학회

2020, IROS (International Conference on Intelligent Robots and Systems), Best RoboCup Paper Award

요약

강연자

이정민 (서울대학교 박사과정)

제목

단위시스템 기반 병렬적이고 모듈화된 동역학 시뮬레이션 (A Parallelized Iterative Algorithm for Real-Time Simulation of Long Flexible Cable Manipulation)

학회

2021 ICRA (International Conference on Robotics and Automation), Best Manipulation Paper Award Finalist

요약

AI 및 가상현실 기술 등의 발달로 동역학 시뮬레이션에 대한 관심이 높아지고 있습니다. 특히 로보틱스 분야의 시뮬레이션은 다중접촉, 멀티바디(유연체, 강체 등)를 포함하면서, 속도와 정확도를 동시에 요구하기 때문에 아직도 상당히 어려운 문제입니다. 본 연구에서는 단위시스템 기반, 병렬적이고 모듈화된 동역학 시뮬레이션 프레임워크를 제시합니다. 이는 최근 발전하고 있는 병렬 컴퓨팅에 적합할 뿐만 아니라, 특정 시스템을 위해 개발된 여러 알고리즘들을 통합할 수 있도록 해줍니다. 여기서 가장 핵심적인 부분은 단위시스템 간 커플링 부분을 안정적이고 효율적으로 푸는 것인데, 이를 어떻게 해결하는지에 대한 방법론이 제시됩니다.