ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals
Imitation Learning (IL) is a promising approach for teaching tasks to robots by human demonstrations, although it faces challenges from long-horizon tasks and diverse goals in real-world settings. These issues stem from (i) a distribution mismatch between demonstrations and real-world execution and...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10811934/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846099116714622976 |
|---|---|
| author | Cynthia Ochoa Hanbit Oh Yuhwan Kwon Yukiyasu Domae Takamitsu Matsubara |
| author_facet | Cynthia Ochoa Hanbit Oh Yuhwan Kwon Yukiyasu Domae Takamitsu Matsubara |
| author_sort | Cynthia Ochoa |
| collection | DOAJ |
| description | Imitation Learning (IL) is a promising approach for teaching tasks to robots by human demonstrations, although it faces challenges from long-horizon tasks and diverse goals in real-world settings. These issues stem from (i) a distribution mismatch between demonstrations and real-world execution and (ii) existing policy models that typically focus on prelearned final goals, limiting efficiency with diverse goals. To address this situation, we propose Interactive Sub-Goal-Planning Imitation Learning (ISPIL), an IL framework that learns hierarchical, goal-conditioned policies. Specifically, a high-level policy sets reachable sub-goals for the final goals, and a low-level policy executes the required actions. ISPIL interactively collects two types of demonstration data based on the novelty criteria: meta-sub-goal data, which represent with symbols the causal relationships between sub-goals, and action data, which consist of the physical robotic actions required to achieve these sub-goals. Meta-sub-goal data enable effective planning using a Regression Planning Network (RPN), and a sub-goal switching function helps reduce unnecessary data queries at the high level. We validate ISPIL through simulations and real-robot experiments in a kitchen-like environment and demonstrate improved task execution and generalizability across diverse goals. |
| format | Article |
| id | doaj-art-c4e2f9fb0f224dc2955fde8cd4eb1816 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-c4e2f9fb0f224dc2955fde8cd4eb18162025-01-01T00:01:52ZengIEEEIEEE Access2169-35362024-01-011219761619763110.1109/ACCESS.2024.352130210811934ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse GoalsCynthia Ochoa0https://orcid.org/0009-0007-9220-2345Hanbit Oh1https://orcid.org/0000-0002-2368-3321Yuhwan Kwon2https://orcid.org/0000-0001-9058-1379Yukiyasu Domae3https://orcid.org/0000-0002-1366-9657Takamitsu Matsubara4https://orcid.org/0000-0003-3545-4814Division of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology, Ikoma, JapanIndustrial Cyber-Physical Systems Research Center, National Institute of Advanced Industrial Science and Technology (AIST), Fukushima, JapanDivision of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology, Ikoma, JapanDivision of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology, Ikoma, JapanDivision of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology, Ikoma, JapanImitation Learning (IL) is a promising approach for teaching tasks to robots by human demonstrations, although it faces challenges from long-horizon tasks and diverse goals in real-world settings. These issues stem from (i) a distribution mismatch between demonstrations and real-world execution and (ii) existing policy models that typically focus on prelearned final goals, limiting efficiency with diverse goals. To address this situation, we propose Interactive Sub-Goal-Planning Imitation Learning (ISPIL), an IL framework that learns hierarchical, goal-conditioned policies. Specifically, a high-level policy sets reachable sub-goals for the final goals, and a low-level policy executes the required actions. ISPIL interactively collects two types of demonstration data based on the novelty criteria: meta-sub-goal data, which represent with symbols the causal relationships between sub-goals, and action data, which consist of the physical robotic actions required to achieve these sub-goals. Meta-sub-goal data enable effective planning using a Regression Planning Network (RPN), and a sub-goal switching function helps reduce unnecessary data queries at the high level. We validate ISPIL through simulations and real-robot experiments in a kitchen-like environment and demonstrate improved task execution and generalizability across diverse goals.https://ieeexplore.ieee.org/document/10811934/Interactive imitation learninglearning-to-planhierarchical policy |
| spellingShingle | Cynthia Ochoa Hanbit Oh Yuhwan Kwon Yukiyasu Domae Takamitsu Matsubara ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals IEEE Access Interactive imitation learning learning-to-plan hierarchical policy |
| title | ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals |
| title_full | ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals |
| title_fullStr | ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals |
| title_full_unstemmed | ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals |
| title_short | ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals |
| title_sort | ispil interactive sub goal planning imitation learning for long horizon tasks with diverse goals |
| topic | Interactive imitation learning learning-to-plan hierarchical policy |
| url | https://ieeexplore.ieee.org/document/10811934/ |
| work_keys_str_mv | AT cynthiaochoa ispilinteractivesubgoalplanningimitationlearningforlonghorizontaskswithdiversegoals AT hanbitoh ispilinteractivesubgoalplanningimitationlearningforlonghorizontaskswithdiversegoals AT yuhwankwon ispilinteractivesubgoalplanningimitationlearningforlonghorizontaskswithdiversegoals AT yukiyasudomae ispilinteractivesubgoalplanningimitationlearningforlonghorizontaskswithdiversegoals AT takamitsumatsubara ispilinteractivesubgoalplanningimitationlearningforlonghorizontaskswithdiversegoals |