From the state of the art of assessment metrics toward novel concepts for humanoid robot locomotion benchmarking
In order to prepare humanoid robots for real-world applications, it is necessary to assess the robots’ locomotion performance and to determine their readiness level. We have carried out an extensive statistical literature review of 204 publications evaluating what kind of benchmarking scenarios and...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article (Journal) |
| Language: | English |
| Published: |
2020
|
| In: |
IEEE Robotics and automation letters
Year: 2019, Volume: 5, Issue: 2, Pages: 914-920 |
| ISSN: | 2377-3766 |
| DOI: | 10.1109/LRA.2019.2952291 |
| Online Access: | Resolving-System: https://doi.org/10.1109/LRA.2019.2952291 Verlag: https://ieeexplore.ieee.org/document/8894403/authors 
|
| Author Notes: | Felix Aller, David Pinto-Fernandez, Diego Torricelli, Jose Luis Pons, Katja Mombaur |
| Summary: | In order to prepare humanoid robots for real-world applications, it is necessary to assess the robots’ locomotion performance and to determine their readiness level. We have carried out an extensive statistical literature review of 204 publications evaluating what kind of benchmarking scenarios and performance indicators are used and if they are sufficient to take a robot from prototype to production. We report the results of our analysis and discuss the most relevant findings such as the overall increase in the number of publications and motion tasks considered. Previous benchmarking efforts have been devoted to the functional assessment of the robotic system itself. A trend towards goal level oriented performance indicators such as robustness to external disturbance and dependability can be observed. We also identify a deficiency in benchmarking of non-functional aspects like safety or human-robot interaction. Based on our results, we outline the creation of a benchmarking framework with respect to current benchmarking approaches also introducing testbeds considering currently neglected motion tasks with a focus on a high degree of standardization. |
|---|---|
| Item Description: | Date of publication: 08 November 2019 Gesehen am 26.02.2020 |
| Physical Description: | Online Resource |
| ISSN: | 2377-3766 |
| DOI: | 10.1109/LRA.2019.2952291 |