Scope
The purpose of this workshop is to take stock of the situation regarding reproducibility and benchmarking issues in robotics research after 20 years from the first workshop on ‘Benchmarking in Robotics’, organized by Angel P. del Pobil at IROS 2006 in Beijing. Since then, tens of workshops and tutorials at major conferences, a couple of seasonal schools, have been organized; likewise, several special issues on major journals have been published. It makes sense to ask ourselves as a community, how much progress have we made? Were those activities useful? Did they had a real impact? Reproducibility in robots remains non-mainstream. This circumstance impedes both the research process and the transfer of technological findings. The reproducibility of outcomes is fundamental to the scientific process that underpins contemporary engineering. Widely shared benchmarking best practices allowing for the objective operational measurement of intelligent robot behaviors are still lacking. We will gather a number of speakers among those who have been more active on those issues in the latest 20 years. We will leave the floor open to new contributions and to novel perspectives by open calls and public discussion. In this way, we will put together the ‘established experts’ and the ‘new interested parties’ aiming at mixing up approaches and methodologies currently employed for robotic reproducibility and fresh new ideas and trends. Will reproducible research and benchmarking be mainstream 20 years from now?
