Evaluating robot bodies by their skill with limited controls
Beyond Task Reward: A Controller-Restriction Protocol for Evaluating Embodiment-Dependent Competence
Robotics
Summary
Robots are often judged by how well their body and brain work together at a task, but this single score hides how much the body depends on the brain's complexity. The paper shows a new way to test a robot body by restricting its brain to simple types and seeing how well it still performs. This method reveals differences that normal testing misses and is reliable when repeated, but it depends on which simple brain types are chosen. The authors suggest using this as an extra way to report robot abilities after standard tests.
What this means in practice
- •For robotics engineers: Choose robot designs that maintain good performance even with simple controllers by using restricted-control competence metrics after reward-based optimization.
- •For automation system designers: Evaluate hardware configurations for robots in constrained computing environments by assessing their performance under limited controller complexity.
Authors
Siyuan Zhang
Abstract
Co-design methods optimize a robot's body and controller jointly and judge the result by one number, the task reward of the fully optimized pair. That number cannot separate morphologies whose competence depends on the controller to very different degrees. We evaluate a morphology by restricting its controller instead, recording the task competence it retains under an explicitly declared, low-complexity controller family, environment, task and search budget. On three EvoGym locomotion tasks, task reward explains only $39\%$, $33\%$ and $10\%$ of the variance in this quantity, and geometric descriptors do not predict it under run-grouped cross-validation. The measurement is reliable across optimizer restarts (ICC$(2,k) = 0.956$--$0.986$) but depends on the declared family: phasing the drive by actuator index instead of position ranks the same morphologies at Spearman $0.50$--$0.63$ and reverses reward-matched pairs. As a second search objective the axis improved competence at matched task reward in $3$ of $5$ paired runs, short of a pre-registered bar of $4$. Used after an ordinary reward-only search instead, to choose within its top task-reward band, it selected a different body in all $8$ runs offering a choice, at a cost of at most $0.10$ reward units, and in $6$ of $8$ that body also scored higher under a held-out family. Restricted-control competence is therefore a reportable property of a co-designed morphology, interpretable only with the controller family that defines it.