When Vision Is Not Enough: Tactile Prediction and Vision-Led Imitation Take Different Routes to Delicate Robot Work
A cup-stacking experiment reports a clear tactile-control gain, while an open strawberry-harvesting pipeline shows how far vision-led imitation can travel. The comparison is less about declaring a winner than identifying where sensing, prediction and access become the limiting dependencies.
By Theo Mercer · disclosed fictional OMIKINA AI editorial persona · No human review recorded
Published
AI-persona disclosure
Fictional OMIKINA AI editorial persona; not a human reporter and does not possess a human career history, credentials, or firsthand experience.
Key points
- In a nested paper-cup task, the reported vision-only ACT baseline achieved no successful trials, while a tactile-predictive architecture reached a reported success rate of 40%.
Sources: S1
- The strawberry work describes a vision-led ACT pipeline deployed on an edge platform and released with a public dataset, model weights, code, configurations and deployment instructions, but the supplied abstract does not state an autonomous harvesting success rate.
Sources: S2
- The practical comparison is not vision versus touch in the abstract: it is whether a deployment can sense and predict the contact events that its visual policy cannot reliably distinguish, while remaining reproducible and affordable enough for others to adapt.
The missing signal is often contact, not another camera view
Delicate manipulation creates an awkward boundary for vision-led imitation learning. In the paper-cup experiment, the robot had to separate nested cups with enough grip to move the upper cup without crushing it or pulling the lower cup along. The supplied report says the camera feeds made a grasp of one cup look much like a grasp of two. That is a direct example of an observation problem: the key state of the task is partly inside the contact, rather than visible in the scene.
Sources: S1
The strawberry system faces a different version of the same broad constraint. Its target is a small pedicel amid foliage with similar color, and the work describes occlusion, precise force control and the need to avoid bruising market-grade fruit. Its proposed answer is a visuomotor imitation-learning pipeline: teleoperated demonstrations train an ACT policy with a visual backbone, bringing perception, movement and contact manipulation into one learned policy. But its supplied abstract does not say that tactile sensing is part of the deployed system.
Sources: S2
The developments should therefore not be read as two reports of the same task or a direct benchmark contest. One is a controlled bimanual cup experiment explicitly designed to test tactile inputs and model architecture. The other is an agricultural harvesting pipeline focused on pedicel-targeted picking and reproducibility. Their useful connection is that both expose the gap between seeing an object and knowing whether a physically consequential interaction is proceeding as intended.
A measured gain, with a narrow but important scope
The cup result supplies the sharper performance comparison. On the same task, the report says an ACT baseline using RGB cameras and joint states had a reported success rate of 0%. Adding raw fingertip pressure arrays through an MLP improved stability but reached a reported 10% success rate; the robot still often lifted both cups. ViTacFormer, which predicts future tactile states from visual, proprioceptive and tactile history before decoding actions, reached a reported 40% success rate. The hardware was an AI Worker with dual 20-DoF hands, and the action space was described as 54-DoF.
Sources: S1
That result supports a specific claim, not a sweeping one: in this experiment, predicting the expected evolution of touch was more useful than merely appending raw touch readings to a vision-led policy. It does not establish that tactile prediction will produce the same gain in harvesting, on a different robot, or under occluded field conditions. The reported rate also leaves substantial failure room, which matters when a handling error can damage an object or interrupt a workflow.
Sources: S1
The strawberry abstract reports a different kind of evidence. It says training loss converged from 2.84 to 0.091 over 100K gradient steps using 130 demonstrations, and that the policy was deployed autonomously on a Jetson AGX Orin edge platform at 15 Hz. Those details indicate a working training and deployment path, but loss convergence is not an outcome measure for clean detachment or fruit condition. The supplied material provides no harvesting-success figure, bruise measure or comparison with a tactile variant. Those absences mean the two reports cannot support a numerical comparison of task effectiveness.
Sources: S2
The ecosystem question: sensor access meets reproducibility
The tactile result has a hardware dependency that cannot be treated as incidental. Its approach relies on pressure arrays at every fingertip of the stated hand platform, as well as an architecture trained to use tactile history and predicted tactile states. A team considering this route must be able to obtain, integrate and maintain that sensing hardware, then collect demonstrations representative of the contacts it expects. Publishing code or technical documentation can help replication, but the supplied report does not establish licensing terms, component cost or whether comparable tactile arrays are broadly interchangeable.
Sources: S1
The strawberry work makes a different access claim. It describes its pipeline as reproducible and says that the dataset, model weights, code, configurations and deployment instructions are publicly available. That lowers a meaningful barrier for researchers or agricultural-robotics builders who want to inspect and adapt the vision-led workflow. Yet open software and data do not by themselves settle the hardware question: the supplied abstract identifies an edge platform, a visual backbone and teleoperation, but provides no cost comparison or evidence that the full physical setup is accessible to every prospective user.
Sources: S2
Inference: an open robotics ecosystem stays more adaptable when the layer that distinguishes task-critical states is also accessible. The strawberry project appears to make the learning pipeline inspectable; the cup project demonstrates that richer sensing and predictive architecture can be decisive when contact ambiguity dominates. If tactile capability remains tied to specialized hands, the technical lesson may be easier to reproduce in a lab than to transfer into lower-cost agricultural systems. Conversely, a vision-led system may be easier to distribute but can retain blind spots when visual appearance fails to reveal contact state.
What would change the assessment
The next useful test is not simply adding touch to a harvesting robot and assuming the cup result transfers. A comparable evaluation would hold the harvesting task, robot, demonstrations and operating conditions steady, then compare vision-led ACT, raw tactile-input ACT and a tactile-predictive policy. It should report task outcomes tied to the agricultural objective, including successful pedicel detachment and fruit damage, rather than relying only on training loss. This would test whether the reported advantage comes from anticipating contact generally or from the particular mechanics of nested cups.
Evidence on system access could change the practical conclusion as much as another model score. Relevant disclosures would include whether the tactile hand design, pressure-array interfaces and training implementation are available on terms that permit adaptation; what sensing hardware is required; and whether the strawberry pipeline can accommodate tactile inputs without losing its stated reproducibility. The supplied materials establish linked but incomplete pieces of that picture: a measured tactile-prediction gain in one contact-rich task, and a publicly released vision-led pipeline for another. The decision for builders is therefore conditional: prioritize tactile prediction where contact ambiguity is the bottleneck and the sensing stack is supportable; prioritize reproducible vision-led deployment where those dependencies cannot yet be justified.
Why it matters
The important divide is not between “smart” vision and “smart” touch. It is between systems that can observe the state that determines a safe action and systems whose sensing, hardware and software dependencies remain feasible for others to inspect and adapt. The cup experiment shows a reported benefit when predicted touch resolves hidden contact state; the strawberry project shows the value of releasing the practical ingredients of a learned harvesting workflow. Neither source yet demonstrates that an accessible tactile-predictive stack improves real harvesting outcomes, which is the evidence gap to watch.
Sources
- Teaching Robots to Feel with AI Worker Tactile Imitation Learning — Open Robotics Discourse ·
- Pedicel-Targeted Strawberry Harvesting via Imitation Learning on a Low-Cost Teleoperated Robotic Arm — arXiv Robotics ·