You might also like
Dr. Ziwei Wang is an Assistant Professor in the School of Electrical and Electronic Engineering, at Nanyang Technological University (NTU) in Singapore, where his lab focuses on robot manipulation and learning.
Research goal: improving data collection efficiency
Dr Wang's lab collaborates closely with industrial companies in Singapore, in particular in the manufacturing sector, to address competitiveness and labor shortage issues, and participates in the Republic’s effort to reinforce its global leadership in embodied AI. Practical applications include building robotic systems capable of autonomously handling the precise, physical demands of real tasks in the electronics industry, like sorting electronics components such as GPUs, CPUs and RAMs; performing screwing operations on PCBs; and inserting components into carriers with millimetre-level accuracy.
One area of research is to improve the data collection necessary to train autonomous robotics systems. Dr Wang's Research Fellow Haichao Liu aimed at increasing the performance of teleoperation systems for dexterous manipulation tasks. His approach is to develop DexTeleop-0, a force-balance controller (or "small brain" for simplicity) added to the teleoperation system. It consists of a tactile-driven adaptive controller that corrects a rough teleoperation mapping to millimetre-level accuracy in real time, because perfecting mapping of human hands is impossible due to the localization errors and lack of feedback.
The "small brain" was deployed on two different teleoperation set-ups to perform several different tasks: (1) precise insertion of a component into a carrier performed by a gripper on a robotic arm; (2) screwing components to a gearbox, assembling a smooth ball to a tube, performed by two Sharpa Wave hands on two UR7e arms, managing 56 DoF. The gripper system was teleoperated using GELLO; while the Sharpa Wave was teleoperated either using a Manus glove or a Meta Quest 3 VR headset. For more details on the research, the paper can be found on Arxiv.
Results: Tactile-enabled "small brain" increases the success rate of insertion during teleoperation
The most concrete achievement of adding a "small brain" to the teleoperation set-up was an improvement of the success-rate on precise insertion tasks: from roughly 10% to over 90% when tactile feedback is incorporated. For contact-rich tasks, vision becomes occluded precisely during the critical approach and insertion phase. Tactile signals overcome this problem. They are also less prone to overfitting: the sim-to-real gap is substantially smaller for tactile-based policies than for vision-based ones. This is a strong argument in favor of using end effectors with tactile sensors, that can support tactile manipulation policies.
“With tactile feedback, the success rate on insertion tasks improved from 10% to over 90%. That's a massive difference which makes large-scale collection of precise dexterous hand teleoperation data feasible.”
Dr. Ziwei Wang, Assistant Professor, NTU
Why the Sharpa Wave was considered
Low DoF hands and grippers fall short on tasks like screwing and meshing
Dr. Wang's lab had previously worked with lower-DoF hands — including 6-DoF platforms — and found them fundamentally limited for the tasks they care about. For screwing and meshing specifically, a 6-DoF hand failed on two counts: insufficient positional precision and too few degrees of freedom to perform the required wrist-finger coupling motion. He tried one other hand model and found it too large for the confined workspaces required by gearbox assembly tasks. Grippers were sufficient for sorting, but not for screwing, scooping, or fine insertion.
"For some tasks, a dexterous hand is simply necessary. Screwing with a gripper is almost impossible: you need to reposition every single time, and cycle time can't meet factory requirements. For most sorting tasks, grippers are good enough. But for complicated tasks — screwing, scooping, locking and unlocking, slicing in household scenarios — you definitely need a dexterous hand".
Dr. Ziwei Wang, Assistant Professor, NTU
Precision, tactile sensing and ecosystem were the key decision points
Sharpa came to the lab's attention through conference demonstrations and word of mouth within Singapore's robotics community — Sharpa's presence as a Singapore-headquartered company, and introductions through NRP colleagues, created multiple discovery touchpoints.
The decision to adopt the Wave was driven by three explicit criteria:
-
Precision: millimetre-level teleoperation accuracy
-
Ecosystem: URDF files, deployment toolbox, teleoperation integration, and the accumulated experience built through months of direct collaboration with the Sharpa team
-
Tactile sensing: built-in fingertip tactile feedback enabling contact-rich policies.
“From the perspective of precision, ecosystem, and tactile sensing — Sharpa is, I would say, the best product so far.”
Dr. Ziwei Wang, Assistant Professor, NTU
The Wave's high DoF, precision and tactile sensing increases the success rate of accurate assembly
The 22 DoF of the Sharpa Wave and specific joint configuration enables the wrist-finger coupling motion required for screwing and meshing, which is categorically impossible below a certain DoF threshold. Fingertip size also mattered: the hand had to be small enough to operate in confined gearbox workspaces.
Because the Sharpa Wave can reliably perform the above dexterous manipulation tasks, Haichao Liu could isolate in his research the impact of the teleoperation set-up, and in particular the “small brain”, in the success rate of the task.
"For specific tasks, the extra joints are critical. It really depends on which joint it is and what motion it enables. For our tasks, the lateral mobility of the Sharpa Wave, particularly the abduction/adduction DoF at the metacarpophalangeal (MCP) joints, matters a lot."
Dr. Ziwei Wang, Assistant Professor, NTU
Beyond the lab: real-world use cases where the Sharpa Wave brings value
In the short term, Dr. Wang identifies screwing and precise insertion, for instance, in PCB manufacturing, as a good use case requiring a high DoF, and a precision that grippers cannot provide up to the speed requirements of the Electronics & Semiconductor industry. In general, he thinks it is already possible to deploy sensible use cases in manufacturing because it does not require too much generalization but rather high precision, robustness and efficiency.
In the longer term, as precision and robustness improve, more complex and high-value industry tasks, especially in assembly, will be unlocked. In addition, increasing generalization ability of AI models will open up many tasks relevant to the household and service industry (retail, hospitality, restaurants, etc.).
