Learning Keypoints from Synthetic Data for Robotic Cloth Folding

Robotic manipulation of deformable objects is a difficult activity. However, responsibilities like cloth manipulation are handy in daily options. A latest paper released on arXiv.org proposes a novel solution for the towel folding task.

Graphic credit score: Piqsels, CC0 General public Domain

Scientists advise employing a convolutional neural network (CNN) as a keypoint detector to estimate the 2D positions of the towel corners from a solitary RGB picture. Then, a scripted, open up-loop grasp and a quasi-static fold trajectory are executed based on these semantic keypoints. The keypoint detector is educated solely on artificial information.

It is revealed that artificial knowledge is suited for the detection of keypoints for cloth folding. The evaluation of the overall performance of the system reveals that the grasp achievements amount is about 77%, and the fold good results charge is about 53% therefore, a lot more in depth tuning is demanded to completely overcome the reality gap.

Robotic fabric manipulation is challenging thanks to its deformability, which can make identifying its whole point out infeasible. However, for fabric folding, it suffices to know the position of a couple semantic keypoints. Convolutional neural networks (CNN) can be employed to detect these keypoints, but require large amounts of annotated information, which is high priced to gather. To get over this, we propose to learn these keypoint detectors purely from synthetic info, enabling low-charge data assortment. In this paper, we procedurally make illustrations or photos of towels and use them to prepare a CNN. We examine the effectiveness of this detector for folding towels on a unimanual robotic set up and find that the grasp and fold achievement prices are 77% and 53%, respectively. We conclude that mastering keypoint detectors from artificial details for fabric folding and connected tasks is a promising investigation direction, examine some failures and relate them to future work. A video clip of the procedure, as nicely as the codebase, additional information on the CNN architecture and the training set up can be observed at this https URL.

Investigation post: Lips, T., De Gusseme, V.-L., and wyffels, F., “Learning Keypoints from Artificial Info for Robotic Fabric Folding”, 2022. Website link: https://arxiv.org/abs/2205.06714

You can provide your link to a webpage which is suitable to the subject of this post.