NVIDIA teaches dexterity to a robot hand

-
NVIDIA hand

The setup for NVIDIA’s DeXtreme project using a Kuka robotic arm and an Allegro Hand. | Source: NVIDIA

Robotic hands are notoriously complex and difficult to control. The human hands they imitate consist of 27 different bones, 27 joints and 34 muscles, all working together to help us perform our daily tasks. Translating this process into robotics is more challenging than developing robots that use legs to walk, for example. 

Methods typically used to teach robot control, like traditional methods with precisely pre-programmed grasps and motions or deep reinforcement learning (RL) techniques, fall short when it comes to operating a robotic hand. 

Pre-programmed motions are too limited for the generalized tasks a robotic hand would ideally be able to perform, and deep RL techniques that train neural networks to control robot joints require millions, or billions, of real-world samples to learn from.  

NVIDIA, instead, used its Isaac Gym RL robotics simulator to train an Allegro Hand, a lightweight, anthropomorphic robotic hand with three off-the-shelf cameras attached, as part of its DeXtreme project. The Isaac simulator is able to run more than simulations 10,000 times faster than the real world, according to the company, while still obeying the laws of physics. 

With Isaac Gym, NVIDIA was able to teach the Allegro Hand to manipulate a cube and match provided target positions, orientations or poses. NVIDIA’s neural network brain learned to do all of this in simulation and then the team transplanted it to control a robot in the real world. 

Training the neural network

In addition to its end-to-end simulation environment Isaac Gym, NVIDIA used its PhysX simulator, which simulates the world on the GPU that stays in the GPU memory while the deep learning control policy network is being trained, to train the hand. 

Training in simulations provides a number of benefits for robotics. Besides NVIDIA’s ability to run simulations much faster than they would play out in the real world, robot hardware is prone to breaking after a lot of use. 

According to NVIDIA, the team working with the hand often had to stop to repair the robotic hand, things like tightening screws, replacing ribbon cables and resting the hand to let it cool, after prolonged use. This makes it difficult to get the kind of training the robot needs in the real world. 

To train the robot’s neural network, NVIDIA’s Omniverse Replicator generated around five million frames of synthetic data, meaning NVIDIA’s team didn’t have to use any real images. With NVIDIA’s training method, a neural network is trained using a technique called domain randomization, which changes lighting and camera positions to give the network more robust capabilities. 

All of the training was done on a single Omniverse OVX server, and the system can teach a good policy in about 32 hours. According to NVIDIA, it would take a robot 42 years to get the same experience in the real world. 

The post NVIDIA teaches dexterity to a robot hand appeared first on The Robot Report.



from The Robot Report - Robotics News, Analysis & Research https://ift.tt/vDbCmMK
via artificialconference

Comments

Post a Comment

Popular posts from this blog

Chipotle testing avocado processing robot in test kitchen

-
Image
Chipotle Mexican Grill announced Autocado, an avocado processing cobotic prototype that cuts, cores, and peels avocados before they are hand mashed to create the restaurant’s famous guacamole. The prototype, developed in collaboration with Vebu , is currently being tested at the Chipotle Cultivate Center in Irvine , California.  How Autocado works  A team member loads Autocado with a full case of ripe avocados and selects the size setting. Autocado can hold up to 25 lbs. of avocados at once. One at a time, avocados are vertically oriented, then transferred to the processing device. The avocados are sliced in half. Their cores and skin are automatically removed, and the waste is discarded.  The fruit is safely collected in a stainless-steel bowl at the bottom of the device.  A team member removes the bowl of avocado fruit and moves it to the counter where they add additional ingredients and hand mash the avocados to make Chipotle’s signature guacamole.  How the Autocado idea s
Read more

Noah Medical’s Galaxy System is used in the US for the first time

-
Image
Noah Medical announced that its Galaxy System, an integrated system for navigated robotic bronchoscopy, was used for the first time in the United States at the University of Chicago Medicine by Dr. Kyle Hogarth.  This procedure marks the first commercial release of the Galaxy System in the US following its recent FDA clearance and first-in-human trials at Macquarie University Hospital in Sydney, Australia. The system was designed to give physicians more confidence in locating and navigating to lung nodules during biopsies as part of a more effective and efficient bronchoscopy procedure.   “The Galaxy System is the first robot for peripheral lung navigation that includes integrated imaging, which is the solution bronchoscopists have been asking for,” Hogarth said. “We are excited to be the first hospital in the United States to implement this next-generation robotic solution for our patients which we believe will improve and expedite the early diagnosis and treatment of lung cancer.
Read more

Inside Comaus testing of Intrinsic Flowstate

-
Image
Intrinsic tested its latest platform and Flowstate with Comau , an Italian-based robot manufacturer and global systems integrator. Comau has been working with Intrinsic for several years now to validate Intrinsic’s platform technology and developer product, Flowstate, through real-world use cases.  Flowstate, announced earlier this year in May , is a robotic software development platform. It’s Intrinsic’s first developer product built on the Intrinsic platform. Comau was also introduced as the company’s first industry partner in May.  The Intrinsic Flowstate software  adds a layer of abstraction to traditional robotic software tools with the goal of reducing the development skillset required while making it easier than ever before to program a robot in simulation. “Partnering with Comau over the years has brought our team a tremendous amount of expertise, knowledge, and real-world use cases to focus our efforts on. We’re excited for what’s next and what it will mean to the solutio
Read more