From Space To Earth: How An Astronaut In The ISS Operated Ground Robots

Space agencies often think about the problem of how to control spacecraft from the ground, but it's useful sometimes to be able to send commands in the other direction as well: for astronauts in space to be able to control robots on Earth. That's the idea behind a series of "Surface Avatar" tests being performed on the International Space Station (ISS) to trail the method called telerobotics.

A test was performed in 2023 by NASA astronaut Frank Rubio, who was positioned within the station's Columbus module. He was able to control three robots which were located in the German Aerospace Center (DLR) at a location near Munich.

Rubio controlled the trio of robots to perform a task similar to what astronauts might need to do on a Moon mission. He led the robots to remove an instrument called a seismometer, which detects vibrations underground from a lunar lander, and place it onto the ground, where it would need to be located to do its job and detect moonquakes.

The idea was not only to test out the communications aspect but also to see how a human controller can work alongside robots that have a degree of autonomy.

"With Surface Avatar, we [are] combining direct teleoperation approach with supervised autonomy to control multiple robotic assets to accomplish complex tasks: picture a building site on Earth, where a crane might work alongside a bulldozer or excavator to get a big task done," said the head of the European Space Agency's Human Robot Interaction Lab, Thomas Krueger.

In the recent test, Frank Rubio controlled the robots as a team: a robotic lander arm, a humanoid robot named Rollin' Justin, and a rover named Interact. The variety of robot shapes and tools helped Rubio to perform complex operations during the test, like removing a stuck pin or passing over a sample tube to be placed into a lander.

More tests are planned in the series, with Danish astronaut Andreas Mogensen set to operate several robots from the ISS once he arrives there. This will include him operating the same humanoid robot and a robot "dog" the engineers have named Bert.

This future test will involve Mogensen acting as director of the group, sending commands to each robot to perform a particular task while he oversees the mission from the ISS. The system uses inputs such as a joystick which provides resistance feedback to help the operator feel the forces of gravity.

The robots have mounted cameras that send images and videos back to the ISS. Although there's a short communication delay of around 800 milliseconds between Earth and the ISS, it's not enough to cause a problem for near real-time operations.

The creators of the test like to keep the details of exactly what job the astronaut and the robots will perform a secret so that they can see how they respond to novel challenges and whether the team can adapt to changing situations — as they would have to on a real mission.

The use of telerobotics by astronauts is vital for future missions, especially to the Moon. Because of the communication delay of several minutes between Earth and the Moon, controllers on the ground can't control robots on the Moon in real time. The idea is that astronauts could be in a spacecraft in orbit around the Moon, for example, then send robot explorers down to the surface and control them to do tasks like collecting scientific samples.

The same principle could be applied to more extreme environments, such as Mars or even other planetary bodies. This would be especially useful for environments that could be potentially dangerous to human astronauts because of factors such as temperature, radiation, or weather, which robots might be able to withstand better.

"We are excited to be a step closer to giving astronauts and experts on Earth a wide range of possibilities to command and manage teams of different robots in space," said the lead researcher for the project, Neal Y. Lii of the DLR Institute of Robotics and Mechatronics. "We will be able to use our robots on the surface as their physical avatars and intelligent co-workers to execute ever more complex tasks."