Skip to main content

Georgia Tech researchers upgrade robot arm with point-and-click interface

Georgia Tech Creates Point and Click Method for Controlling Robots
The Georgia Institute of Technology has developed a new, intuitive method for controlling a robotic arm: Point and click. While not exactly like the adventure games of old, it definitely draws its inspiration from the ability to highlight objects and perform a small selection of preselected tasks.

Although typical robotic arm controls offer a wide variety of options thanks to six degrees of motion, that can be cumbersome for people just getting to grips with the technology. What the new solution looks to do, is make it possible for novice robotics owners to utilize the technology in a manner that has a much shorter and shallower difficulty curve.

“Roboticists design machines for specific tasks, then often turn them over to people who know less about how to control them,” said David Kent, a Georgia Tech robotics doctoral student who headed up the project. “Most people would have a hard time turning virtual dials if they needed a robot to grab their medicine. But pointing and clicking on the bottle? That’s much easier.”

While simpler on the surface, such a system is a lot more complicated under the hood. Instead of displaying 3D information to the user, they simply offer a video feed. However, the arm itself needs to understand the 3D environment and know that when a person clicks on an object in the background and tells it to pick it up, it must differentiate that from objects in the foreground. It needs to know the distance that it must travel and the correct angle to “attack” it from.

The algorithm that the research team has come up with is capable of analyzing the geometry of objects, even making assumptions about them when the camera can’t see their entirety. This is something that the human brain does automatically, the researchers explained, but it was a challenge to make the robotic arm infer that detail for itself.

Although this technique of control does bequeath much of the responsibility of control to the algorithm and the arm, the system is actually less error prone than giving humans full control. In a trial, researchers found the point and click method delivered only one mistake per task, whereas the more traditional six degrees of motion control method had as many as four per task.

By giving users control over what to grab and how to do so when the arm is in the right position, researchers hope that robotic technology could more easily be adopted by those without training, or with limited mobility themselves, like the elderly. There are also potential applications for such robotic arms in sensitive settings, like search and rescue operations or in space travel.

Jon Martindale
Jon Martindale is the Evergreen Coordinator for Computing, overseeing a team of writers addressing all the latest how to…
A dangerous new jailbreak for AI chatbots was just discovered
the side of a Microsoft building

Microsoft has released more details about a troubling new generative AI jailbreak technique it has discovered, called "Skeleton Key." Using this prompt injection method, malicious users can effectively bypass a chatbot's safety guardrails, the security features that keeps ChatGPT from going full Taye.

Skeleton Key is an example of a prompt injection or prompt engineering attack. It's a multi-turn strategy designed to essentially convince an AI model to ignore its ingrained safety guardrails, "[causing] the system to violate its operators’ policies, make decisions unduly influenced by a user, or execute malicious instructions," Mark Russinovich, CTO of Microsoft Azure, wrote in the announcement.

Read more