University of South Carolina and West Virginia University researchers work together to explore safer ways for people and robots to work together.
As robots and other advanced technologies become more common in manufacturing, workers are increasingly being asked to operate alongside and collaborate with machines and digital systems that can be difficult to understand. Researchers at the University of South Carolina and West Virginia University are working to make that transition safer.
A collaboration between USC’s Molinaroli College of Engineering and Computing and West Virginia University’s Statler College of Engineering and Mineral Resources is exploring how augmented reality, digital twins and robotics can improve workers training and safety.
The work is part of a National Science Foundation (NSF)-funded research project focused on workforce development through mixed reality and digital twins - a virtual representation of a physical object or system that can reflect its movements and behavior. The technology is considered part of Industry 4.0, the era of industrial development which combines technologies such as artificial intelligence, robotics and connected digital systems with traditional manufacturing.
While advances in manufacturing technology can make production more efficient, they also demand new skills from the people who operate and work alongside increasingly complex equipment. By bringing together complementary expertise from USC and West Virginia University, the research team is exploring how technology can better prepare workers before they enter an industrial environment.
“We bring our augmented reality, human safety and ergonomics expertise alongside USC’s strength in digital twins, smart manufacturing and human-robot collaboration,” says JuHyeong Ryu, an assistant professor in WVU’s Department of Industrial and Management Systems Engineering.
Ryu submitted the NSF proposal and selected USC as the host site because of its research facilities and established a relationship through a seed grant from the NSF funded F2F project. Mechanical Engineering Professor Thor Wuest, a former colleague of Ryu at WVU, is currently working on the project.
“It’s a wonderful extension of the work that we are doing by incorporating his unique perspective in human factors and safety,” Wuest said.
Nishat Alam, a WVU industrial engineering doctoral student has spent the past two summers conducting research at USC. During her first summer, Alam explored how digital twins and robotics could work together. This past summer, she conducted experiments in the Future Factories lab at the college’s McNair Center for Aerospace Innovation and Research, with a focus on using augmented reality to develop safety training.
“I designed a step-by-step assembly task where I work alongside the robot, and I studied how we can make that interaction safer and more effective,” Alam says.
Alam used Unity software to create interactive 3D environments and develop a virtual robot model connected to the physical robot. “After creating a model, the question was, ‘How can I connect the digital twin robot with the physical robot?’” she says.
Alam then designed an assembly task to explore how workers could be trained to safely collaborate with a robot, with an emphasis on maintaining a safe distance and following general safety instructions. The training incorporated Microsoft HoloLens, a mixed-reality headset that can place digital information into a user’s field of view. The system can also provide workers with information about potential risky body postures while they practice a task, allowing them to learn safer ways of performing the task in a controlled environment.
“Sometimes workers are more committed to productivity than safety, but with this training, they can learn how to safely collaborate with a robot beforehand and transfer those skills them to real-world settings,” Alam says.
USC mechanical engineering doctoral student and graduate research assistant Md Irfan Khan worked alongside Alam. He programmed the robot for the study, helped develop its digital twin and assisted with experiments and data collection.
“What impressed me most was seeing how people interacted with the robot and how their behavior and responses changed during the human-robot collaboration,” Khan says. “It was especially interesting to see how technologies can be used to prepare people for interacting with robotic systems before working directly with the physical robot.”
Khan added that the project showed him how many elements must come together to conduct research involving people and robots.
“Beyond developing and programming the technology, I learned about experimental design, working with human participants, troubleshooting technical issues and adapting the system based on what we observed during the experiments,” he says.
Wuest said Alam quickly adapted to the Future Factories lab, despite having limited robotics experience before joining the project.
“She’s a very thorough researcher, and I was impressed with how quickly she understood the lab and the robots,” Wuest said. “Working with robotics can be overwhelming, but she picked it up quickly. She’s a self-starter and connected well with my team.”
For Ryu, the research is ultimately about more than introducing new technology into manufacturing. It is about finding better ways to prepare people for workplaces that are changing rapidly, a concern that runs through his work in occupational safety and ergonomics.
“Technology and industry have changed so fast, but our training methods still lag behind,” Ryu said. “Our goal is to find a balance for a new generation of workers and better prepare them for the industry. The way we prepare people has to change just as fast.”