Computer Science Ph.D. student Misagh Soltani is passionate about artificial intelligence. As a graduate researcher at the Artificial Intelligence Institute of USC (AIISC), he channels his passion into furthering model-based deep reinforcement learning. It is work he hopes will make tasks easier and information more accessible for the betterment of society.
Reinforcement learning is a branch of machine learning where an agent interacts with an environment and learns, through trial and error, how to make decisions to efficiently achieve the desired goals. Reinforcement learning approaches can be used in a variety of domains from manufacturing, where it optimizes robot motion and automation, to the natural sciences.
In the automotive industry, it helps develop self-driving vehicles, optimize regenerative braking systems, and improve decision-making in driver-assistance systems. These algorithms can be trained in simulated environments where they learn from many different scenarios, such as learning to navigate around obstacles or adjusting to different road conditions. Even the financial industry can benefit from reinforcement learning with algorithmic trading and portfolio-optimization strategies.
Soltani’s passion for AI and machine learning stems from their ability to draw patterns and correlations from data and reveal different perspectives that may not otherwise be easily seen by humans.
“When I see how algorithms are performing, I am excited about how AI algorithms can uncover patterns and map real-life events or interactions,” Soltani says. “Sometimes these patterns are clear for us as humans, but with others we don’t always think about it in a particular way. This can lead to discovering new knowledge and approaching problems differently.”
Since AI is widely available in tools such as ChatGPT, Soltani views it as benefiting society by making daily tasks easier and information more accessible. Despite concerns about performance safety of robots, the accuracy of large language models, and ethical issues such as copyright infringement, Soltani believes AI be to a valuable tool that increases productivity.
“Like any other tool or technology, there are positive and negative impacts,” Soltani says. “It is more about how human beings are using AI-powered tools than how they actually interact. If used ethically and responsibly, it makes education and resources more accessible. And if I can develop an algorithm that makes people’s lives easier, then I have achieved my goal.”
When it comes to ethical issues such as originality of content, Soltani argues that technology has made original content, such as written work and photographs, more valuable. He refers to the invention of the camera, which made portraits more accessible, yet original paintings more meaningful.
“Painters didn’t lose their jobs when cameras were invented; it redefined their role and made their work more valuable,” Soltani says. “The same is true for content creators today such as writers and graphic designers when they produce something original.”
For the last year and a half, Soltani has been the only student researcher compiling reinforcement learning data through learned simulation under Assistant Professor Forest Agostinelli in the AIISC.
“To solve a real-word problem using AI, you need data, but real-world data is expensive because you are conducting trial and error with robots,” Agostinelli says. “In simulation, we can get as much data as we want because it all takes place in a computer. Misagh is leading this project using a learned simulator to solve a problem and transfer that back to the real world.”
In the lab, Soltani applied this technology to the real-world problem of the Rubik’s cube. In a simulation, the machine learned how to solve the puzzle using pictures to determine how to manipulate the cube.
“We hope to apply this to object manipulation with robots,” Agostinelli says. “If we ask a robot to assemble something, such as in manufacturing, it may involve many steps of reasoning. We are interested in teaching it how to do that.”
Prior to his work at the AIISC, Soltani interned with TESTIFI GmbH (now Nenya GmbH), a Munich-based software as a service company specializing in enterprise software test automation, where he contributed to the implementation of test-automation solutions. While not directly related to his current research in reinforcement learning, the position gave him experience collaborating across different teams and perspectives, a soft skill important to his research in the lab.
“Working with Misagh is a pleasure,” says Rojina Panta, a computer science Ph.D. student and Soltani’s lab mate. “Although our research applications differ, we work on similar problems, specifically pathfinding algorithms using reinforcement learning and heuristic search. He is always ready to provide information and assistance on topics he is knowledgeable about, and he is approachable, making it easy to discuss new ideas with him.”
Soltani is scheduled to earn his doctorate in May 2027. While considering careers in both industry and academia, his goals align more closely with academia, where he can teach and continue his own research.
“When I think about academia, it is more interesting to me,” he says. “You are able to develop your own ideas and follow your interests, while at the same time interacting with others from different fields, all with diverse ideas that help you develop yours even further.”