Using control theory and artificial intelligence to enable the safe deployment of distributed autonomous systems is a specialty of Patrick J. Martin, Ph.D., assistant professor in the Department of Electrical and Computer Engineering. He recently applied these concepts to a dance collaboration between a human and a robot.
How did you get involved in creating human-robot teams, such as your recent project with a dancer and a robot?
During my Ph.D. work at Georgia Tech, I researched and developed a robotic puppetry system, where I collaborated with the Center for Puppetry Arts in Atlanta. I also worked with other colleagues who mapped human motions to mechanical systems with lower degrees of freedom. When I came to VCU, I wanted to continue this work by engaging with the VCU Department of Dance + Choreography. I also wanted to expand my prior work to build teams of humans and autonomous robots that naturally adapt to each other in real-time.
How will these human-robot creative collaborations benefit society -- including people who don't like to dance?
Roboticists have much to learn from dancers and choreographers, who are experts in expressive motion and coordination in multi-person performances. The architectural and algorithmic outputs from this research will inform how we integrate autonomous robots into situations where there are multiple humans and robots working together to achieve complex mission goals, while maintaining the safety of all involved. These advances will allow robots to be deployed in challenging areas where humans are in close proximity, from transportation to disaster response to healthcare.
Tell us a little bit about how the technology behind this works.
This research effort focuses on two aspects: 1) learning new motions from humans and 2) assembling a performance from multiple robot behaviors. Our algorithm uses machine learning to extract key motion parameters of demonstrated motions and allows the user to mix these parameterized motions to achieve different expressive motifs, such as tempo. We build our robot performance using hierarchical state machines where each state represents a robot behavior, such as the learned arm motions or a controller moves the robot in a smooth arc. Combining these two capabilities provides a framework for more flexible human-robot choreography.
What is the role of creativity in engineering?
Creativity is a key aspect in the design phase of any engineered system. The fruition of new solutions takes many iterations, in which ideas are prototyped, tested and refined. This process is inherently creative, as each new cycle allows an engineer to reach outside their technical comfort zone and make connections that will improve the design.
Who else from VCU is working with you on this project?
This project is a joint exploration with Dr. Kate Sicchio, who is in both the Department of Dance + Choreography and Department of Kinetic Imaging in VCUArts. Charles Dietzel, who was the lead developer of our motion learning and generation algorithm, is one of my M.S. students in the Department of Electrical and Computer Engineering. I also have several Vertically Integrated Project (VIP) undergraduate researchers who support the development and integration of new robot sensor capabilities.
Fun fact: I am a fan of James Joyce novels, but like to balance these classics with sci-fi and cyberpunk novels.