Lifu Huang. Photo by Peter Means for Virginia Tech.
Considering the millions of research papers and reports from open domains such as biomedicine, agriculture, and manufacturing, it is humanly impossible to keep up with all the findings.
Constantly emerging world events present a similar challenge because they are difficult to track and even harder to analyze without looking into thousands of articles.
And she is part of the Virginia Tech team from the Sanghani Center that is one of 10 finalists chosen to compete in the Alexa Prize SimBot Challenge. The challenge focuses on advancing the development of next-generation virtual assistants that continuously learn and gain the ability to perform common sense reasoning to help humans complete real-world tasks.
Wang’s specific role on the team — advised by Lifu Huang, who is also her academic advisor — is to establish knowledge graphs from instructional articles, images, and video demonstrations on the internet, such as WikiHow. She will also concentrate on collectively grounding entities and actions extracted from text to video to associate each entity or action with a visual image or video clip.
In her research, Wang focuses on natural language processing and machine learning, particularly information extraction with full or limited supervision.
Information extraction, she said, poses challenges because of its sophisticated annotation needs and variance benchmarks, she said.
“I am trying to automatically extract structured information from unstructured data,” Wang said. “For example, in the sentence ‘Melony was married just a month before she left for Iraq,’ the word ‘she’ indicates Melony, and her marriage occurs before the movement event. My research focus is to extract this information from the input sentence.”
Wang said that as a young child she wanted to understand foreign languages but knew that it would take a great effort to do so. “When I learned about machine learning as an undergraduate student, I was really drawn to it because of how we can use its model fitting and pattern learning abilities to automatically understand visual content.”
Wang holds a bachelor’s degree in vehicle engineering from Southwest Jiaotong University in China and a master’s degree in computer science from Washington University in St. Louis. She was drawn to Virginia Tech and the Sanghani Center for a Ph.D. computer science program because of the experienced professors and their cutting-edge research in artificial intelligence and data science. “Their work and achievements and all the passionate students around me have motivated me to work harder,” she said.
Being a Ph.D. student has made her realize how much time and effort it takes to become a successful academic researcher, she said. “So after graduation, I will be looking for a postdoc position or other research opportunities at private and research labs to become better equipped to become a research scientist.”
One of 10 finalists in the Alexa Prize SimBot Challenge, Virginia Tech’s team members meet regularly for updates on their specific work and overall progress on the project. The winner will be announced in 2023. Photo by Andrew Cybak for Virginia Tech.
A Virginia Tech team from the Sanghani Center for Artificial Intelligence and Data Analytics is one of 10 finalists chosen to compete in the Alexa Prize SimBot Challenge. The challenge focuses on advancing the development of next-generation virtual assistants that continuously learn and gain the ability to perform common sense reasoning to help humans complete real-world tasks.
“The SimBot should be able to understand the intention of a task as well as any instructions or feedback it receives from a user and interpret the environment to correctly predict what action is needed to complete it,” said Lifu Huang, assistant professor of computer science and faculty at the Sanghani Center. Click here to read more about how the team will tackle this challenge.
