Mandy B. Korpusik
Assistant Professor of Computer Science | Loyola Marymount University
Los Angeles, CA, UNITED STATES
Seaver College of Science and Engineering
Media
Publications:
Documents:
Photos:
Videos:
Audio/Podcasts:
Biography
Professor Mandy Korpusik received her B.S. in Electrical and Computer Engineering from Franklin W. Olin College of Engineering in May, 2013. She completed her S.M. in Computer Science at MIT in June, 2015 and received her Ph.D. from MIT in June, 2019. Her primary research interests include natural language processing and spoken language understanding for dialogue systems. Professor Korpusik used deep learning models to build the Coco Nutritionist application for iOS that allows obesity patients to more easily track the food they eat by speaking naturally. Her long-term research goal is to deploy a collection of AI-based conversational agents that improve the health, well-being, and productivity of people.
Education (3)
Franklin W. Olin College of Engineering: B.S., Electrical and Computer Engineering 2013
Massachusetts Institute of Technology: S.M., Electrical Engineering and Computer Science 2015
Massachusetts Institute of Technology: Ph.D., Electrical Engineering and Computer Science 2019
Areas of Expertise (4)
Spoken Dialogue Systems
Natural Language Processing
Electrical Engineering and Computer Science
Deep Learning
Industry Expertise (2)
Computer Software
Health and Wellness
Media Appearances (3)
Voice-controlled calorie counter
MIT News online
2016-05-24
Spoken-language app makes meal logging easier, could aid weight loss.
MIT researchers launched a new AI system for counting calories.
Inside AI Newsletter online
2019-01-24
The app is called Coco, and users simply say what they have eaten to log their meals. Data is held anonymously and the app is part of an ongoing MIT research project on speech understanding. — COCO NUTRITION
Exploring the nature of intelligence
MIT News online
2019-02-21
Undergraduate research projects show how students are advancing research in human and artificial intelligence, and applying intelligence tools to other disciplines.
Sample Talks (5)
Conference: A Comparison of Deep Learning Methods for Language Understanding
M.Korpusik, Z. Liu, J. Glass Interspeech 2019, Graz, Austria
Workshop: A Food Logging System for iOS with Natural Spoken Language Meal Descriptions
M. Korpusik, S. Taylor, S.Das, C. Gilhooly, S. Roberts, J. Glass Nutrition 2019, Baltimore
Conference: Dialogue State Tracking with Convolutional Semantic Taggers
M. Korpusik, J. Glass ICASSP 2019, Brighton, UK
Workshop: Convolutional Neural Endoder for the 7th Dialogue System Technology Challenge
M. Korpusik, J. Glass DSTC7 Workshop, Honolulu (2019)
Conference: Convolutional Neural Networks for Dialogue State Tracking without Pre-trained Word Vectors or Semantic Dictionaries
M. Korpusik, J. Glass SLT 2018, Athens
Patents (2)
Behavior prediction on social media using neural networks
14/966438
2015-12-11
Patent pending. Example implementations include a system and method of recognizing behavior of a user. In example implementations, a first post and at least one subsequent post indicative of a product and associated with a first social media account is obtained. A relevance probability is calculated for each of the obtained first post and the at least one subsequent post. The obtained first post and the at least one subsequent post are sequentially analyzed by a second neural network to determine output values relevant to probability of purchasing the product. A probability of purchasing the product is calculated based on the determined output values associated with each post and the calculated relevance probabilities. Product-related information is transmitted to the user associated with the obtained first post based on the determined probability of purchasing the product.
A System and Method for Semantic Mapping ofNatural Language Input to Database Entries via Convolutional NeuralNetworks.
15/92239
2018-03-16
Patent pending. A system for associating a string of natural language with items in a relational database includes a first subsystem having a pre-trained first artificial neural network configured to apply a semantic tag selected from a predefined set of semantic labels to a segment of a plurality of tokens representing the string of natural language. A second subsystem includes a second artificial neural network configured to convert the plurality of labeled tokens into a first multi-dimensional vector representing the string of natural language. A third subsystem is configured to rank the first multi-dimensional vector against a second multi-dimensional vector representing a plurality of items in the relational database.
Articles (3)
Deep Learning for Database Mapping and Asking Clarification Questions in Dialogue Systems
IEEE Transactions on Audio, Speech, and Language Processing (Volume: 27 , Issue: 8 , Aug. 2019)M. Korpusik, J. Glass
2019-05-23
Abstract: Food logging is recommended by dieticians for prevention and treatment of obesity, but currently available mobile applications for diet tracking are often too difficult and time-consuming for patients to use regularly. For this reason, we propose a novel approach to food journaling that uses speech and language understanding technology in order to enable efficient self-assessment of energy and nutrient consumption. This paper presents ongoing language understanding experiments conducted as part of a larger effort to create a nutrition dialogue system that automatically extracts food concepts from a user's spoken meal description. We first summarize the data collection and annotation of food descriptions performed via Amazon Mechanical Turk (AMT), for both a written corpus and spoken data from an in-domain speech recognizer. We show that the addition of word vector features improves conditional random field (CRF) performance for semantic tagging of food concepts, achieving an average F1 test score of 92.4 on written data; we also demonstrate that a convolutional neural network (CNN) with no hand-crafted features outperforms the best CRF on spoken data, achieving an F1 test score of 91.3. We illustrate two methods for associating foods with properties: segmenting meal descriptions with a CRF, and a complementary method that directly predicts associations with a feed-forward neural network. Finally, we conduct an end-to-end system evaluation through an AMT user study with worker ratings of 83% semantic tagging accuracy.
Spoken Language Understanding for a Nutrition Dialogue System
IEEE Transactions on Audio, Speech, and Language Processing (Volume 25 , Issue: 7 , July 2017)M. Korpusik, J. Glass
2017-04-17
Abstract: Food logging is recommended by dietitians for prevention and treatment of obesity, but currently available mobile applications for diet tracking are often too difficult and time-consuming for patients to use regularly. For this reason, we propose a novel approach to food journaling that uses speech and language understanding technology in order to enable efficient self-assessment of energy and nutrient consumption. This paper presents ongoing language understanding experiments conducted as part of a larger effort to create a nutrition dialogue system that automatically extracts food concepts from a user's spoken meal description. We first summarize the data collection and annotation of food descriptions performed via Amazon Mechanical Turk (AMT), for both a written corpus and spoken data from an in-domain speech recognizer. We show that the addition of word vector features improves conditional random field (CRF) performance for semantic tagging of food concepts, achieving an average F1 test score of 92.4 on written data; we also demonstrate that a convolutional neural network (CNN) with no hand-crafted features outperforms the best CRF on spoken data, achieving an F1 test score of 91.3. We illustrate two methods for associating foods with properties: segmenting meal descriptions with a CRF, and a complementary method that directly predicts associations with a feed-forward neural network. Finally, we conduct an end-to-end system evaluation through an AMT user study with worker ratings of 83% semantic tagging accuracy.
Use of Natural Spoken Language With Automated Mapping of Self-reported Food Intaketo Food Composition Data for Low-Burden Real-time Dietary Assessment: Method Comparison Study
Journal of Medical Internat ResearchSalima Taylor*, Mandy Korpusik*, Sai Das, Cheryl Gilhooly, Ryan Simpson, James Glass, Susan Roberts (*authors contributed equally)
2021-12-06
Background: Self-monitoring food intake is a cornerstone of national recommendations for health, but existing apps for this purpose are burdensome for users and researchers, which limits use. Objective: We developed and pilot tested a new app (COCO Nutritionist) that combines speech understanding technology with technologies for mapping foods to appropriate food composition codes in national databases, for lower-burden and automated nutritional analysis of self-reported dietary intake. Methods: COCO was compared with the multiple-pass, interviewer-administered 24-hour recall method for assessment of energy intake. COCO was used for 5 consecutive days, and 24-hour dietary recalls were obtained for two of the days. Participants were 35 women and men with a mean age of 28 (range 20-58) years and mean BMI of 24 (range 17-48) kg/m2. Results: There was no significant difference in energy intake between values obtained by COCO and 24-hour recall for days when both methods were used (mean 2092, SD 1044 kcal versus mean 2030, SD 687 kcal, P=.70). There were also no significant differences between the methods for percent of energy from protein, carbohydrate, and fat (P=.27-.89), and no trend in energy intake obtained with COCO over the entire 5-day study period (P=.19). Conclusions: This first demonstration of a dietary assessment method using natural spoken language to map reported foods to food composition codes demonstrates a promising new approach to automate assessments of dietary intake.
Social