Experts Matter. Find Yours.
Connect for media, speaking, professional opportunities & more.
Virtual reality training tool helps nurses learn patient-centered care
University of Delaware computer science students have developed a digital interface as a two-way system that can help nurse trainees build their communication skills and learn to provide patient-centered care across a variety of situations. This virtual reality training tool would enable users to rehearse their bedside manner with expectant mothers before ever encountering a pregnant patient in person. The digital platform was created by students in Assistant Professor Leila Barmaki’s Human-Computer Interaction Laboratory, including senior Rana Tuncer, a computer science major, and sophomore Gael Lucero-Palacios. Lucero-Palacios said the training helps aspiring nurses practice more difficult and sensitive conversations they might have with patients. "Our tool is targeted to midwifery patients,” Lucero-Palacios said. “Learners can practice these conversations in a safe environment. It’s multilingual, too. We currently offer English or Turkish, and we’re working on a Spanish demo.” This type of judgement-free rehearsal environment has the potential to remove language barriers to care, with the ability to change the language capabilities of an avatar. For instance, the idea is that on one interface the “practitioner” could speak in one language, but it would be heard on the other interface in the patient’s native language. The patient avatar also can be customized to resemble different health stages and populations to provide learners a varied experience. Last December, Tuncer took the project on the road, piloting the virtual reality training program for faculty members in the Department of Midwifery at Ankara University in Ankara, Turkey. With technical support provided by Lucero-Palacios back in the United States, she was able to run a demo with the Ankara team, showcasing the UD-developed system’s interactive rehearsal environment’s capabilities. Last winter, University of Delaware senior Rana Tuncer (left), a computer science major, piloted the virtual reality training program for Neslihan Yilmaz Sezer (right), associate professor in the Department of Midwifery, Ankara University in Ankara, Turkey. Meanwhile, for Tuncer, Lucero-Palacios and the other students involved in the Human-Computer Interaction Laboratory, developing the VR training tool offered the opportunity to enhance their computer science, data science and artificial intelligence skills outside the classroom. “There were lots of interesting hurdles to overcome, like figuring out a lip-sync tool to match the words to the avatar’s mouth movements and figuring out server connections and how to get the languages to switch and translate properly,” Tuncer said. Lucero-Palacios was fascinated with developing text-to-speech capabilities and the ability to use technology to impact patient care. “If a nurse is well-equipped to answer difficult questions, then that helps the patient,” said Lucero-Palacios. The project is an ongoing research effort in the Barmaki lab that has involved many students. Significant developments occurred during the summer of 2024 when undergraduate researchers Tuncer and Lucero-Palacios contributed to the project through funding support from the National Science Foundation (NSF). However, work began before and continued well beyond that summer, involving many students over time. UD senior Gavin Caulfield provided foundational support to developing the program’s virtual environment and contributed to development of the text-to-speech/speech-to-text capabilities. CIS doctoral students Fahim Abrar and Behdokht Kiafar, along with Pinar Kullu, a postdoctoral fellow in the lab, used multimodal data collection and analytics to quantify the participant experience. “Interestingly, we found that participants showed more positive emotions in response to patient vulnerabilities and concerns,” said Kiafar. The work builds on previous research Barmaki, an assistant professor of computer and information sciences and resident faculty member in the Data Science Institute, completed with colleagues at New Jersey Institute of Technology and University of Central Florida in an NSF-funded project focused on empathy training for healthcare professionals using a virtual elderly patient. In the project, Barmaki employed machine learning tools to analyze a nursing trainee’s body language, gaze, verbal and nonverbal interactions to capture micro-expressions (facial expressions), and the presence or absence of empathy. “There is a huge gap in communication when it comes to caregivers working in geriatric care and maternal fetal medicine,” said Barmaki. “Both disciplines have high turnover and challenges with lack of caregiver attention to delicate situations.” UD senior Rana Tuncer (center) met with faculty members Neslihan Yilmaz Sezer (left) and Menekse Nazli Aker (right) of Ankara University in Ankara, Turkey, to educate them about the virtual reality training tool she and her student colleagues have developed to enhance patient-centered care skills for health care professionals. When these human-human interactions go wrong, for whatever reason, it can extend beyond a single patient visit. For instance, a pregnant woman who has a negative health care experience might decide not to continue routine pregnancy care. Beyond the project’s potential to improve health care professional field readiness, Barmaki was keen to note the benefits of real-world workforce development for her students. “Perceptions still exist that computer scientists work in isolation with their computers and rarely interact, but this is not true,” Barmaki said, pointing to the multi-faceted team members involved in this project. “Teamwork is very important. We have a nice culture in our lab where people feel comfortable asking their peers or more established students for help.” Barmaki also pointed to the potential application of these types of training environments, enabled by virtual reality, artificial intelligence and natural language processing, beyond health care. With the framework in place, she said, the idea could be adapted for other types of training involving human-human interaction, say in education, cybersecurity, even in emerging technology such as artificial intelligence (AI). Keeping people at the center of any design or application of this work is critical, particularly as uses for AI continue to expand. “As data scientists, we see things as spreadsheets and numbers in our work, but it’s important to remember that the data is coming from humans,” Barmaki said. While this project leverages computer vision and AI as a teaching tool for nursing assistants, Barmaki explained this type of system can also be used to train AI and to enable more responsible technologies down the road. She gave the example of using AI to study empathic interactions between humans and to recognize empathy. “This is the most important area where I’m trying to close the loop, in terms of responsible AI or more empathy-enabled AI,” Barmaki said. “There is a whole area of research exploring ways to make AI more natural, but we can’t work in a vacuum; we must consider the human interactions to design a good AI system.” Asked whether she has concerns about the future of artificial intelligence, Barmaki was positive. “I believe AI holds great promise for the future, and, right now, its benefits outweigh the risks,” she said.
Digital platforms have emerged as powerful tools for people impacted by the Russo-Ukrainian War. One professor at the University of Delaware has, for over two years, provided reading resources specifically for the children whose lives have been forever changed by this conflict. Roberta Michnick Golinkoff, the Unidel H. Rodney Sharp Chair and Professor at UD's College of Education and Human Development, has developed a website with free interactive e-books, games and other resources to Ukrainian children. A nationally known expert in childhood literacy, Golinkoff worked together with developers to stock the site, Stories with Clever Hedgehog, with materials in both Ukrainian and English. The multilingual platforms allows displaced families all over the world to engage in shared reading with their children, facilitate early literacy development and promote well-being during a time of stress. In addition to enhancing learning experiences, digital platforms provide an essential sense of community and connectivity for students isolated by conflict. Golinkoff, who has appeared in numerous national outlets including NPR, ABC News and The Conversation, is available for interviews on the site as well as literacy in general. Just click her profile to get in touch.
Among the casualties of the Ukraine war are the 2 million-plus children who have been forced to flee their homes, which has upended what should be a joyous time filled with laughing and learning. In an effort to restore some sense of normalcy to their education years, University of Delaware professor Roberta Michnick Golinkoff has developed a website with free interactive e-books, games and other resources to Ukrainian children. Golinkoff—a nationally known expert in childhood literacy—worked together with developers to stock the site, Stories with Clever Hedgehog, with materials in both Ukrainian and English. The multilingual platforms allows displaced families all over the world to engage in shared reading with their children, facilitate early literacy development and promote well-being during a time of stress. Golinkoff, who has appeared in numerous national outlets including NPR, ABC News and The Conversation, is available for interviews. Just click the "View Profile" button to get in touch.
Aston University to launch Aston Centre for Applied Linguistics
The Centre is an interdisciplinary, multilingual group of researchers made up of academic staff and research students It aims to build on Aston University’s longstanding expertise in research into language education, languages, and applied linguistics The hybrid launch event will take place at the University on 14 September Aston University is launching a new research centre within its College of Business and Social Sciences. The Aston Centre for Applied Linguistics (ACAL), formerly known as the Centre for Language Research at Aston (CLaRA), aims to build on Aston University’s longstanding expertise in research into language education, languages, and applied linguistics by promoting interdisciplinary collaboration and establishing national and international networks and partnerships. ACAL is an interdisciplinary, multilingual group of researchers – academic staff and research students – who work in the field of language and language education research. The Centre will officially be launched through a hybrid event at the University on 14 September 2022. There will be talks by Aston University’s Dr Lucia Busso & Dr Marton Petyko, Dr Marcello Giovanelli, Dr Megan Mansworth and Dr Emmanuelle Labeau as well as guest lectures from Professor Zhu Hua (IOE faculty of education and society, UCL) and Terry Lamb (professor of languages and interdisciplinary pedagogy, University of Westminster and Aston University language graduate). The event will conclude with a celebration of the major publications of ACAL members in 2021-22. Dr Emmanuelle Labeau, director of ACAL, said: “Language actually is all around us: we use it to articulate all our human activities. “Languages actually are all around us: over 100 languages are spoken in Birmingham “My recent AHRC-funded project BRUM (Birmingham Research for Upholding Multilingualism) has shown that research in language(s) is needed in local schools, businesses, public services and culture. “ACAL wants to put the ‘applied’ into linguistics to serve the University, the city, region and beyond. Our researchers are a great asset to the University’s ambitions, and we cannot wait informing and helping shape the Aston University 2030 Strategy.”
Aston University appoints translator in residence to inspire future literary translators
Aston University and Writing West Midlands have appointed Sarah Letza, a professional translator from the region to the role She will be working on community engagement activities at Aston University and with the West Midlands community The project is funded by Arts Council England. Aston University has appointed a translator in residence as part of a project developed by the translators Daniel Hahn and Ros Schwartz, a new one-off literary translation residency for 2022 based in Birmingham. Sarah Letza, a professional translator from the region, has been appointed to the position until March 2023. The project is funded by Arts Council England and will be run in partnership with Aston University and Writing West Midlands. Sarah’s residency will encourage an interest in literary translation and translated literature, and the valuing of languages, among communities in and around Birmingham through workshops, talks and other public engagement and collaborative participation projects. Sarah Letza, translator in residence at Aston University, said: “I hope to draw attention to the opportunities that translation and language learning can lead to and capitalise on the wealth of talent that can be drawn from Midlands schools and universities. “There is the potential to be involved with the many translation research projects already under development at Aston University as well as promoting and supporting languages and translation pathways. “I want to showcase Birmingham as a city that has been built on cultural crossings, intellectual exchange and creativity, all of which are the bedrock of both translation and language learning.” Dr David Orrego-Carmona, a lecturer in English, languages and applied linguistics at Aston University, said: “Birmingham is an incredibly multilingual city and Aston University is particularly well-placed to raise awareness and showcase the value of languages, translation and intercultural exchanges. “We are certain the programme will provide inspiring experiences for our beneficiaries in Birmingham and the region, and we are looking forward to the engaging activities that Sarah is already planning.” For more information about our courses in translation click here.