Doorstep scams are scams in which a con artist has a convincing, but fraudulent, story with the purpose of coming into your house and/or stealing money. Often these scams appear at the doorstep, for example when somebody wants to enter your house because they must check the electricity or with similar excuses. However, it also happens that the con artist calls you by phone (telling a story about fraudulent payments, aiming to get banking information for example) or approaches you on the street. Elderly people are often the victims of such doorstep scams, which usually have a high impact on their lives.

Within this project, we are creating a tablet application that can be used to train how to verbally act in such situations. The users both learn what to say, and how to say it, in various scenarios. They will both receive automated feedback on how they dealt with the different situations (what they said) and on the assertiveness of their voice (using an algorithm to analyze vocal signals).

This project is a collaboration between the VU and Unie KBO-PCOB.

VU staff working on this project: Romy Blankendaal, Tibor Bosse, Daniel Formolo, Charlotte Gerritsen, Laura van der Lubbe, Marco Otte

This project introduces the concept of “virtual bad guys”: intelligent virtual agents that take a negative or even aggressive stance towards the user. Although they pave the way to various interesting applications, it is hard to create virtual bad guys that are taken seriously by the user, since they are typically unable to apply serious sanctions. To address this issue, this study experimentally investigated the effect of “consequential” agents that are able to physically threaten their human interlocutors. A consequential agent was developed by equipping users with a (non-functioning) device, through which they were made to believe the agent could mildly shock them. Effects on participants’ levels of anxiety and (physiological and self-reported) stress were measured, and the role of presence and perceived believability of the virtual agent was assessed. The consequential agent triggered a stronger physiological stress response than the non-consequential agent, whereas self-reported levels of anxiety and stress did not significantly differ. Furthermore, while presence and believability were substantially associated with users’ stress response, both states did not mediate or explain the effect of a consequential vs. non-consequential agent on stress, as they did not significantly differ between conditions. Implications of these findings and suggestions for follow-up studies on “virtual bad guys” are discussed.

The Tech Labs helped develop the virtual environment, using the custom developed Galvanic Skin response sensor and supplying the VR equipment to run the experiment.

Researchers: Tibor Bosse, Tilo Hartmann, Romy Blankendaal, Nienke Dokter, Marco Otte

In this study a virtual environment was build to see if a virtual training agent could help autistic children in trying to relate to other children. The idea was that a virtual environment feels safer and the children will more easily accept instructions and interact with the virtual agents.

The participants would sit in a chair, wear a virtual reality headset (Oculus Rift DK2) and wear a Blood-Volume-Pulse sensor on their index finger to measure heart rate.

The scene was a classroom with two children, boy and girl, and an adult trainer (male). The virtual trainer would ask the virtual children and the participant simple questions. The actual answer of the participant was not important, but the change in heart rate was. By determining a baseline heart rate at the beginning of each trial, the software checked the actual heart rate at several points in the scenario. If the heart rate was above a certain threshold, the virtual trainer would first try to calm the participant down before continuing the conversation.

Reseachers: Laura Helgering, Michel Klein