Training the Elderly

Doorstep scams are scams in which a con artist has a convincing, but fraudulent, story with the purpose of coming into your house and/or stealing money. Often these scams appear at the doorstep, for example when somebody wants to enter your house because they must check the electricity or with similar excuses. However, it also happens that the con artist calls you by phone (telling a story about fraudulent payments, aiming to get banking information for example) or approaches you on the street. Elderly people are often the victims of such doorstep scams, which usually have a high impact on their lives.

Within this project, we are creating a tablet application that can be used to train how to verbally act in such situations. The users both learn what to say, and how to say it, in various scenarios. They will both receive automated feedback on how they dealt with the different situations (what they said) and on the assertiveness of their voice (using an algorithm to analyze vocal signals).

This project is a collaboration between the VU and Unie KBO-PCOB.

VU staff working on this project: Romy Blankendaal, Tibor Bosse, Daniel Formolo, Charlotte Gerritsen, Laura van der Lubbe, Marco Otte

Virtually Bad

This project introduces the concept of “virtual bad guys”: intelligent virtual agents that take a negative or even aggressive stance towards the user. Although they pave the way to various interesting applications, it is hard to create virtual bad guys that are taken seriously by the user, since they are typically unable to apply serious sanctions. To address this issue, this study experimentally investigated the effect of “consequential” agents that are able to physically threaten their human interlocutors. A consequential agent was developed by equipping users with a (non-functioning) device, through which they were made to believe the agent could mildly shock them. Effects on participants’ levels of anxiety and (physiological and self-reported) stress were measured, and the role of presence and perceived believability of the virtual agent was assessed. The consequential agent triggered a stronger physiological stress response than the non-consequential agent, whereas self-reported levels of anxiety and stress did not significantly differ. Furthermore, while presence and believability were substantially associated with users’ stress response, both states did not mediate or explain the effect of a consequential vs. non-consequential agent on stress, as they did not significantly differ between conditions. Implications of these findings and suggestions for follow-up studies on “virtual bad guys” are discussed.

The Tech Labs helped develop the virtual environment, using the custom developed Galvanic Skin response sensor and supplying the VR equipment to run the experiment.

Researchers: Tibor Bosse, Tilo Hartmann, Romy Blankendaal, Nienke Dokter, Marco Otte

Can VR help autistic children?

In this study a virtual environment was build to see if a virtual training agent could help autistic children in trying to relate to other children. The idea was that a virtual environment feels safer and the children will more easily accept instructions and interact with the virtual agents.

The participants would sit in a chair, wear a virtual reality headset (Oculus Rift DK2) and wear a Blood-Volume-Pulse sensor on their index finger to measure heart rate.

The scene was a classroom with two children, boy and girl, and an adult trainer (male). The virtual trainer would ask the virtual children and the participant simple questions. The actual answer of the participant was not important, but the change in heart rate was. By determining a baseline heart rate at the beginning of each trial, the software checked the actual heart rate at several points in the scenario. If the heart rate was above a certain threshold, the virtual trainer would first try to calm the participant down before continuing the conversation.

Reseachers: Laura Helgering, Michel Klein

Tech Labs VR featured in Discovery Channel documentary

As most of you have probably seen in the Tech Labs Newsletters, the Tech Labs have been working on a virtual neighborhood for studying burglary. These studies are a collaboration between the NSCR (Jean-Louis van Gelder), the University of Porstmouth (Claire Nee) and the Network Institute’s Tech Labs (Marco Otte). The previous version of the Virtual Environment was used in several studies including one carried out in British prisons using actual criminals.

Discovery Channel made a short documentary of this study which features our virtual neighborhood.

At the moment we are working on the second version of the neighborhood featuring many more houses with different features offering an even more realistic environment to use in studies.

Sun Joo (Grace) Ahn (Stanford) talks about Immersive VR Sept 15

As part of the unfolding activities of Network Iinstitute’s VR group, we would be excited if you like to save the date for an upcoming presentation by Sun Joo (Grace) Ahn, former Stanford Ph.D. of Jeremy Bailenson. Grace will talk about “Persuasive Technologies: Using Immersive Virtual Environments to Promote Attitude and Behavior Change.”
Grace will visit the VU Communication Science department Tuesday September 15. Her presentation is scheduled for 15:30 – 17:00. The exact room/location at the VU will be announced shortly!
If you like to meet with Grace 1:1 on Tuesday to talk about research, please let me know so that I can bundle and forward requests to her.
Abstract of presentation: 
“Immersive virtual environments offer several novel affordances that allow users to interact with and experience mediated events that were difficult or impossible with traditional media. The effects of these virtual experiences transfer into the physical world to transform attitudes and behavior, shifting our traditional understanding of communication patterns. This talk addresses how immersive virtual environments may be used as tools of persuasion in the realms of health and consumer behavior, discussing the tripartite interactions between reality, virtual reality, and humans.
By the way: Also note the upcoming Dutch VR days in Amsterdam (you can pre-register now)!  – an excellent opportunity to try out VR technology, demos and applications.

Are Aggressive Agents as Scary as Aggressive Humans?

This question has recently been investigated by a team of researchers from the VU Computer Science department and the NSCR. The results have been published at the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), including the two Virtual Reaklity Group members Tibor Bosse and Jeroen de Man ( see for full article http://stress.few.vu.nl/publications/2015-BlankendaalEtAl-AAMAS.pdf). A summary of the article is provided below:

Most research into intelligent virtual agents focuses on agents with a positive stance towards the user. Nevertheless, the development of virtual agents that show aggressive behavior may also be interesting for a range of application domains, varying from aggression de-escalation training to anti-bullying education. However, ensuring that such aggressive agents achieve the desired effect is not easy, as they need to be believable in a number of aspects. In particular, they need to bring their human conversation partners into a serious state of anxiety. To investigate to what extent this can be achieved using state-of-the-art virtual agent technology, an experiment was performed in which the impact of an aggressive virtual agent was compared with that of an aggressive human. By randomly distributing a group of 28 participants over two conditions (virtual and human) and measuring their physiological and subjective emotional state before and after an aggressive outburst of their conversation partner, the difference between virtual and human aggression was studied. The results point out that both types of aggression induced a substantial stress response, but that the impact of the human aggression was higher than that of the virtual aggression.

vr_blogTibor

Figure note: This figure depicts the dynamics of the electrodermal activity over time in microSiemens during the relevant part of the experiment, averaged over all participants in each condition. The horizontal axis denotes a period of 2 minutes, i.e., 1 minute before the start of the aggressive outburst and 1 minute after it. The vertical line indicates the moment the outburst started.

Feeling present in an impossible Virtual Reality

For most people, the term Virtual Reality will bring to mind visions of imaginary worlds, complicated devices and gaming teens, but nowadays, VR is increasingly being used for purposes other than gaming. Surgeons, for example, use it to operate using a robot. It is also used in training and education in the military, sports, healthcare and engineering. These tasks benefit tremendously from VR, because the environment engages multiple senses and creates a feeling of being located in a virtual world. This feeling is called Presence.

We know certain things are disruptive to this sense of Presence – the equipment that is used for example: head mounted displays are often heavy, and sometimes glasses steam up or get smudges on them. Noises from the physical environment, or bumping into a table can disturb the feeling of Presence. These small disturbances can remind a user that he or she is using a VR system, and is not actually in the physical environment. Technical problems, such as sudden changes in frame rate or unresponsive controls, can also cause a person to lose the feeling of location in the virtual world.

vr001

Users can feel spatially present even if the virtual environment presents impossible or unlikely sceneries (Picture taken from the Tetris Apocalypse VR environment; see https://www.youtube.com/watch?v=aEEHV5k0ep0)

To date, very little research has focused on what happens to people when unrealistic events occur in virtual reality, such as strange social responses from avatars or skies that change color too rapidly. The effects of these kind of events are sometimes mentioned as a side note when discussing interviews with participants, but hardly receive any real attention.

The concept often applied to explain how people deal with unrealistic events in fiction is called ‘suspension of disbelief’. According to this concept, people choose not to pay attention to certain aspects of fiction that hinder their enjoyment of it: they simply ignore the fact that what they see is impossible. However, it seems unlikely that when unrealistic portrayals are encountered, they can be simply ignored. In my own study, I started out from the assumption discussed in literature that people have certain expectations about what is acceptable. When viewing science fiction movies, for instance, no one will be surprised to see an alien life form show up half way through, but if the main character suddenly starts to use magic this will grab the attention of the viewer and prompt him or her to question the likelihood of such an event happening within the setting of the story. But after expectations are revised by the viewer, he or she may accept the level of realism and continue watching, no longer paying attention to the addition of magic to the story world.

So if expectations have something to do with how acceptable a media portrayal is, what does this mean for VR and Presence? Sudden unrealistic portrayals (e.g., an alien suddenly appearing in an otherwise realistic telemedicine environment) could break the feeling of Presence once, but subsequent similar portrayals may not cause any interruption to Presence. This hypothesis was tested in my study. In my experiment, participants were guided through a virtual house. One of the rooms they explored was flipped upside down, so that the furniture was on the ceiling.

Most of the participants had a strong reaction the first time they entered the room and noticed it was upside down. Many asked for confirmation (“Is this room upside down?”), and spent some time viewing the room. But when returning to the room later on, they not only spent less time there, often just glancing at it, or ignoring it altogether. When asked what they thought of the house later, many responded with “this house is really realistic” and only mentioned that the upside down room was not realistic when they saw it again. The more participants experienced the environment as realistic, the more they also felt present in the house. Most importantly, participants indicated that while they felt less present the first the time they visited the upside down room than they did on average, these breaks in presence did not occur the second or third time they entered the room.

This knowledge can help with the design of virtual environments. It means the use of VR does not have to be limited to realistic portrayals, but also that it is not essential for all aspects of a VR to be perfectly realistic.

vr002

The inverted room as displayed in the experiment, VR was displayed using Oculus Rift.

Contact:
Paula den Ouden

VR against aggressive behaviour in the public sector

Aggressive behaviour against employees in the public sector, such as
police officers, tram conductors, and ambulance personnel, is an ongoing
concern worldwide, and in the Netherlands in particular. As an
illustration, the Amsterdam public transport company GVB reports about
500 incidents of aggression against employees (e.g., insulting,
threatening) per year. In general, such confrontations with aggressive
individuals have a very negative impact on employees’ work performance
and wellbeing. Therefore, there is a strong need to better prepare them
for such incidents, by means of dedicated training.

To train employees in their (verbal) aggression de-escalation skills,
the STRESS project aims to develop a simulation-based
training environment for public service workers. The project is a
collaboration between VU, NSCR, TNO, the company IC3D Media, the
Amsterdam public transport company GVB, and the Police Academy. The
training system is based on a 3D Virtual Reality (VR) environment that
is projected on a computer screen. During the training, users are placed
in a virtual scenario in a particular domain (e.g., selling tram
tickets), which involves a dialogue with a virtual character that
suddenly starts behaving aggressively (e.g., insulting the tram driver
because he is late). The users’ task is to de-escalate the aggressive
behaviour of the character by applying the appropriate communication
techniques.

One of the prototype systems that have been developed is a training
environment for tram conductors. This prototype is currently being
evaluated in an experiment involving 30 tram conductors from the GVB.
Recently, this research has received quite some attention from the
media. Some examples can be found here:

Article in De Oplosser (Dutch)

Article in De Volkskrant (Dutch)

Article on Blendl (Dutch)

STRESS_screenshot

The Oculus Rift is not just “another gaming device”

The Oculus Rift is not just “another gaming device” – it promises to initiate the next step in the evolution of media

Among the video gamer it’s talk of the town, and soon it may be among many more people: the Oculus Rift, a new high-quality but low-cost Virtual Reality head-mounted display that may be officially released by end of this year (https://www.oculus.com/rift/). Once a Kickstarter-funded project, Oculus Rift was bought by Facebook in 2014 for 2 billion USD – showing that the device promises not only to change the way of video gaming, but of social media, too. Therefore it’s great that the Network Institute – a competence center for interdisciplinary academic research on the digital world – is involved in virtual reality research, too!

oculusrift

But what’s so special about the Oculus Rift? The best way to understand this is to try it out yourself (the NI Gamecella features currently 3 Oculus Rift headsets, including this version of a Paris’ apartment; https://www.youtube.com/watch?v=kVL9yKwx_aY; another nice examples is this one: https://www.youtube.com/watch?v=AHDQ8hzOGqw). Once the display is properly set up and your eyes a completely covered, you will realize that you feel like if your whole body is spatially transported into the virtual world. You perceive real three-dimensional depth. It feels as if you can actually touch objects. And as if the room really extends both in front of you and behind you – you feel enveloped by the virtual environment. This is a completely new media experience! Something that was mostly only possible for the last 20 years in highly specialized labs with expensive equipment. But the plans are that this high-end VR technology will soon enter the living rooms.

Scholars address the experience that people have when wearing the Oculus Rift as Presence (or spatial presence or telepresence). Presence is simply about “the feeling of being there”. If present, people feel like their body is really located in the virtual room. They know that this is not true, but their senses provide them with a strong feeling as if it was true. People that feel present intuitively duck their head if walking through a small door and feel as if they could really touch objects in the environment. Or they refrain from walking over a steep cliff (https://www.youtube.com/watch?v=v-mK5oNkr-I).

As a communication scholar with a media-psychological perspective, I examine Presence for about 9 years now. I would argue that something fundamentally changes if we feel spatially present in a virtual environment. Really feeling spatially present is different than watching TV or even a 3D movie or a video game on a computer screen. We may understand the space depicted on television, and we may even temporarily forget a bit about our actual surrounding, but we never feel this striking sensory experience of really being located in the perceived TV space. It is as if our brain simulates the space (e.g., as if drawing a map), but doesn’t position our own body inside of it. People probably won’t feel like they could reach out to a can of Coke, if watching an advertisement. But if wearing an Oculus Rift this changes. It feels as if the perceived spatial environment actually is the real environment. People feel like and actually do reach out to a can of Coke if they see it. (These thoughts I addressed together with colleagues in a theoretical model of Spatial Presence formation; see http://dare.ubvu.vu.nl/bitstream/handle/1871/39459/202481.pdf?sequence=1).

What is a virtual object for us? Where is it located? What does it really “mean”? And if we say, “this feels real”, what does this imply? My feeling is – and I plan to further examine this question – that things “come into existence” if we feel spatially present. That is, if we watch a cup of Coke on TV, we just feel it’s a cup of Coke, but we also feel it is not located in our environment, it has no volume, no body, no real existence. This reminds us of what we already know: that the cup of coke is just represented on a TV screen, it does not really exist right here and now in the living room. However, if we perceive the cup of Coke while wearing an Oculus Rift, it will automatically feel as if the cup has a volume, a body, as if it would really “claim space” (like real objects do), and we can also locate it easily in what feels to be our own spatial environment. My impression is that due to this change, things switch from being perceived as mere representations – like objects displayed on a TV screen – to objects that are intuitively perceived to exist in the here and now. In addition, the thing we always know, namely that “these things are not true, they are just illusions” becomes less apparent in our mind, as if we are temporarily blinded by the power of the illusion and forget about it more than if watching TV.

If this assumption is true, it would have great implications. For example, think about encountering virtual people. If you would feel as if they would actually exist (because they are located in the here and now), you would probably treat them differently than if you would be fully aware that they are just symbolic representations or “pixels on a screen”. You would probably treat them with respect, similar to the way you treat real people. In my own research I have shown that video game users felt guilty if shooting innocent virtual people, although they of course knew that these were just “pixels on the screen”. I assume that the feeling of guilt most be much stronger if shooting innocent virtual people in an Oculus Rift environment. Simply, because it felt as if these people actually existed. This is something we like to study in the NI Gamecella.

Want to learn more or reply to this comment?

Please contact t.hartmann@vu.nl or visit my website: http://www.fsw.vu.nl/en/departments/communication-science/staff/hartmann/index.asp