Using Virtual Reality to train bus drivers

In collaboration with the Rotterdam public transport organisation (RET), Allerto and the Artificial Intelligence group (Tibor Bosse & Romy Blankendaal) at the VU, the Network Institute developed a virtual reality training application to help bus drivers cope with aggressive passengers.
The application uses true immersive virtual reality by using VR-goggles and VR-controllers to give the user control over a virtual bus driver. The bus driver will be confronted with passengers who could get aggressive if not treated properly. By making the correct choices, the bus driver can avoid aggression and solve the situation with the potential for violence.
The VR training will be used beside the more traditional classroom and role-playing sessions to make sure bus driver are well instructed in how to handle potentially aggressive passengers.

In the news
De Telegraaf


Anti-scam training app in the news

Over the past two weeks there’s been a lot of media attention about the Babbeltruc App. This tablet app was created by the Network Institute in collaboration with Artificial Intelligence (Tibor Bosse & Laura van der Lubbe) at the VU and the KBO-PCOB (a national elderly organisation) to help the elderly in coping with doorstep scams.
The app confronts the users with several scenarios where a virtual human tries to either gain access to their house or get sensitive information. The user has to choose from several possible answers at each step and is then asked to say aloud the chosen response. A newly developed algorithm (by Daniel Formolo, AI-CS-VU) will determine how assertive the response was spoken.
With all this information the app can advise the user in getting better at avoiding being scammed.

The Babbel Truc app in the news

VU Beta website
VU Centraal
De Telegraaf
Max Vandaag
Telegraaf (video)
Noord Holland nieuws (@ 09:15m)


Fall 2018 Newsletter Tech Labs is out!

After a very busy period here at the Network Institute Tech Labs, we’ve finally been able to create an other newsletter. The newsletter offers some insight in the projects being done at the Tech Labs and gives you all the important news about the Tech Labs.

Please note: The Game Cella’ Lab will move to its temporary location in the Transitorium (KE09) on November 1, 2018!

Find the Newsletter Fall 2018 under the Tech Labs menu or simply download it here!

VR Training App


Training the Elderly

Doorstep scams are scams in which a con artist has a convincing, but fraudulent, story with the purpose of coming into your house and/or stealing money. Often these scams appear at the doorstep, for example when somebody wants to enter your house because they must check the electricity or with similar excuses. However, it also happens that the con artist calls you by phone (telling a story about fraudulent payments, aiming to get banking information for example) or approaches you on the street. Elderly people are often the victims of such doorstep scams, which usually have a high impact on their lives.

Within this project, we are creating a tablet application that can be used to train how to verbally act in such situations. The users both learn what to say, and how to say it, in various scenarios. They will both receive automated feedback on how they dealt with the different situations (what they said) and on the assertiveness of their voice (using an algorithm to analyze vocal signals).

This project is a collaboration between the VU and Unie KBO-PCOB.

VU staff working on this project: Romy Blankendaal, Tibor Bosse, Daniel Formolo, Charlotte Gerritsen, Laura van der Lubbe, Marco Otte


Virtually Bad

This project introduces the concept of “virtual bad guys”: intelligent virtual agents that take a negative or even aggressive stance towards the user. Although they pave the way to various interesting applications, it is hard to create virtual bad guys that are taken seriously by the user, since they are typically unable to apply serious sanctions. To address this issue, this study experimentally investigated the effect of “consequential” agents that are able to physically threaten their human interlocutors. A consequential agent was developed by equipping users with a (non-functioning) device, through which they were made to believe the agent could mildly shock them. Effects on participants’ levels of anxiety and (physiological and self-reported) stress were measured, and the role of presence and perceived believability of the virtual agent was assessed. The consequential agent triggered a stronger physiological stress response than the non-consequential agent, whereas self-reported levels of anxiety and stress did not significantly differ. Furthermore, while presence and believability were substantially associated with users’ stress response, both states did not mediate or explain the effect of a consequential vs. non-consequential agent on stress, as they did not significantly differ between conditions. Implications of these findings and suggestions for follow-up studies on “virtual bad guys” are discussed.

The Tech Labs helped develop the virtual environment, using the custom developed Galvanic Skin response sensor and supplying the VR equipment to run the experiment.

Researchers: Tibor Bosse, Tilo Hartmann, Romy Blankendaal, Nienke Dokter, Marco Otte


Can VR help autistic children?

In this study a virtual environment was build to see if a virtual training agent could help autistic children in trying to relate to other children. The idea was that a virtual environment feels safer and the children will more easily accept instructions and interact with the virtual agents.

The participants would sit in a chair, wear a virtual reality headset (Oculus Rift DK2) and wear a Blood-Volume-Pulse sensor on their index finger to measure heart rate.

The scene was a classroom with two children, boy and girl, and an adult trainer (male). The virtual trainer would ask the virtual children and the participant simple questions. The actual answer of the participant was not important, but the change in heart rate was. By determining a baseline heart rate at the beginning of each trial, the software checked the actual heart rate at several points in the scenario. If the heart rate was above a certain threshold, the virtual trainer would first try to calm the participant down before continuing the conversation.

Reseachers: Laura Helgering, Michel Klein


Network Institute VR at IDFA 2017

Take a look at this short impression of the VR display at the IDFA this year.


Tech Labs VR featured in Discovery Channel documentary

As most of you have probably seen in the Tech Labs Newsletters, the Tech Labs have been working on a virtual neighborhood for studying burglary. These studies are a collaboration between the NSCR (Jean-Louis van Gelder), the University of Porstmouth (Claire Nee) and the Network Institute’s Tech Labs (Marco Otte). The previous version of the Virtual Environment was used in several studies including one carried out in British prisons using actual criminals.

Discovery Channel made a short documentary of this study which features our virtual neighborhood.

At the moment we are working on the second version of the neighborhood featuring many more houses with different features offering an even more realistic environment to use in studies.


Video on Virtual Reality in Academic Research

Erik Verwiel, master student at the UvA, made a short video about the role of immersive Virtual Reality in academic research at the moment.


Sun Joo (Grace) Ahn (Stanford) talks about Immersive VR Sept 15

As part of the unfolding activities of Network Iinstitute’s VR group, we would be excited if you like to save the date for an upcoming presentation by Sun Joo (Grace) Ahn, former Stanford Ph.D. of Jeremy Bailenson. Grace will talk about “Persuasive Technologies: Using Immersive Virtual Environments to Promote Attitude and Behavior Change.”
Grace will visit the VU Communication Science department Tuesday September 15. Her presentation is scheduled for 15:30 – 17:00. The exact room/location at the VU will be announced shortly!
If you like to meet with Grace 1:1 on Tuesday to talk about research, please let me know so that I can bundle and forward requests to her.
Abstract of presentation: 
“Immersive virtual environments offer several novel affordances that allow users to interact with and experience mediated events that were difficult or impossible with traditional media. The effects of these virtual experiences transfer into the physical world to transform attitudes and behavior, shifting our traditional understanding of communication patterns. This talk addresses how immersive virtual environments may be used as tools of persuasion in the realms of health and consumer behavior, discussing the tripartite interactions between reality, virtual reality, and humans.
By the way: Also note the upcoming Dutch VR days in Amsterdam (you can pre-register now)!  – an excellent opportunity to try out VR technology, demos and applications.

Are Aggressive Agents as Scary as Aggressive Humans?

This question has recently been investigated by a team of researchers from the VU Computer Science department and the NSCR. The results have been published at the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), including the two Virtual Reaklity Group members Tibor Bosse and Jeroen de Man ( see for full article A summary of the article is provided below:

Most research into intelligent virtual agents focuses on agents with a positive stance towards the user. Nevertheless, the development of virtual agents that show aggressive behavior may also be interesting for a range of application domains, varying from aggression de-escalation training to anti-bullying education. However, ensuring that such aggressive agents achieve the desired effect is not easy, as they need to be believable in a number of aspects. In particular, they need to bring their human conversation partners into a serious state of anxiety. To investigate to what extent this can be achieved using state-of-the-art virtual agent technology, an experiment was performed in which the impact of an aggressive virtual agent was compared with that of an aggressive human. By randomly distributing a group of 28 participants over two conditions (virtual and human) and measuring their physiological and subjective emotional state before and after an aggressive outburst of their conversation partner, the difference between virtual and human aggression was studied. The results point out that both types of aggression induced a substantial stress response, but that the impact of the human aggression was higher than that of the virtual aggression.


Figure note: This figure depicts the dynamics of the electrodermal activity over time in microSiemens during the relevant part of the experiment, averaged over all participants in each condition. The horizontal axis denotes a period of 2 minutes, i.e., 1 minute before the start of the aggressive outburst and 1 minute after it. The vertical line indicates the moment the outburst started.


Feeling present in an impossible Virtual Reality

For most people, the term Virtual Reality will bring to mind visions of imaginary worlds, complicated devices and gaming teens, but nowadays, VR is increasingly being used for purposes other than gaming. Surgeons, for example, use it to operate using a robot. It is also used in training and education in the military, sports, healthcare and engineering. These tasks benefit tremendously from VR, because the environment engages multiple senses and creates a feeling of being located in a virtual world. This feeling is called Presence.

We know certain things are disruptive to this sense of Presence – the equipment that is used for example: head mounted displays are often heavy, and sometimes glasses steam up or get smudges on them. Noises from the physical environment, or bumping into a table can disturb the feeling of Presence. These small disturbances can remind a user that he or she is using a VR system, and is not actually in the physical environment. Technical problems, such as sudden changes in frame rate or unresponsive controls, can also cause a person to lose the feeling of location in the virtual world.


Users can feel spatially present even if the virtual environment presents impossible or unlikely sceneries (Picture taken from the Tetris Apocalypse VR environment; see

To date, very little research has focused on what happens to people when unrealistic events occur in virtual reality, such as strange social responses from avatars or skies that change color too rapidly. The effects of these kind of events are sometimes mentioned as a side note when discussing interviews with participants, but hardly receive any real attention.

The concept often applied to explain how people deal with unrealistic events in fiction is called ‘suspension of disbelief’. According to this concept, people choose not to pay attention to certain aspects of fiction that hinder their enjoyment of it: they simply ignore the fact that what they see is impossible. However, it seems unlikely that when unrealistic portrayals are encountered, they can be simply ignored. In my own study, I started out from the assumption discussed in literature that people have certain expectations about what is acceptable. When viewing science fiction movies, for instance, no one will be surprised to see an alien life form show up half way through, but if the main character suddenly starts to use magic this will grab the attention of the viewer and prompt him or her to question the likelihood of such an event happening within the setting of the story. But after expectations are revised by the viewer, he or she may accept the level of realism and continue watching, no longer paying attention to the addition of magic to the story world.

So if expectations have something to do with how acceptable a media portrayal is, what does this mean for VR and Presence? Sudden unrealistic portrayals (e.g., an alien suddenly appearing in an otherwise realistic telemedicine environment) could break the feeling of Presence once, but subsequent similar portrayals may not cause any interruption to Presence. This hypothesis was tested in my study. In my experiment, participants were guided through a virtual house. One of the rooms they explored was flipped upside down, so that the furniture was on the ceiling.

Most of the participants had a strong reaction the first time they entered the room and noticed it was upside down. Many asked for confirmation (“Is this room upside down?”), and spent some time viewing the room. But when returning to the room later on, they not only spent less time there, often just glancing at it, or ignoring it altogether. When asked what they thought of the house later, many responded with “this house is really realistic” and only mentioned that the upside down room was not realistic when they saw it again. The more participants experienced the environment as realistic, the more they also felt present in the house. Most importantly, participants indicated that while they felt less present the first the time they visited the upside down room than they did on average, these breaks in presence did not occur the second or third time they entered the room.

This knowledge can help with the design of virtual environments. It means the use of VR does not have to be limited to realistic portrayals, but also that it is not essential for all aspects of a VR to be perfectly realistic.


The inverted room as displayed in the experiment, VR was displayed using Oculus Rift.

Paula den Ouden


VR against aggressive behaviour in the public sector

Aggressive behaviour against employees in the public sector, such as
police officers, tram conductors, and ambulance personnel, is an ongoing
concern worldwide, and in the Netherlands in particular. As an
illustration, the Amsterdam public transport company GVB reports about
500 incidents of aggression against employees (e.g., insulting,
threatening) per year. In general, such confrontations with aggressive
individuals have a very negative impact on employees’ work performance
and wellbeing. Therefore, there is a strong need to better prepare them
for such incidents, by means of dedicated training.

To train employees in their (verbal) aggression de-escalation skills,
the STRESS project aims to develop a simulation-based
training environment for public service workers. The project is a
collaboration between VU, NSCR, TNO, the company IC3D Media, the
Amsterdam public transport company GVB, and the Police Academy. The
training system is based on a 3D Virtual Reality (VR) environment that
is projected on a computer screen. During the training, users are placed
in a virtual scenario in a particular domain (e.g., selling tram
tickets), which involves a dialogue with a virtual character that
suddenly starts behaving aggressively (e.g., insulting the tram driver
because he is late). The users’ task is to de-escalate the aggressive
behaviour of the character by applying the appropriate communication

One of the prototype systems that have been developed is a training
environment for tram conductors. This prototype is currently being
evaluated in an experiment involving 30 tram conductors from the GVB.
Recently, this research has received quite some attention from the
media. Some examples can be found here:

Article in De Oplosser (Dutch)

Article in De Volkskrant (Dutch)

Article on Blendl (Dutch)



Tech Labs en VR Criminal Research

In the Correspondent an interview was posted about criminological research using Virtual Reality.  Jean-Lious van Gelder (NSCR) collaborated with Marco Otte of the Network Institute Tech Labs who created the Virtual Environment to research burglary behaviour (see Tech Labs Newsletter).

To read the full article, click here.

Screen Shot 2015-03-17 at 21.49.37