Sounds, layouts and Custom Interactions – Airport Security

In our experience the multisensory nature of Immersive Sims greatly helps with learner engagement and to increase their feeling of immersion within the learning context. Where used well, it may also generate richer learning stimuli that learners can encode and bind into memory more elaborately and thus retrieve more easily.

A key element of this is sound. Ambient sound can be used to found the scene in reality, whether it is the noise of traffic or the hum of machines in a factory. It can also be used to stimulate different moods, be it excitement or calm or horror. Then we have scene specific sounds such as a telephone ringing and of course, human communication.

Having spent a lot of time recently in airports and the security checks I thought that this would be a good context in which to explore sounds. At the same time we can have a look at changing default layouts and building custom interactions.

Here we are going to have the beginning of a sim scenario. There is a line of holidaymakers going through security, a guy on the scanner and two officials managing people coming through the metal detector. We’ll start with a view of the scene, it will be noisy and busy, with airline announcements and people moving. The girl at the front of the queue will walk through the metal detector which will set off the alarm beeper. At the same time her bag is moving through the scanner. The user is then provided with a custom display through which they can change camera views and make actions – the first of which is to choose which one of the security officials should search the girl. That’s it. You can decide what happens after that yourselves.

To get started I have loaded in some free to use sounds and icons from the web. These are not to be used in commercial projects without their owners prior consent. I went to http://www.freesound.org and picked up some background airport sounds. I have used these as two sounds in the sim – AirportAmbient and Beep.

I used icons for buttons which I got at http://www.websiteicons.net/ – this site is terrific for icons.

Right lets get started.

Setting up the Scene

 Using the Airport environment I have set up a scene around the security check. There are 4 people in line, plus a guy operating the scanner and two security officials. I have a set of random animations running on these characters.

There are 3 paths in the scene. One will take a character through the metal detector. The other two will lead either to the male or female security official.

There are a few markers in the scene. 3 are on the conveyor belt and will be used to move objects along the conveyor belt through the scanner. 4 are used for the camera fly through. The remaining marker is positioned in the middle of the metal detector. I use this to detect when a character passes over this point. If they do then we can set off the beeping alarm if we want to.

We then build in some simple interactivity using:

·         Communication Interaction

·         Custom Interaction

·         Play Sound nodes

My storyboard looks like this on the scene flow canvas:

In the SetupScene I change the camera; remove the inventory icon from the screen by using the HUD control node; and set the values for a few counters using the Counter Control node (I don’t actually use all of these in the demo but they are in there to extend the scenario).

Then we move into the start action:

In Move Bag group – the Bag object is moved to markers along the conveyor belt using the Manipulate Object node. A Timer node is used to pause this movement in the middle of the conveyor to suggest that the computer operator is looking at a scan of the bag. I use the Play Sound node to play a sound file at this point called AirportAmbient – this is a nice audio sequence that has a lot of background crowd noise and airport announcements. I downloaded it as wav file from http://www.freesound.org. I then converted it to OGG file type using a free program called Switch Sound File Converter – this also comes with an editing package.  OGG is the filetype we use for sounds in Thinking Worlds. Its open source and it cruches sounds down nicely. The original wav file was about 12MB but as an OGG it is 2MB. I always bear this in mind for web delivery. We try and keep Immersive Sims in packages of 4MB or under. If you include a lot of sound then this can be tricky on filesize.

The Play Sound node is easy to setup. I can give it an identifying name to remind myself what I am using it for. Then in the white boxes on the right of the node I can fill in values:

Select the sound file to open. Clicking here will open another window – Edit Sound Property – see below. Using this window I can scroll through the sounds in my library or I can click on Add Sound and load more OGG files into the library. When I am done I click on OK and the sound file name will appear in the appropriate white box in the node.

I can then change the volume setting for the sound file and finally select whether I want the sound to loop when it has finished playing. In this case I am selecting a long playing sound file that is going to provide background audio throughout the sim; so I set loop to ON.

In Openingcam group – I use the Manipulate Object node to move a camera on a journey through different markers (1-4).

When the camera reaches Marker 1 I then want to activate two different processes:

Firstly I want the character Girl1 to follow the spline path though the metal detector to the other side of the security barrier. I use a Follow Spline node to do this which asks me simply to select the path that is to be followed and which character will follow the path.

Secondly, I want to know exactly when Girl1 reaches the marker in the middle of the metal detector. So I set a Distance Watch node to activate at the same time. This then watches for when Girl1 is 1 unit (approximately yard) from the marker called “InsideScanner”. When this happens I want to play an alarm sound using the Play Sound node.

 

I populate the Play Sound node as before. This time I set Loop to OFF as I only want this sound to play once.

In the Animate Characters group – exist a series of sub groups. All are clones of each other. They use the Play Animation and Random Event nodes to assign random animations to characters. Then to loop around so that the characters are constantly changing animation.

See below for an example of one of the groups:

At the same time I trigger a Communication Message to appear on the screen:

I have edited the layout of the Communication Message by  clicking on the settings button on the main authoring interface

The Scene Settings window will appear. Click on Themes. In the example below, I then choose to edit the Communication Interaction layout. I select one that I have used before called “Searchinghouse”, however you can create a new one, the process is the same:

On the layout editor I can then change the position and texture of the visual components that make up the Communication Interaction (Background, text box, button). In the screenshot below I have selected the background and I am changing its texture to one that I have created.

NOTE – all textures must be saved as Direct Draw Surfaces (DDS). I use a free program Paint.NET to do this but there are other free ones if you search on Google. Also, the textures must be saved in the TEXTURES folder root or sub folder in the main Thinking Worlds authoring tool directory. When Thinking Worlds starts it looks inside the TEXTURES folder to find any textures that you are using in your sim.

In this example I have changed the position of the layout elements so that it is near to the bottom of the screen and more central. I have then changed the background texture to a purple type colour and the button image. For the button I downloaded a free icon from http://www.websiteicons.net/.

On completing the Communication Interaction we activate the ‘CameraSwitch Custom Interaction’ group. For this I have created a custom interaction that shows the user two buttons on the screen – a Camera button and an Advance button

On pressing the camera button then the user will switch between different cameras in the world. This loops around until the user clicks on the Arrow to advance. You can make out these different paths below;

The Interaction Control node activates the custom interaction called Interface1. This has two button controls in it. There are two options leading from this interaction. One activates a camera view, the other ends this group. The first step is to create a custom interaction. Return to the main 3D authoring view and select Create Interaction from the top menu icons. From the drop down list select Custom Interaction and give it an identifying name. In the demo I have created the following interaction:

This is a nice simple interaction – I have minimised the background that appears as a default and selected a transparent image for it so that it will not show on screen. Then I have created two buttons by clicking on the toolbox ‘Button’ option and then drawing a square boxes on the main layout canvas to the right.

 

In the Properties tab you can set values for each device:

Close the editor and the interaction will save. You can edit the interaction via the Interaction Editor Form – one of the main menu icons.

Returning to my Story Board we can see how this is utilised on the Scene Flow canvas. Select the Interaction Control node and select the custom interaction named Interface1. The node will take on the properties of this interaction. Then when I drag wires from this node, it realises that I have embedded two buttons in the interaction and provides those options to me – see below

On the Action button being pressed then I drag a wire that ends this group. On the Continue button being pressed then I initialise nodes that change the camera views in a loop that arrives back to the Interface1 interaction Control node. In the sim the user can keep pressing the Continue button (which has a camera icon) and changing the camera view. Then when they press the Action button (an arrow) the group ends and they see two different icons appear. This is from a second custom interaction that shows male and female button icons. When the user pressed one of these then Girl1 will move along a different path to either the male or female security official. If this was a real sim then sending a girl to a male security official would be a mistake as he cannot search her.

That’s where this demo ends. We were thinking about different elements to incorporate to take it further:

·         Random incidents as the queue is processed

·         Aggressive customers

·         Lost boarding cards

·         Reacting to scan images

·         Body searches via cameras

·         Multiple alarms on same person

·         Time based decisions

·         Queue build up

·         Restricted items

·         Progression bars to show scoring – customer happiness; time efficiency; legality bars etc