Lost in Space

Experimental Setup

• two blocks with different hand models
• each trial consisted of three stages
  • pre-localization: self and external
  • bimanual object interaction for 40 seconds; used to introduce visual offset
  • post-localization: self and external
• after each trial, participants were asked to move their hands out of the tracking range, after this, the visual offset was removed
• at the end, the igroup presence questionnaire (IPQ) was administered

Experimental Setup: VR Setup

• Oculus Rift© DK2 stereoscopic head-mounted display (Oculus VR LLC, Menlo Park, California)
• Leap Motion© near-infrared sensor (Leap Motion Inc, San Francisco, California, SDK version 2.3.1)
• Unity® engine 5.2.2 using the C# API interface
• hand models and flower obtained from the Leap Unity assets (not available any more)

Experimental Setup: VR Setup Overview

Experimental Setup: Hand Models

Experimental Setup: Task Space I

• the asymetrical layout of the task space was due to the sensor
• the tracking range is more broad than deep
• hence we put the basket at the outer right

Experimental Setup: Task Space II

Experimental Setup: Tasks

Experimental Setup: Object Interaction I

• this sample shows the object interaction, the transparent overlay shows the actual hand position
• please note that the participants only saw the solid, shifted hand
• the shift only commenced while the hands were moving and the flower was grasped, it took about ten seconds of interaction to introduce the full shift
• in general participants complied to the task, overall the mean number of picked petals was 4.5 (SD = 1.4) that is one flower per trial
• please note that this is a lower bound, since these are only those petals that landed in the basket

Experimental Setup: Object Interaction II

Experimental Setup: Localization I

• this sample shows the localization which had to be performed before and after the interaction phase
• due to the error checks, the task was quite demanding
• hence, participants were given quite a lot of time to practice this task (about 10 minutes)

Experimental Setup: Localization II

Experimental Setup: Factors

Experimental Setup: Offset Conditions

Experimental Setup: Measures Part I

Experimental Setup: Measures Part II

• the next slide provides an interactive overview of the measures
• you may need to refresh your browser ([F5]), if it still does not work, your browser settings do not allow to load local javascript
• the yellow hands indicate the physical hands above the sensor, the red hands indicate the visual hands in the VR
• with the buttons on the right you can switch between the dependent measures
• with the 'x-offset' slider you can manipulate the extend of the left / right offset; please note that this will make the yellow (actual) hands move, since the visual hands had to be kept in the center to allow a succesfull interaction with the flower
• the 'visual capture' slider is basically a scaling factor that exaggerates or attenuates the offset effect

Experimental Setup: Measures Part III

Participants, Procedure and Data Preparation

• Participants
  • 33 students, 14 right-handed, 11 females
  • one left-handed participant, no glasses
  • Mean age: 21.7, SD: 2.5
• 2 × 12 trials
  • two blocks, localization with visible, or invisible hands, order balanced over participants
  • 12 trials each: Four offset conditions, each repeated three times
  • self- and external localization before and after 40 seconds of object interaction
• Data Preparation
  • per localization, 50 data samples of the palms, index finger, and thumb tips were recorded
  • data were averaged into a single 2D [x,z]-centroid to compute dependent measures
  • signs were assigned depending on offset condition, hence measures remained centred around zero

Results: Data Analysis

• Drift, Disparity, and Discrepancy
  • Linear mixed effect model analysis using R statistical software
  • model identification with fixed effects (and interactions) for
    • hand visibility (visible vs. invisible)
    • localization reference (self vs. external)
    • visual offset (left, right, forward right, forward left)
  • maximal, converging random effect structure
  • checks against zero at the level of the reference × visibility interaction
• IPQ Scores
  • results were compared with reference data from the IPQ database
  • t-Tests to check for significant improvement on the different scales

Results: Model Identification

Results: Visualization

• the next slide provides an interactive overview of the data on the aggregated over the different offset conditions
• you may need to refresh your browser ([F5]), if it still does not work, your browser settings do not allow to load local javascript
• on the y axis you can see the dependent measure
• the x axis displays the four reference × visibility combinations
• you can switch between boxplots and individual data via the 'bar' and 'value' buttons
• you can change between the dependent measure with the lower buttons
• all means differ from zero, except for the discrepancy value in case of invisible hands and pointing towards oneself

Results: Reference × Visibility

Results: IPQ

Results: Summary

• visibility affected all measures
• reference only affected disparity and discrepancy
• more pronounced effects in case of visible hands and external localization
• type of offset only affected discrepancy
• measures differed from zero in all but one condition

Main Results

Results on disparity and discrepancy show that adaptation was not restricted to hand space