|
||||||||||
|
|
Integrative Sensorimotor CapabilitiesObservations on Sensorimotor Integration of Eye-Head-Hand Coordination:
Virtual "Reality" Environments: Foundations
Example: Use of Virtual Environments in Rehabilitation
Example: Efforts at Multimodal Accessibility Standards (further developed in Module 4)
Integrative Sensorimotor CapabilitiesWe have now covered a host of sensory and motor subsystems. In many cases, especially in the previous two sections on positioning and touching, we have been implicitly considering sensorimotor capabilities in that the "sensory" and "motor" subsystems are inherently intertwined. One can go much farther down this path, but for the purposes of this course, here we simply focus on a few examples that relate strongly to rehabilitation: eye-head-hand coordination, some brief comments on motor learning and on extended physiological proprioception, and a brief review of two applications: the "multimodal considerations" in accessibility standards (i.e., providing alternative sensory or motor modes for access) and virtual reality technologies applied to rehabilitation. Many of these areas can be further developed in final student projects. Observations on Sensorimotor Integration of Eye-Head-Hand Coordination:The topic of sensorimotor integration as related to manual control and eye-head-hand coordination is huge, worthy of its own 3-credit class. Here we focus on the sensorimotor developmental stages for two simple reasons: 1) they help illuminate degrees of sophistication from lower to higher, and 2) with pathology, often lower levels end up being expressed. Here is a summary:
Dynamic Reaction and Movement TimesBoth movement and behavioral scientists often use stimulus-response tests to evaluate behavior. Key metrics are timing between observable events, with these often given names such as "reaction time" and "movement time," and some type of "quality of movement" metric. The latter are more difficult to quantify. Reaction times depend on the "state" of the subject. Here are some typical time windows (for healthy young adults):
Movement times depend on muscles and inertial dynamics (takes longer to move a bigger mass):
Factors affecting reaction and movement times:
Note that many of these factors can add to the reaction and movement times provided previously. Speed-Accuracy Tradeoff (Fitt's Law) and Predictive CapabilityThis well-tested concept says that there is a tradeoff between speed (and movement time) of a movement and the accuracy. Simply put, Fitt's Law states that fast goal-directed movements that must be accurate are made slower (take more time). For instance, when the "window" defining a successful endpoint is larger, the movement can be made faster. Furthermore, movements that are of greater amplitude take more time. There is a famous logarithmic relation that helps capture this phenomena and has been fine-tuned by many research teams. But the bottom line is the basic observations, which should make intuitive sense to you. (Note that for certain well-practised skills this wouldn't apply.) The above framework assumes that the subject cannot predict what will happen. The human neuromotor system is proactively predictive by nature. Humans are always trying to predict, especially using vision and the remarkable capabilities of the visual cortex. This is useful, for instance, in driving a car or flying a plane or playing a video game. Interestingly, in terms of capability to predict under time pressure, the hand is better (faster and more accurate) than the head, which is better than the eye. Yet even the eye is a very good predictive tracking system. Also, because its saccadic eye movements are completed so quickly (see below) and the smooth pursuit system is always trying to predict velocities, even if it is worse at predicting it might still arrive at a visual destination of interest first. "Virtual" Spatial Mapping Capability: Conceptual Eye-hand CoordinationHave you ever thought of the skill it takes to use a hand pointer (e.g., mouse) to move a cursor image on a monitor screen? It's a non-trival mapping. Humans are good that developing such skills, and often such skills are a critical part of rehabilitation simply because they are needed in everyday life. Here we just make a few comments:
"Extended Physiological Proprioception" (EPP)This concept relates to the capacity to encompass a technology as if it were an extension of the body. A key requirement is that there be bi-causal power transfer (force & velocity/position across interface). This is a skill, and practice is needed to develop the skill. Often, indeed most of the time, an EPP skill developed for one hand (e.g., dominant) does not transfer to the other without a whole new practice paradigm. Examples include body-powered upper extremity prosthesis and commercial products such as a a tennis racquet, a golf club, a baseball bat, a pencil, a brush or a screw driver. Virtual "Reality" Environments: FoundationsThe integration of various sensors, displays, and computers can allow users to interact with artificial computer environments in a reasonably natural and synergistic manner. As defined here, a virtual environment (VE) infrastructure includes a graphics workstation and software (to coordinate the actions of the various sensors, process user inputs, and define the nature of the user's virtual world), plus sensors (to measure real-time position/orientation of any prescribed body segment). Typical sensors include a head tracking system (so that display changes as the head rotates), a wrist tracking system (so that the virtual environment moves with the arm), and a glove (to estimate hand movements). [A simpler paradigm might be a monitor and joystick.] The conventional cornerstones of virtual reality (VR) are interactivity and immersion. However, in my opinion, the key aim for rehabilitation applications needn’t be full sensory immersion, which has proven difficult to achieve. For instance, most people will eventually feel nauseous after being embedded in a full VR system for a while, even for the high-tech VR systems. This is because the sensory perspective is not quite real, with slight mismatches between "real" and "virtual" eye-head-hand sensory information, with the most challenged sensorimotor subsystem likely the VOR. Rather, immersion is an attentional state of mind, and in my opinion a person can be remarkably immersed in an movie or TV show, or even a good book. The aim of VR is to integrate these technologies into a coherent unit that promotes effective interaction and has embedded mechanisms for data collection. In my opinion systems typically needn't employ head tracking or a full field of view. A related concept is telepresence – the use of telecommunications to allow a remote operator to be “present” (e.g., to manipulate objects, televisits). The perceptual aspect of telepresence – a person’s sense of being at another place – ties to VR. From a "systems" perspective, a potential advantage of a VE is that parameters within the interface system can be mathematically manipulated. This can, in principle, be of advantage for both assessment of performance and interactive therapy. For instance, a task can be broken up or remapped into something at which the client can actually succeed, rather than be frustrated. And performance measures can in principle be embedded within a task. VE's can also break out communication and manipulation parts of a task Example: Virtual Reality Applications in RehabilitationAs suggested in the previous section, "virtual reality" (VR) uses a "virtual environment" (VE) to created an interface that, if appropriately customized to the client's abilities and therapeutic needs, can make “success” more achievable and “therapy” more fun. The target for such therapeutic intervention is typically on individuals with compromised motor and/or cognitive function who seek to reacquire motor or interpersonal skills to function in the real world. The therapeutic value of such a rehabilitation regimen rests whether it does in fact help accelerate recovery and the reacquisition/relearning of certain skill sets. A practical advantage is that it can enable clients to practice in a safer environment. Another potential advantage of the VE based therapy approach is the ability to embed real-time diagnostics into the therapeutic simulations. For rehabilitation applications, there are also potential concerns. These start with the fact that the ultimate aim of the rehabilitative process is for clients to function effectively in the real (not virtual) world. As we've seen previously, this can mean maximizing "independence" and "participation" and minimizing impairment and disability. Attaining and maintaining postures is somewhat subconscious for most of us. VR systems can cause problems of sensory conflict and motion sickness. Interestingly, virtual reality applications such as using head-mounted displays that are attempting full immersion are an example (e.g., VOR being off from the visual environment during head movements, causing dizziness and a need for smooth pursuit compensatory movements; eye and hand sensing conflict within the brain, because both are sensing artificial worlds that are likely not changing instantaneously and may having spatial mapping errors). Still another is that therapy can be technology-centered, i.e. is designed around the available interface technologies (e.g., glove) rather than what might be the optimum therapeutic needs. Finally, there is the issue of cost and technology access. VE’s have been used for orthopedic and neuromotor therapy, for addressing psychotherapy, as a cuing device for individuals with Parkinson’s akinesia, for autism, neurocognitive assessment, to train ataxic individuals to use proprioceptive rather than visual feedback to maintain balance or enhance gait, and for driving simulation. However, to date, despite the hype few VR applications have made it into common practice. The onus is on the VR advocate to make the case that therapy in a virtual world has some advantage over that in the real world (which is where individuals must ultimately function). Some examples:
Example: Interfaces for Multimodal AccessibilityMultimodal sensory interfaces display information content (and perhaps controls) in more than one sensory mode. It can relates to accessibility and reduction of social exclusion, and can help compensate for sensory impairment. Clearly it ties to the focus of Module 2, and this section reviews some of the ongoing national and international activities in this area. The W3C is the international body involved in setting up web standards. As seen from the W3C's Multimodal Interaction Activity and Web Accessibility Initiative (part of a collection of Standards/Guidelines for the evolving Web), they have been quite involved in considering accessibility (WAI) and multimodal interaction. The W3C Multimodal Interaction working group aims to develop specifications to enable access to the Web using multiple modes of interaction, such as speech, pen, keypad, bitmapped displays etc. This activity is part of a suite of specifications for multimodal systems, and provides details of a platform and language neutral software interface that enables applications to dynamically determine and respond to changes in device capabilities, device configuration, user preferences and environmental conditions, such as low battery alerts or loss of network connectivity. For more information try out some of the following links:
Europe's ETSI EG 202 191: Human Factors: Multimodal Interaction, Communication and Navigation Guidelines addresses multimodality to reduce social exclusion, improve accessibility, and human factors recommendations for multimodal interfaces. This is a great educational document for understanding the possibility for multimodal interfaces and human factors design principles and processes. The V2 Standard for "Universal Remote Consoles" (URCs) for "Target" devices and services, such as home appliances directly relates to multimodal interfaces, including natural language communication. It is being developed by the International Committee for Information Technology Standards (INCITS) of the American National Standards Institute (ANSI). V2 allows manufacturers to install an "interface portal" on their product, which then allows customers to use a wide variety of remote controls with the product, usually through wireless or network technology. These "interface portals" give a user access to all of the controls and displays for the product. V2 describes a standard way of creating these portals so that they can be used by a wide variety of URCs. Because both displays and controls are available through the V2 portal, the URC is able to tell the status of the device as well as the available commands. It also may allow more sophisticated controls, such as natural language. The V2 Standard is based on existing standards for connections and transport. In special interest are V2 applications that build on implementation on the popular new UPnP (Universal Plug and Play) standard. TheV2 simulation environment from Trace Center can be used to help understand how URCs and Targets can communicate. The interface documents at the Target make extensive use of XML and W3C tools.
|
|
|
|
