Human Information Processing: Overview and Intro to Module 2

In the HAAT (human - activity - assistive technology) model presented in Module 1, the human was considered the "intrinsic enabler." In ergonomics, the human is often called the "human operator." In rehabilitation, the human is often called to "patient" or "client." In usability engineering, the human is called the "user." This module provides a perspective that is complementary to these but comes from more of a "systems" perspective: the human as an "information processor."

From this perspective, the human system can be viewed as consisting of three components: sensors, central processing, and effectors:

Sensors:

Input from sensors enables the human system to obtain information from the environment. This is accomplished through sensory neurons, which have nerve endings embedded within tissue that function as sensory transducers for physical phenomena such as light, sound, pressure, temperature and length. Key considerations are sensitivity (minimum detectable level) and range (modes of variation in the phenomena that is measurable). Some assistive technologies are specifically designed to compensate for impaired sensory function (e.g., hearing aids, reading systems).

Specific types of sensors will be covered in subsequent sections within this module.

Effectors:

The observable output of the human system is through neuromusculoskeletal elements that provide movement (e.g., mobility of the human, postural adjustments, manipulation of objects, speech, head and eye movements). The "final common pathway" of the central nervous system (CNS) are the motor neurons, which provide excitation to muscles that serve as the actuators that produce (length-sensitive) forces that then act on skeletal structures to enable postural maintenance and movement (or motor output). Limitations can arise from impairments to neuromusculoskeletal elements (e.g., disease at neuromuscular junction, muscle injury, bone fracture). Effectors often provide motor outputs that can be used to control assistive technology systems (e.g., hand movements acting on joystick to control powered wheelchairs). In some cases such assistive technologies help the human compensate for an impairment to the effectors (e.g., muscle weakness).

Central Processing:

In the "human as information processor" model, located between the sensors and effectors are central processing capabilities that are commonly broken into the functions of perception, cognition, motor planning and movement control, and memory. Interneurons serve as the physical units for this capability, and indeed constitute the vast majority of neurons in the human.

• Perception: Interpretation and assignment of meaning to data received from sensors, through integration of information from sensors and memory.
• Cognition: The processes of problem-solving, decision-making, language, and other aspects of integrative performance based on cognitive function. Cognitive processes include:
  • Problem-solving and decision-making represent a complex phenomena, and a useful sequence of steps bears resemblance to steps common in the engineering analysis and design process: problem recognition, problem definition, goal definition, strategy selection, alternative generation, alternative evaluation and alternative selection and execution.
  • Language is a form of communication that is commonly considered to consist of five basic elements (and associated rules): phonology (sounds used in a certain language and rules for their organization), morphology (rules for organizing the smallest meaningful structural units of language), syntax (rules for organizing words into meaningful utterances), semantics (relationship between words and their meaning), and pragmatics (relationship between language and its users).
• Motor planning and control: the result of integration of sensory, perceptual and cognitive components into a motor pattern that is executed by the effectors: motor planning is a CNS process by which purposeful movements are executed to accomplish a purposeful task; motor learning occurs via practicing motor tasks and using sensory information to adjust central processing units such as neuronal synapses; motor control is the result of integration of sensory, perceptual and cognitive components into motor activity (motor pattern, motor sequence) that is executed by the effectors.
• Memory: Human memory is often considered as having three components:
  • Sensory memory (duration of 1/4 to 2 sec): a short-lasting memory that is used to help provide a temporarily sensory "feel."
  • Short-term memory (20-30 sec): a type of "working" memory that enables recall of items or tasks for up to 20-30 secs, with up to 7 items able to be stored in this active temporary memory (which may come from external or internal sources).
  • Long-term memory (duration is ~ permanent): memory with lasting value, encoded as information within neurocircuitry, that is potentially retrievable but can be "forgotten" (e.g., proactive interference where information learned previously interferes with storage and use of new information, and retroactive interference where new information interferes with retrieval of old information).

Sensorimotor Capabilities, Information Processing and Interface Design

The above figure is one of many that could be shown to represent the inherent ties between human capabilities, human information processing and interface design (in this example for medical device interfaces). The key concept is that the choice of design of an interface brings out different capabilities, and modes of interaction. Here we use the analogy, common in computer programming, to recognize that there are different "layers" of interaction, from low (e.g., sensors, actuators) to high (e.g., understanding a product, learning through an interface).

Physics of the Physical Layer: One-Way and Two-Way Interfaces

Scientifically, interfaces in the physical world can be inherently two-way (e.g., physical contact between a human and device) or one-way (e.g., sound waves triggering a multi-stage mechanical process within the inner ear that cause signal changes in sensory neurons).  Interfaces can be viewed as transmitting power or information, depending on the “impedance” across the interface – if it is reasonably matched, interaction is usually viewed as a two-way power transfer, while if it is dramatically mismatched, one-way information (a “signal”) is transferred. 

Interfaces involving physical contact, for instance between the hand and device, often inherently involve two-way information flow (or bi-directional or bi-causal) unless the device impedance is either really high (e.g., pushing against a wall) or really low (“pushing” against air).  Such interfaces exist in physical space, and have location(s) and orientation(s).  One example is hand tools.  With experience and practice (to develop skill), the body can often “discover” such two-way interfaces to the point where they can become an almost subconscious extension of the body.  This is the “extended physiological proprioception” (EPP) concept that was first proposed by Simpson, which while originally applied to body-powered upper extremity prostheses, also applies to use of products such as a tennis racquet, a pencil, or many machine tools.  We will return to it later in this module. For a successful EPP interface, it turns out that the key criteria are:

1. that a pair of signals (e.g., a force-velocity pair, whose product is mechanical power) cross the interface,
2. that the mechanical behavior on the other side of the interface be predictable so that it can be discoverable by the neuromotor system; and
3. that there be enough richness in the interaction so as to assist this discovery process. 

The result can be wonderful: a technology that is assistive, and functions essentially as a subconscious extension of the body.  Similarly, other types of well-designed physical interfaces are subconsciously forgotten, for instance forgetting that one is sitting in a comfortable chair or wearing a hat.  Thus a well-designed two-way interface becomes subconscious: either it is used as a subconscious extension of self, or its existence is forgotten.  The most effective designs exhibit both features.

One-way “information transfer” interfaces are also common, and indeed more commonly the subject of analysis.  Often these are integrated into a pair so as to include one-way information in both directions.  If information is being transferred from the device to the human without human intervention, we often call it a display.  With this definition, displays can take many forms, the most common of which is visual.  If information is being transferred from the human to the device, we often call it a control.  Controls also can take many forms.

Conceptual Layer Interfaces: Challenges and Opportunities in Rehab and Disability

The human-device interface is much more than physical.  Indeed, often the focus of a usability analysis is on what is here called the conceptual interface – that part of the interface that intangibly connects to the perceptual, cognitive, motivational and adaptive part of the human user.  With technological advances of the past few decades, access barriers such as distance are breaking down, and products and services are starting to access consumers in new, creative ways.  This implies opportunity, and also suggests that companies that don’t adapt may be left behind.  It also suggests the importance of outside-the-box thinking about the interface as a means for interaction within an increasingly flattened world where technical access should improve, and importantly, the (conceptual) interface takes on increasing importance.   

The number of emerging software technologies and standards that are attacking the challenge of supporting more conceptual and “intelligent” interfaces is staggering, and will not be reviewed here. Here we briefly highlight some key trends:

• Decoupling and modularizing of physical and middleware layers in information technology standards, promoting greater product interoperability and leveraging investments, and enabling innovation at the physical layer as well as the conceptual layer.
• Continued growth in computing power (memory times speed), including in smaller devices.
• Networking and interface standards that promote ubiquitous and distributed computing.
• Emergence of a international computing language for expressing structure and concepts (XML) that can used as a base to establish consensus protocols and guidelines that can be used by the technical community interested in accessibility.
• Emergence of the semantic web standard for structuring more natural interaction between human and machine, which among other applications can promote access for persons with cognitive disabilities and performance enhancement for individuals with many forms of activity limitations.
• Advances in intelligent systems, specifically intelligent agents and intelligent assistants.

The personal computer is often called the greatest assistive technology ever, in that it is has a greater impact on the quality of life of persons with disabilities of any other technology, at least in developed countries. This now extends to the Web as a tool for information access and for a huge variety of services. Thus technological developments impact on disability. But at its core, this makes basic understanding of the components of human information processing even more important.

People with disabilities, by definition, have certain deficits in abilities, often especially at the lower "physical" layer (e.g., vision loss) but sometimes at the higher "conceptual" layer (e.g., cognitive abilities). In Module 2 we focus on primarily on the "modes" of the "physical" layer, in each case looking at basic physiology and pathology, at implications on information processing, and at technical standards (e.g., video codecs) that relate to interaction across the mode under study.