"The sensation of looking around seamlessly is an important fictionalization created by the brain."
Making a peanut butter and jelly sandwich. Walking on the sidewalk. Picking up trash. Avoiding 6-foot tall virtual purple rectangles.
Learning how humans complete these simple tasks and millions of other ordinary activities is fundamental to understanding how the brain functions, says professor of computer sciences Dana Ballard.
Ballard and psychology professor Mary Hayhoe, principal investigators in the Virtual Reality Laboratory, are decrypting what Ballard calls “micro operations,” distinct sets of instructions a person’s brain executes when performing simple tasks.
The brain can be understood as a kind of computer, says Ballard, capable of running several simultaneous micro operations. These operations live in the cortex as “hard storage,” like millions of burned CDs ready to be called into action at anytime.
Ballard’s wants to know how many operations can be performed simultaneously by the brain and how the brain switches focus from one thing to another.
Guided by the old adage, “The eyes are the windows to the soul,” Ballard focuses on human vision. By studying how people react to images brought into the brain through the eyes, he can glean insight into the choices the brain is making.
People gather visual input in distinct series of eye fixations, called “cicadic jumps.” The brain rapidly strings the individual still images together like a motion picture.
“You don’t get information from the world for free. You have to strobe it in,” Ballard said. “The sensation of looking around seamlessly is an important fictionalization created by the brain.”
To figure out how the brain strings images together, Ballard and Hayhoe turn to a virtual world. They were the first to mount eye trackers (devices that monitor eye movements) inside head-mounted virtual reality goggles. People equipped with the goggles and touch and motion sensors are placed in virtual 3-D environments, like a room with a table or a field with a sidewalk, and carry out instructions from the researchers.
The virtual environment allows for an infinite number of scenarios, ranging from picking up color-changing cubes and placing them on a conveyor belt to driving a car. The researchers record precisely the visual input people immersed in the virtual world receive as they perform tasks.
Ballard compares the data to computer programs he’s written to simulate thinking processes in humans. His ultimate goal is to lay the groundwork for a comprehensive psychological model of the human brain.
Written by: Patrick Brendel
Photo: Wyatt McSpadden
Comments