top of page

How do we keep track of things?

  • I have conducted many studies examining how we track objects as they move around, change their appearance, and disappear & then reappear.  The general goal of these studies was to learn about the cognitive mechanisms underlying tracking & the information that these mechanisms rely on. 

  • These findings have important implications for user interface (UI) design especially since animations are so ubiquitous in UI.  Hence, I describe here an overview of my object tracking studies and bridge this to applications in UI design.

Red Light Art
Learning about our ability to track information by disrupting & altering what we track
​

  • A useful strategy to learn how we track objects is to interrupt tracking tasks & see how we recover from these interruptions. Using this strategy I have addressed a range of questions:
     

  • What type of information do we use when tracking? What is the role of attention, memory & eye-movements on our ability to track objects?   What is the role of external cues such as an arrow or some signal turning on suddenly?  When tracking is interrupted, do we extrapolate where an object will be in the future based on what we observed in the past?  Does our memory serve us to get us back on track? Or do we need cues to remind us of where we left off? 
     

  • By studying such questions we have gained important insights into what information, and what cognitive strategies, we use to maintain our ability to track things despite the many interruptions that occur while performing these tasks.
     

  • Among the key insights we have learned:

  1. ​Tracking is guided by a number of factors including abrupt cues, landmarks, eye-movements & our memory of the positions of recently tracked items. 

  2. Some of these factors are not obvious & are revealed through controlled experiments where tracking is disrupted & information is altered during the interruption.

  3. A counterintuitive example is how the positions of recently viewed objects stick in our mind so that when they disappear, we (behave as though) we expect the objects to reappear in the same position where we last saw them. Keep in mind, this happens when objects were moving prior to their disappearance where commonsense suggests that movement of objects should continue when those objects temporarily disappear from view -- as often happens when one object blocks another.

  4. But various cues  such as abrupt signals, or landmarks, do guide interrupted tracking to make it appear as though we naturally  predict where tracked objects will reappear after being interrupted.

  5. Without cues, we rely on our memory of previously viewed object positions, and surprisingly (when tracking moving objects), we behave as though reappearing objects will be in the same position where they were last scene.  

  6. These finding have important implications for using animation in UIs. To help users keep track of targeted information that may move or be interrupted by irrelevant information,  designers need to be strategic to achieve an intended effect: 1) Use appropriate cues to keep users on track of relevant information so that they can accomplish the task that they set out to do.  2) When a designer's intent is to pull a user's attention away from the task at hand, only do so in an unobtrusive way & provide appropriate landmark cues so that users can easily get back on track.

O
What type of information, and cognitive mechanisms, do we use when tracking?

  • We have the ability to track objects by tracking various features such as their color, size, or shape. But tracking these features is not necessary & sometimes not possible when tracking objects that are identical to non-tracked objects.    
     

  • To illustrate, consider the following demonstration where all the items are blue. Try tracking the 4 items that blink on & off at the beginning of the video:

 

  • Most people are able to track the 4 target items even though they are the same as the non-target items.  This illustrates that we do not need to track particular features such as color to successfully track items. In fact, we may be better off not tracking particular object features  since in the real world, these change all the time. When objects move around, they project images into our eyes that are constantly changing such as with an object moving away from us, (in the image) it shrinks in size or diminishes in clarity & saturation.  
     

  • One compelling alternative to tracking features of an object (as proposed by Zenon Pylyshyn) is to use "mental pointers" which link to objects as they move around. But surely such a pointer needs some sort of information about an object to make a connection with that object.  The most likely candidate is object position.  To test this we introduce interruptions to the tracking task & manipulate the speed, & hence object position as it changes over time. 

How do interruptions affect our ability to track information & how do we get back on track?
​

  • In the following demo, I have added white rectangular blocks which occlude the blue items as they move around the display.   Try playing the demo to see if the task seems more difficult than before.  (Note:  Rectangles may not appear on mobile devices due to Wix bug)

  • Here, items continue to move as in the previous demonstration.  If your visual system assumes that items continue to move along a similar path, tracking should be comparable to when occluding rectangles are not present. But our research shows this is not true (when tracking more than one item).  Tracking is more difficult, but not because there are simply distracting squares on the display.  More difficult tracking occurs because the visual system does not "expect" items to continue to move when they disappear behind other objects.  We demonstrate this by altering the speed of items when they disappear.  And it turns out, surprisingly, people are best at tracking when items move at the slowest speeds when they are hidden (& then return to their original faster speed when visible).   And yes, the very best performance occurs when objects halt while hidden.  This is evidence that people rely on their memory of where they last saw the moving objects, rather than assuming that object motion is extrapolated during periods of disappearance.

  • More evidence on this surprising finding is forthcoming along with demonstrations of how we can use cues & other information to predict the future position of moving objects that temporarily disappear. In the meantime, you can learn about this in my work below where many of these studies use eye-blinks as a natural form of interruption & perform blink-contingent changes to items so that no outside information can signal that a change has occurred. These eye-blink induced changes help reveal how our tracking system behaves without external cues.  The bottom-line: without external information, tracking is suspended to our memory of what was last seen.

  • Aks, D.J., Naqvi, M., Zish, K., Planer, R., Kevin Zish, Pylyshyn, Z. (2013). Role of attention, eye‐movements, and landmarks in tracking an occluded object.  Journal of Vision, 13(9): 1278. https://doi.org/10.1167/13.9.1278
     

  • Naqvi, M.,  Zish, K., Planer, R., Aks, D.J. &  Pylyshyn (2012). Effect of Occlusion and Landmarks on single object tracking during disrupted viewing. Journal of Vision, 12(9): 550, 1404. https://doi.org/10.1167/12.9.550
     

  • Aks, D.J. Kourtev, H.,  Haladjian, H., Pylyshyn, Z., & Shen, J (May, 2011b). Blink-Induced-Blindness (BIB) in Multiple-Object-Tracking (MOT) shows when vision does not extrapolate. http://www.visionsciences.org/demo_night.html) [Abstr]
     

  • Alley, L. Rathakrishnan, V., Harman, C., Kourtev, H., Haladjian, H., Aks, D.J., Pylyshyn, Z. (2011c). Tracking objects and tracking our eyes during disrupted viewing. Journal of Vision, 11(11), 277. https://doi.org/10.1167/11.11.277
     

  • Aks, D.J., Haladjian, H., Kosmides, A., Annamraju, S., Kourtev, H. & Pylyshyn, Z. (2010). Blink-induced masking and its effect on Multiple Object Tracking: It’s easier to track those that stop during interrupted viewing. Journal of Vision, 10(7),301. https://doi.org/10.1167/10.7.301
     

  • Aks, D.J., Haladjian, H., Pylyshyn, Z., & Hakkinen, A. (2009b). Eye-blinks and Tracking:  Multiple Object Tracking with blink-contingent scene changes. Journal of Vision, 9(8):249. https://doi.org/10.1167/9.8.249 

bottom of page