HOME
 
VISION
 
PROJECTS
 
PUBLICATIONS
 
PEOPLE
 
COURSES
 
CAREERS
 
 Touch Screen Gestures
 Introduction

Touch screen is a screen display that responds and is sensitive to the touch of a finger or stylus and provides direct interaction with information. It is most widely used in automatic teller machine (ATM), interactive displays in retails environment, GPS systems, computers, and recently in mobile phones. Penetration of touch screen mobile phones has made a significant jump especially after the introduction of Apple’s i-phone. Touch screens can be easy to use with almost no training or less training if application is designed appropriately.

There has been development of touch screen in almost all possible areas of mobile application, health care, railway and bus route information application, museum, touch screen devices as city guides, office environment, educational application, industrial control panels, entertainment sector, gaming, computers and laptops, house hold appliances and many more.

In the current scenario, due to technological advancement and ease of use, a wide variety of touch screen devices are available such as mobile touch devices, tablets, interactive kiosks, large wall mounted, free standing and encased touch screen devices. Here, we introduce you to touch screen technologies, finger and touch target consideration, patterns and gestures used to make touch screen interfaces and design consideration of designing touch screen interfaces.

Patterns for touch screen described here, in some cases can also be used for free form interactive gestures. These examples showcase a set of touch gestures, though these are not meant to be the only set of touch gestures. The touch gesture may vary and heavily dependent upon context of use, type of application, touch screen technology etc.

 Tap to open/activate/select

Tapping on a specific area or specific interface objects will open or activate the element. Whenever there is a need for a target object to be activated or a button to pushed to active, tap to open/activate can be used. In addition to open/activate, tap can be used to select as well. Here, one tap selects an element and another tap opens/activates. Tap to select is used where user taps on a target object to select for manipulation. Also, when user needs to select multiple options from list of choices, tap to select can be used.
Tap is a simplest touch gesture and is a better replacement of left mouse click. As mouse over event occurs in pointer based interactions, there is less possibility to trigger the same event on touch based interactions. There are two types of tap to open/activate

Open/Activate/Select on press &Open/Activate/Select on release
Most of the mechanical buttons uses activate on press method, though in case of touch screens, it is better to use open/activate/select on release method as it gives user time to move off the button if he changes his mind, especially if tapping performed to an action that cannot be undone

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Tap to stop

When an action takes longer time to perform or user wants to interrupt during ongoing performance, especially if the ongoing action is displaying a set of screens or contents that user may want to act on it, tap to stop can be used. When a particular action is performed, tapping can be used to stop the performance of the action. e.g. tapping on the screen to stop the scroll Tap to stop can cancel/stop the action, or pause the action depending upon the need

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Double tap

In few handheld devices, double is tap is used to select an object and perform action associated with that object. When user needs to choose from multiple options associated with the selected object, double tap used. 
First tap on the screen selects the object, second tap can be either choosing from multiple options or viewing the details of selected object.

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Drag and drop

Drag and drop is basically placing a finger on the top of an object and moving it along towards a new location, with the help of a finger. Users need a gesture to move object from one place to another, so pushing and pulling an object to different place is a natural gesture to move them. It can be used when object needs to be placed at different locations by user, for scrolling purpose, slider etc.

In drag and drop, object needs to tapped and then pushed or pull to other location. Drag and drop can also be combined with tap to select, where object is selected first and then moved it across the screen. Drag and drop can also be constrained to be moved in a particular direction. E.g moving an element just on left side or right side. Similar way element can be dragged only horizontally, not vertically. Use shorter drags, users may not be able understand longer drags and may cause accidental drops

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Flick

If a set of contents, a screen needs to move away, flick gesture can be used. A slight flick by index finger in any direction can be used to flip through screens, move away contents and help quickly flip through contents

Here, the index finger slightly touches the screen and moves in a straight line in one direction. If a tip of a finger is used, it can be considered as a flick and if the finger pad is used, which relatively takes more time on screen, can be considered as push or pull. Based on the speed of the flick, the movement of the object will continue after the gesture is complete, slowing to a gentle stop

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Touchdown and drag

When contents such as a number of files, folders, text etc. needs to be selected together on a screen, needs to place at different location or perform any action, touchdown and drag is used. It is similar to dragging a mouse around the computer desktop to select target objects.

Index finger is used to touch the screen (where object is not present), slides the finger around the selected area of the screen. A visual feedback is essential to display the selected area on the screen and the selected objects. Further, necessary actions can be performed on the selected objects depending on the objects. E.g. a number of images can be selected to touchdown and drag and together they can be moved to trash.

 
 Touchdown and hold

When a number of actions are associated with the selected object, and needs to be shown to user to perform any of those actions, touchdown and hold gesture is used. It is similar to a right click on mouse based interaction. An index finger is placed on the target object and kept on hold until the information is displayed, a set of actions are showcased to the user. User can choose the appropriate action to complete the task. It is recommended to showcase the set of action close to the fingertip, so action can be chosen easily.

For example, touchdown and hold can be used to quickly edit images on a touch screen interface. User can choose target image, hold it and set of actions such as crop, blur, sharpen etc. can be showcased to the user for image editing.

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Slide to scroll

Many screens have a smaller visible area, specifically mobile devices, where all the contents are not possible to show at one go. Slide to scroll is used when user wants to see the contents that resides outside the visible screen area. For example, viewing a map, list of items that go beyond the visible screen area such as phonebook contact, messages etc.

Though this is similar to drag and drop movement, but specific to sliding a finger on the screen in one direction, that helps scroll the screen or list of the items in one direction. Slide to scroll should not be used to with two fingers to scroll for clarity purpose

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Slide and hold for continuous scroll

One rotation on the scroll can go through the full list of items on the screen. A speed or time threshold can also be set so that after it is reached the scrolling can speed up. Rapid scrolling can be difficult for users to follow, so it is best if the user understands the structure of the list (e.g. list can be alphabetical order). It can also be difficult to precisely stop on a particular item. User often goes back and forth to select the chosen item. 

Scroll bar indicating position can also be helpful; either making is always visible to making it visible whenever necessary. Visual and haptic feedback can be helpful to follow the scrolling of the items. A visual feedback is also helpful such as haptic feedback, visual bounce or sound. Also it is helpful to indicate the user when he reaches to end of the scroll

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Spin to scroll

In addition to slide to scroll, spin to scroll is used to view the contents that are not visible on the screen due to smaller screen size. Typically a user moves his thumb or index finger in a circular motion to scroll. A clockwise gesture scrolls down/forward/right and a counter clockwise gesture scrolls up/backward/left.

Spin to scroll can with used on touch screen as well as on a circular track pad such as ipod. Spin to scroll allows for rapid scrolling with a motion that is more continuous and smoother than up and down or side to side scroll

One rotation on the scroll can go through the full list of items on the screen. A speed or time threshold can also be set so that after it is reached the scrolling can speed up. Rapid scrolling can be difficult for users to follow, so it is best if the user understands the structure of the list (e.g. list can be alphabetical order). It can also be difficult to precisely stop on a particular item. User often goes back and forth to select the chosen item. 

Scroll bar indicating position can also be helpful; either making is always visible to making it visible whenever necessary. Visual and haptic feedback can be helpful to follow the scrolling of the items

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Fling to scroll

In simple words, fling to scroll is a powerful version of flick. Fling to scroll is a strong fling of index finger in any direction moves the screen in that direction. Fling to scroll is used to rapidly go through the large number of screens, contents or text. Flick and fling to scroll are natural interaction to move around the screen contents that mimics the physical world.

The tip of the index finger (though it can be other fingers as well) touches the screen and moves in a straight line in one direction. The rate of the fling is translated into momentum, which is slowed and eventually stopped by simulated friction. A feedback to the gesture is very important. Feedback can be a visual bounce, haptic, sound etc.

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Two fingers to scroll

In addition to slide to scroll, drag and drop, there is another way to go through large amount of contents or screens. two fingers, typically index finger and a middle finger is used to scroll through screens, large amount of text, contents etc. two fingers can be placed anywhere on the screen and then typically moved straight in one direction to go upward/downward/right/left

Two fingers to scroll is generally used in track pad (e.g track pad used in apple macbook series). Two fingers to scroll requires less attention to scroll through contents as it can be placed anywhere on the screen, while slide to scroll requires conscious attention on the slider to scroll. This gesture allows control of number of elements, while retaining control of individual objects via cursor. (In case of track pad). To accommodate this gesture, the surface needs to be sensitive to two touches on the surface.

 Push button to scroll

In addition to drag and drop, flick and fling to scroll, another way of scrolling through large number of information can be a simple push button. A simple push button can be placed at the bottom of the screen, tapping it whenever user needs to scroll. A long time can do the scrolling faster based on time duration the button is pressed.

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Pinch to shrink and spread to enlarge

To check out every detail in a small touch screen devices, users need to zoom in and zoom out the contents to check the details. When a document size, photograph, maps etc. need to be increased or decreased, pinch to shrink and spread to enlarge gesture is used. 

two fingers, either the thumb and index finger of the same hand or index finger of both hands are used and they are brought closer together (pinch to shrink) or further apart (spread to enlarge) while on the top of the object, they decrease and increase the size respectively. Pinch to shrink and spread to enlarge are generally found together

These two gestures require multi-touch capability as system should be able to detect two touch events on the screen. Typically elements those sizes are to be increased and decreased are scaled proportionally. 

The starting points of two fingers are usually equal to 100% of the object's starting size. As fingers move closer together or further apart, the element scales in real time to match the new distance between two fingers at a ratio that designers need to determine based on the sizes of the original elements and the size of the screen. e.g. if the fingers are 50% closer, the element can be 50% of it original size. Designers need to create a point beyond that users can not shrink or enlarge the object

spread to enlarge
 Swipe to delete

In addition to selecting an object and command them to delete through a delete button, swiping the object out of the window can be used to delete the content. When a number of selected objects are to be deleted from a long list of objects, swipe method can be used.

From a list of objects, user can select an object with finger tip and make a gesture to throw it out of the screen will help delete the object. Though there is a possibility, if gesture is not performed accurately, it may lead to drag and drop gesture. So appropriate care needs to be taken to design swipe gesture

Content on this page requires a newer version of Adobe Flash Player.

Get Adobe Flash player

 Ghost fingers

Ghost fingers are visualization of user's fingers when the fingers are out of users' line of sight such as on the other side of a mobile device, inside an object or on the other side of a wall. When the user has to see her fingers to manipulate controls but prevent them from seeing because of the hardware and environment, ghost fingers are used

Through ghost fingers you may be able to overcome the limitation of opaque surfaces and screen coverage, place controls in more ergonomic way and in places that makes more sense. If the back of the device can be used, this gesture allows the use of all 10 fingers at once. Ghost finger gesture requires sensors to detect the positioning of hands. a second touch pad or additional cameras can be added to detect the positioning of the fingers. Shadow like fingers can be used for the visualization for ghost fingers

ghost fingers
 Free form gestures

In touch screen interfaces, freeform gestures can be used to upgrade personalized security measure for users. For example, a personalized freeform gesture can be incorporated to unlock the keypad and further operations for use in a touch based mobile device.

User should be given a freedom to choose the personalized gesture, so they can better remember it. It is very essential to understand the negative side of providing freeform gestures. Proper care should be taken while giving freedom to choose a personalized gesture and there must be a way out if user forgets the gesture. There are endless possibilities to use freeform gesture depending on the context of use, application area and content.