HOME
 
VISION
 
PROJECTS
 
DESIGNING FOR MILLIONS
 
PUBLICATIONS
 
PEOPLE
 
COURSES
 
CAREERS
 
 One-handed Interactions for Target Zooming on Rectangular Flexible Mobile Devices

One handed usage of handheld devices is commonly observed in situational impairments, e.g.walking, sitting and standing where the second hand is busy doing real world tasks. Prior research findings show 74% users prefer to use mobile phone with one hand. Hence, the efficiency of one-handed interaction and usage becomes critical in common scenarios of situational impairment where it is extremely difficult for the user to handle the device bi-manually.

The standard handheld device poses several challenges such as limited reachability, reduced accuracy and increased occlusion. These problems are primarily attributed to factors like the constantly increasing size of the device, hand-thumb morphology and interactions that demand the attention of both hands to perform specific tasks. This leads to poor user experiences of interaction with hand-held devices, especially applications that demand target zooming & directional panning, e.g. maps and image gallery that rely heavily on these interactions.

Although extensive research is conducted in improving one-handed interactions on rigid handheld devices, investigations targeting problems of one-handed interactions in flexible handhelds have been rarely explored. These investigations are important as flexible devices offer a new opportunities of input interactions that can be easily performed with one hand, improving the user experience in situational impairment.

This project proposes a method of target zooming (for maps, image gallery or any applications that demand zooming & directional panning) through new input interaction techniques and UI presentation. In this project, we define target zooming as zoom-in that requires panning (SW-NE diagonal movement, horizontal and vertical movements) to achieve a user-defined target in a map or an image. They are designed for flexible rectangular hand-held devices. It uses a set of different bend gestures performed at different locations and a new graphical user interface.

Funding agency: Nokia Technologies
Team members: Aditi Singh, Priyanka Das and Keyur Sorathia

The above video showcases the input interactions through a prototype. The details of each input interactions are presented in following sections.
Problems with one-handed interactions

Prior research work shows a strong preference towards one-handed usage amongst the users. The primary reason being the ability gained to perform real world tasks from the other hand. However, in current devices, one hand-usage of handheld devices comes with the trade-offs such as limited reachability, reduced accuracy, higher occlusion, difficult to perform diagonal swipe and constant need of re-gripping. Moreover, it is extremely difficulty to perform gestures involving more than one fingers e.g. pinching, rotating. For example, enlarging top-left corner of an image representing complex infographics.

The following figure shows a method used extensively to zoom-in on a top left corner of a high resolution and complex infographic. It also presents the problems encountered while performing the task. It presents problems of multi-step interactions, limited zoom-in capacity, difficult diagonal panning and reduced accuracy. Hence, cumulatively result in poor user experiences for target zooming (especially while zooming + navigating in a map application or an image) while using the device with one hand.

The proposed solution

The solution presents a method to perform one handed target zooming to a specific point on a flexible handheld device displaying a map application or a high-resolution image. The interaction technique offers a solution of zooming-in and out to a specific area on an image or a map by controlling the size and position of a virtual window on the flexible handheld device. The window is triggered by a bend gesture in either direction (inwards/outwards). As shown in figure 2 (a), bend can be performed at the top right/left corner (1 & 4), top edge (2), left (5) and right (3) edge. The direction of Bend i .e. inwards or outwards, controls zooming in and out respectively. The starting extent of the virtual window is that of the display and it continues to change its extent (by diminishing, Fig 2 (b)) as long as the Bend gesture is performed. Once the bend is released, the display of handheld flexible device zooms in/out to fit the extent of virtual window on the full display. The angle of bend represents the change in speed of the extent of virtual window.

For explaining the proposed solution, we present a use case of target zooming at all corners onto a maps application. Figure 2 (c) demonstrates different locations/corners which are to be viewed by user in detail.

1.Target zoom-in and zoom-out for specified location 1

In this section, we will present the method for target zoom-in and zoom-out for location 1 as highlighted in Figure 3.

Target zoom-in towards location 1 (bottom right corner)
Zoom in towards the location 1 of a map (e.g. here maps, google maps etc.) is done by performing an inward bend gesture on the top left corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 4 (a)), the top left vertex of which initially coincides with the top left corner of the flexible handheld device starts to move towards the bottom right corner of the device. The virtual window & its movement towards the bottom right corner is presented through a dotted line. This movement along the NW-SE diagonal of the device continues for as long as the Bend gesture (on top left corner) is performed resulting in proportional and continuous decrease in the size of the virtual window towards the bottom right corner of the device (Fig 4 (b)). Upon the release of the bend gesture, the map is zoomed-in to fit the display content of virtual window on the entire display (Fig 4 (c))

Target zoom-out - back to normal state
Zoom out to display content back to normal state (as shown in figure 4(a)) of the map application is done by performing an outward bend gesture on the top left corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 5 (a)), the top left vertex of which initially coincides with the top left corner of the flexible handheld device starts to move towards the bottom right corner of the flexible device (Fig 5 (b)). In this case, the display content diminishes with the virtual window to accommodate the display content beyond the top left corner. This movement along the NW-SE diagonal of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area initially beyond the top left corner. Upon the release of the bend gesture the virtual window disappears displaying contents which were beyond the top left corner (Fig 5 (c))
2.Target zoom-in and zoom-out for specified location 2

In this section, we will present the method for target zoom-in and zoom-out for location 2 as highlighted in Figure 6.

Target zoom-in at location 2 (bottom left corner)
Zoom in towards the location 2 (bottom left corner) of the map application is done by performing an inward bend gesture on the top right corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 7 (a)), the top right vertex of which initially coincides with the top right corner of the flexible handheld device starts to move towards the bottom left corner of the flexible device (Fig 7 (b)). This movement along the NE-SW diagonal of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous decrease in the size of the virtual window towards the bottom left corner of the device. Upon the release of the bend gesture the flexible handheld display device in zooms in the map to fit the area of virtual window on the entire display (Fig 7 (c)).

Target zoom-out – back to normal state
Zoom out to display content back to normal state of the map application is done by performing an outward bend gesture on the top right corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 8 (a)), the top right vertex of which initially coincides with the top right corner of the flexible handheld device starts to move towards the bottom left corner of the flexible device (Fig 8 (b)). In this case, the display content diminishes with the virtual window to accommodate the display content initially beyond the top right corner. This movement along the NE-SW diagonal of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area beyond the top right corner. Upon the release of the bend gesture the virtual window disappears displaying content that was beyond the top right corner (Fig 8 (c)).
3.Target zoom-in and zoom-out for specified location 3

In this section, we will present the method for target zoom-in and zoom-out for location 3 as highlighted in Figure 9

Target zoom-in at location 3 (top left corner)
Zoom in towards the location 3 (top left corner) of the map application is done by performing an inward bend gesture twice on the top left corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 10 (a)), the bottom right vertex of which initially coincides with the bottom right corner of the flexible handheld device starts to move towards the top left corner of the flexible device (Fig 10 (b)). This movement along the NW-SE diagonal of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous decrease in the size of the virtual window towards the top left corner of the device. Upon the release of the bend gesture the flexible handheld display device zooms in the map to fit the area of virtual window on the entire display (Fig 10 (c)).

Target zoom-out – back to normal state
Zoom out to display content back to normal state of the map application is done by performing an outward bend gesture twice on the top left corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 11 (a)), the bottom right vertex of which initially coincides with the bottom right corner of the flexible handheld device starts to move towards the top left corner of the flexible device. In this case, the initial display content diminishes with the virtual window to accommodate the display content beyond the bottom right corner. This movement along the NW-SE diagonal of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area beyond the bottom right corner (Fig 11 (b)). Upon the release of the bend gesture the virtual window disappears displaying content that was beyond the bottom right corner (Fig 11 (c)).
4.Target zoom-in and zoom-out for specified location 4

In this section, we will present the method for target zoom-in and zoom-out for location 4 as highlighted in Figure 12.

Target zoom-in at location 4 (top right corner)
Zoom in towards the location 4 (top right corner) of the map application is done by performing an inward bend gesture twice on the top right corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 13 (a)), the bottom left vertex of which initially coincides with the bottom left corner of the flexible handheld device starts to move towards the top right corner of the flexible device (Fig 13 (b)). This movement along the NE-SW diagonal of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous decrease in the size of the virtual window towards the top right corner of the device. Upon the release of the bend gesture the flexible handheld display device in zooms in the map to fit the area of virtual window on the entire display (Fig 13 (c)).

Target zoom-out – back to normal state
Zoom out to display content back to normal state of the map application is done by performing an outward bend gesture twice on the top right corner beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 14 (a)), the bottom left vertex of which initially coincides with the  bottom left corner of the flexible handheld device starts to move towards the top right corner of the flexible device (Fig 14 (b)). In this case, the display content diminishes with the virtual window to accommodate the display content initially beyond the bottom left corner. This movement along the NE-SW diagonal of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area beyond the bottom left corner. Upon the release of the bend gesture the virtual window disappears displaying content that was beyond the bottom left corner (Fig 14 (c)).
5.Target zoom-in and zoom-out for specified location 5

In this section, we will present the method for target zoom-in and zoom-out for location 5 as highlighted in Figure 15.

Target zoom-in at location 5 – center of right edge
Zoom in towards the location 5 (centre of right edge) of the map application is done by performing an inward bend gesture once around the centre of left edge beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 16 (a)), the left edge of which initially coincides with the left edge of the flexible handheld device starts to move towards the centre of right edge of the flexible device (Fig 16 (b)). This movement along the horizontal axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous decrease in the size of the virtual window towards the centre of right edge of the device. Upon the release of the bend gesture the flexible handheld display device zooms in the map to fit the area of virtual window on the entire display (Fig 16 (c)). 

Target zoom-out – back to normal state
Zoom out to display content back to normal state of the map application is done by performing an outward bend gesture on the left edge beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 17 (a)), the left edge of which initially coincides with the left edge of the flexible handheld device starts to move towards the right edge of the flexible device (Fig 17 (b)). In this case, the display content diminishes with the virtual window to accommodate the display content initially beyond left edge. This movement along the horizontal axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area beyond the left edge. Upon the release of the bend gesture the virtual window disappears displaying content that was beyond the left edge (Fig 17 (c)).
6.Target zoom-in and zoom-out for specified location 6

In this section, we will present the method for target zoom-in and zoom-out for location 6 as highlighted in Figure 18.

Target zoom-in at location 6 – center left edge
Zoom in towards location 6 (centre of left edge) of the map application is done by performing an inward bend gesture once around the centre of right edge beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 19 (a)), the right edge of which initially coincides with the right edge of the flexible handheld device starts to move towards the centre of left edge of the flexible device (Fig 19 (b)). This movement along the horizontal axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous decrease in the size of the virtual window towards the centre of left edge of the device. Upon the release of the bend gesture the flexible handheld display device zooms in the map to fit the area of virtual window on the entire display (Fig 19 (c)).

Target zoom-out – back to normal state
Zoom out to display content back to normal state of the map application is done by performing an outward bend gesture on the right edge beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 20 (a)), the right edge of which initially coincides with the right edge of the flexible handheld device starts to move towards the left edge of the flexible device (Fig 20 (b)). In this case, the display content diminishes with the virtual window to accommodate the display content initially beyond right edge. This movement along the horizontal axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area beyond the right edge. Upon the release of the bend gesture the virtual window disappears displaying content that was beyond the right edge (Fig 20 (c)).
7.Target zoom-in and zoom-out for specified location 7

In this section, we will present the method for target zoom-in and zoom-out for location 7 as highlighted in Figure 21.

Target zoom-in at location 7 – center of bottom edge
Zoom in towards location 7 (centre of bottom edge) of the map application is done by performing an inward bend gesture on the top edge beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 22 (a)), the top edge of which initially coincides with the top edge of the flexible handheld device starts to move towards the bottom edge of the flexible device (Fig 22 (b)). This movement along the vertical axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous decrease in the size of the virtual window towards the bottom edge of the device. Upon the release of the bend gesture the flexible handheld display device zooms in the map to fit the area of virtual window on the entire display (Fig 22 (c)).

Target zoom-out – back to normal state
Zoom out to display content back to normal state of the map application is done by performing an outward bend gesture once on the top edge beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 23 (a)), the top edge of which initially coincides with the top edge of the flexible handheld device starts to move towards the bottom edge of the flexible device (Fig 23 (b)). In this case, the display content diminishes with the virtual window to accommodate the display content initially beyond the top edge. This movement along the vertical axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area beyond the top edge. Upon the release of the bend gesture the virtual window disappears displaying content that was beyond the top edge (Fig 23 (c)).
8.Target zoom-in and zoom-out for specified location 8

In this section, we will present the method for target zoom-in and zoom-out for location 8 as highlighted in Figure 24.

Target zoom-in at location 8 – center of top edge
Zoom in towards location 8 (center of top edge) of the map application is done by performing an inward bend gesture once on the top edge twice beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 25 (a)), the bottom edge of which initially coincides with the bottom edge of the flexible handheld device starts to move towards the top edge of the flexible device (Fig 25 (b)).  This movement along the vertical axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous decrease in the size of the virtual window towards the top edge of the device. Upon the release of the bend gesture the flexible handheld display device zooms in the map to fit the area of virtual window on the entire display (Fig 25 (c)).

Target zoom-out – back to normal state
Zoom out to display content back to normal state of the map application is done by performing an outward bend gesture twice on the top edge beyond a predefined threshold. The bend gesture triggers the virtual window (Fig 26 (a)), the bottom edge of which initially coincides with the  bottom edge of the flexible handheld device starts to move towards the top edge of the flexible device. In this case, the display content diminishes with the virtual window to accommodate the display content initially beyond the bottom edge (Fig 26 (b)). This movement along the vertical axis of the device continues for as long as the Bend gesture is performed resulting in proportional and continuous inclusion of the area beyond the bottom edge. Upon the release of the bend gesture the virtual window disappears displaying content that was beyond the bottom edge (Fig 26 (c)).
Current status - prototype development and evaluation study
The designs are currently being prototyped and prepared for a comparative evaluation study to one-handed interactions for traditional mobile phones. We will publish the results and a paper in coming months