This page outlines Slalom, a system with prototype implementation for concisely describing and implementing a class of touch interactions. Various examples demonstrate Slalom's versatility.

Slalom grew out of my previous article on how to use physics simulations to build touch-driven user interfaces. When writing that, I observed that I was writing much of the same code across the examples and that this repetitive, often inflexible code plagues most touch device engineering. Most touch interactions on today's tablets and smartphones are implemented by writing new code to handle every touch movement, which computes various transforms and updates the application's view tree or the browser's document object model (DOM). The coder needs a reasonable understanding of Newtonian physics to create something that feels good or natural to use. As a result, most apps use only the few touch interactions provided by the operating system: scroll and tap.

Many touch interactions involve the user's dragging a visual object such as a panel, a photo, an icon, a window or any other structural element of the user interface (and the object's moving synchronously under the touch point) and imparting momentum to that object, such that when released the object keeps moving with some momentum. The touch handler that you write will impose some constraints on how and where an object can move, such as constricting it to a single axis and clamping the distance it can travel. If you're implementing a complex interaction, then every time you iterate your idea and want to change something, you have to rewrite a lot of manual, very detailed code.

What if we came up with a system where we just specify how something moves and what the constraints are?

Slalom

Slalom combines the Cassowary linear constraints solver with a minimal physics engine to solve a class of touch interactions declaratively. It has the following components:

1. Linear Constraints are used to place objects relative to each other and to restrict how they can move. Slalom uses the Cassowary linear constraint solver which has found a niche in 2D-layout recently (it's the solver used by Apple's AutoLayout and Grid Style Sheets). If we think of a touch interaction as a slot-car game, then the linear constraints let us define the track on which the slot-cars will travel.
   
   For example, this is how we'd use linear constraints to position an image that is wider than its parent view, whose size is often the size of the screen (in essence, you're making a small viewport onto a large photo, showing just a part of the image):
   scaledPhotoWidth = (parentHeight / photoHeight) * photoWidth

   photo.y = 0

   photo.bottom = parentHeight

   photo.right = photo.x + scaledPhotoWidth
   Note that we did not constrain the photo's x-coordinate. We want that to be unconstrained (or weakly constrained, using Cassowary's constraint priorities) so that we can manipulate the photo. The image doesn't move when you drag it because we haven't yet set up a manipulator...
2. A Manipulator listens for touch events and updates a Cassowary variable in response. The manipulator knows how to use the momentum to create an animation when a touch gesture ends. In our slot-car metaphor, the manipulator is both the hand-held car controller (with accelerator) and car.
   
   To make our image draggable, we would create a manipulator for it like this:
   Drag the photo left and right
   
   // Drags in x on the parentElement will change the photo's x coordinate.

   context.addManipulator(new Manipulator(photo.x, parentElement, 'x'))
   With the above addition, you can drag the image. By default, a manipulator puts momentum into a friction simulation, which is why, after you release the photo, it keeps moving while slowing down. Notice that you can scroll the photo too far to either side, however, beyond the edges of the viewport. To fix this, we need motion constraints...
3. Motion Constraints place limits on how far an object can move or where it can stop. Motion constraints are similar to linear constraints, except they're enforced by physics simulations (typically springs) so that when a variable being controlled by a manipulator violates a motion constraint the object bounces or undertracks the finger. Motion constraints would be like a rubberband stretched across our slot-car track or like a speed bump on the track.
   
   We can add motion constraints to our image example like this:
   Drag the photo left and right and release it
   
   // Prevent the photo's left edge from going right of the parent's left edge:

   // photo.x <= 0

   context.addMotionConstraint(new MotionConstraint(photo.x, '<=', 0))

   // Prevent the photo's right edge from going to the left of the parent's right edge:

   // photo.right >= parentWidth

   context.addMotionConstraint(new MotionConstraint(photo.right, '>=', parentWidth))
   I haven't yet written a parser for either the linear constraints or the motion constraints; that's why a motion constraint looks like a JavaScript call and not like an equation. Check out the source code for this example on GitHub. Now you can drag and flick the image without its vanishing into the distance.

The above Slalom example describes full-momentum scrolling with just one manipulator and two motion constraints. It handles all of the tricky edge cases, such as when the user starts dragging while the photo is already bouncing on one of the edges, or when the user imparts momentum while in the overdrag. In systems where there are multiple manipulators, Slalom identifies the correct manipulator that is causing a motion constraint violation and applies feedback to it.

More examples

Here are some more examples with links to the source. All require significantly less code and the code is cleaner using Slalom than when using the imperative alternative.

Vertical scrolling lists

Twitter for iPad

The next example reproduces the old Twitter for iPad panels interface. The second variant introduces some of the non-default properties of motion constraints.

Animation using gravity

A manipulator doesn't have to use a friction simulation for the animation it creates after you let go of a drag. Here's an example using gravity instead:

Photo viewer

Another really useful operator for motion constraints is modulo, "%". Here we use a motion constraint that isn't enforced during dragging to ensure that our photo viewer animates to show only one photo. This actually uses a variation of modulo called "adjacent modulo" to prevent the user from flying past many photos with one big high-velocity flick.

Scaling like Facebook Paper

In Slalom, we're not limited to just translating objects. If we add a manipulator to the y-coordinate of a box and constrain the bottom of the box, then we can grow and shrink the box. Notice that we're expressing motion constraints on variables that are somehow related to scale but aren't the scale variable. We can do this because Slalom discovers the relationship between two variables by inspecting the Cassowary simplex tableau—this is a huge advantage over a numerical constraint solver where it would be harder to find how two variables relate to each other.

Currently the manipulator consumes all drag events. If it passed on unused drag deltas then we could add a second manipulator to translate the box horizontally, creating something like Facebook's Paper UI.

Note that we could tweak the constraints further. For instance, we could make the box go off the bottom when it gets too small instead of shrinking further. So we have a lot of flexibility with this kind of system.

Also note that there's slip! If you drag from the middle of the box, you'll see that the part of the image you grabbed slips out from under your finger. That's because the manipulator is just operating on the box's y coordinate. If we wanted to avoid slip then we'd have to create a new variable when the finger goes down, relate it to y (that is, fingery = box.y - 123 * scale, based on the current scale and finger start position) and then manipulate fingery instead of box.y. This isn't very hard, but the current manipulator code doesn't support it.

iOS notification and control centers

This next example is an implementation of the iOS notification center and control center UIs. The notification center is actually very interesting because it uses a different physical model when coming down (positive gravity) than when going up (anti-gravity). It looks weird if it slows down while going upwards. I was able to do this by creating a custom motion constraint operator that changed its physics model depending on the end point. (For the purposes of this paper, I opted not to implement the arrow buttons within the UIs but it should be straightforward to do so, either outside Slalom or by extending Slalom.)

iOS application switcher

This example demonstrates a way to make the iOS application switcher using Slalom. There are two manipulators: one which controls the first app's x-coordinate, and one which controls the first icon's x-coordinate. The icons are offset by half the scroll offset of the apps, so dragging the icons (small squares at bottom) moves the apps (large rectangles at top) twice as fast. Even though we have two manipulators, we still need only two motion constraints.

The real iOS app switcher applies some non-linear translation to the icons (based on velocity?) which I am unable to represent with linear constraints.

Windows overscroll

In Windows 7, Microsoft implemented touch overscroll by translating the whole window rather than just the content. We can use linear constraints to make this happen too, and we need only one motion constraint (the window should stay where it is) to bounce things back.

Google Maps details transition

This example implements the complex transitions that occur when the user drags upward within a Google Maps location view. The linear constraints are fairly straightforward, while the motion constraints that prevent the UI's stopping half-way open reflect the complexity of the transitions. The motion constraint is predicated on multiple variables, so I wrote a custom operator to handle it; I'd like to be able to express this more elegantly though, possibly with some kind of a Range type.

Future directions

We can use Slalom to describe a wide variety of touch interactions. There's a lot left to do, however, some involving further research:

• Non-linear constraints are needed for some interfaces (one example is the Chrome for Android tab switcher—the tabs move less when they're at the top) and these can't be expressed with Cassowary. For simple "external" variables with no dependencies it would suffice to provide a non-linear mapping function (and inverse function).
• Manipulators recognize only one gesture currently. I plan to extend the Manipulator class or subdivide it so that it's possible to manipulate multiple variables with one gesture. This would allow the above scaling example to become more like Facebook Paper.
• Various parts of Slalom need better explanation, including the captive flag on the Motion Constraints class.
• Currently there are no state or conditional constraints. To tackle more ambitious interactions, we need a notion of state that enables and modifies constraints, similar to how class names can be used to change CSS properties. Grid Style Sheets handle some of this and manage to avoid feedback loops.
• Currently there is no transition support. Often you want to use a tap to change some value (for example, the iOS notification panel and control center should hide when you tap the arrow buttons). It's also common to want to transition from a resting position to where a touch point (which is moving) will be.
• Slalom could be ported to one of the mobile platforms to ensure it integrates well.
• A parser could be written for linear and motion constraints, once their syntax is defined. I want to do a native port first to identify any syntax or parser changes needed to support integration with a traditional toolkit.