Author: smichaels

Usability: Using the System Usability Scale (SUS) in Practice

Often times we’re asked to perform system usability studies to derisk the launch of a new site. If it’s late in the process (not atypical) one of the first things we’ll do is come up with a high level subjective view of usability. We use our in-house team as well as our network of usability consultants to come up with a simple assessment.

Our assessment uses the popular system usability scale (SUS) framework to come up with an industry standard score that evaluates the system as a whole. SUS comprises a high level questionairre that by design is quite general. We want to identify if the system in question is inherantly problematic or not. If it is problematic, we’ll dig deeper, if not, we’re ready to move on.

The questionairre asks a user to respond from a list of statements using a five-point Likert scale that ranges from Strong Disagreement to Strong Agreement.

Here’s a sample per question response that I’m sure you are all familiar with:

Here’s the standard questionairre. You can also find it over at

  1. I think that I would like to use this system frequently.
  2. I found the system unnecessarily complex.
  3. I thought the system was easy to use.
  4. I think that I would need the support of a technical person to be able to use this system.
  5. I found the various functions in this system were well integrated.
  6. I thought there was too much inconsistency in this system.
  7. I would imagine that most people would learn to use this system very quickly.
  8. I found the system very cumbersome to use.
  9. I felt very confident using the system.
  10. I needed to learn a lot of things before I could get going with this system.

Scoring is simple and normalized to a 100 point scale. There is a possible score per question of either 0, 1, 2, 3, or 4. In order to ensure no bias towards any of the statements with respect to agreement or disagreement, questions 1, 3, 5, 7, and 9 give more points to Strong Agreement (ie Strong Agreement = 4 points), while questions 2, 4, 6, 8, and 10 give more points to Strong Disagreement (ie Strong Disagreement = 4 points).

In order to base this on a 100 point scale for the 10 questions, we simply need to sum the score for each question and multiply that number by 2.5. One great aspect of the SUS is its ability to give you a simple single usability score that you can compare across products and systems (your competitors, best of breed solutions, your old website, etc.).


But how do you know what score is sufficient? If you are greater than 50, 60, 70, or 80, are you at an acceptable level? There isn’t to much literature out there on that topic but the folks at the measuring usability blog have done some interesting analytics and comparisons across industry. We generally take a simple back of the envelope analysis which suggests that if you were to receive 3 points on every question (that is, above the median, but not perfect), you’d score a 75. We take this as a proxy for a job mostly well done. For general purpose websites, we like to see scores that hover around that point and generally 70 is a number that we like to accept.

You won’t necessarily get too much insight into what the drivers causing issues actually are with SUS, although in practice, it’s often times easy to know where to start. We’ll often use SUS as a filter to let us know whether we need to dig deeper into all the tape, heatmaps, and screens that we captured during detailed usability studies. We talk about details in our next post.

Mathematics in Design

You may have heard of the Golden Ratio before and how it’s geometry is aesthetically pleasing. A couple of days ago, we had our own special geometry to solve. Our Creative Director Steve Newman @moodyjive came to me with a challenge. He was designing a compound shape and wanted to divide it into equal parts. He was currently eyeballing it, but something just didn’t feel right about that technique to him. As the perfectionists we are here at @fusebox, I’d thought I could lend a hand or a pencil if you will. I told him we’d solve this with closed form mathematics as his object was simply made up of a combination of circles and rectangles. Quite the skeptic, Steve handed over the problem to me.

The object he wanted to divide, first into 4 equal slices and then 3 equal slices, is pictured below. Here’s the problem statement in words.

    Given a fixed outer radius of the arc Ro, a fixed inner radius of the arc Ri, and a fixed height of the pillar h, determine the slicing point angle from the horizontal such that we will have four equal areas. Our slice is defined as the angle \alpha in degrees but in geometry it’s often more convient to work in radians because it directly relates the length of an arc to a radius and works great in equations. For those that don’t remember or don’t know, 360 degress is equal to 2\pi radians.

First Four equal parts.

Since we have vertical symmetry, we can just use the left side of the inverted U noting the vertical cut is necessary. The area of a complete donut is simply described as the area of a circle with radius Ro minus the area of a circle with radius of Ri. We then need to divide the area of a complete donut by the ratio of the arc length of the inner radius of A1. Recalling high school mathematics, the arc length is simply a radius multiplied by the coverage angle in radians or simply  Ri (\pi/2 - \alpha) . This proportion of entire area of a circular arc is simply the arc length divided by the circumference of the cirle or Ri (\pi/2 - \alpha)  / 2 \pi Ri  . Notice that the proportion is equal 1/4 the area of a circle when the angle is zero.

Therefore the arc area:

A1 = \pi (Ro^{2}-Ri^{2}) Ri * (\pi/2 - \alpha) / 2 \pi Ri =  (\pi/2 - \alpha) (Ro^{2}-Ri^{2}) / 2

The area of the rectangular pillar of height h is then the area of the rectangle with thickness Ro-Ri and height of h plus the area of the complementary arc that sweeps through \alpha.

A2 = h ( Ro - Ri ) +   Ri  \alpha / (2 \pi Ri )    \pi (Ro^2 - Ri^2)     =  h ( Ro - Ri ) + \alpha   (Ro^2 - Ri^2) / 2

What we need to do is solve for \alpha when we set A1=A2. Expanding the equations, and then group by \alpha one can readily show the solution is:

\alpha = [ \pi /4 (Ro^2 - Ri^2) - h(Ro - Ri) ] / (Ro^2 - Ri^2)

When we substitute his values Ro = 51, Ri=27, h=24, we find that \alpha is 0.47771 radians or 27.37 degrees.

Note: because we relied on symmetry, the equation is valid for \alpha between 0 and 90 degrees only.

The math for dividing into three (3) equal areas is quite similar and left as an exercise to the reader, We’re calling the angle \beta. Here’s the sketch. Put your answer in the comments if you dare for the same geometry. I’ll post our answer on Friday. 🙂

UPDATE: 7/26/2011
Given the same geometry, our angle \beta is 48.247 degrees.

“3D in Flash” book release with contributions from Fusebox staff guru Peter Kapelyan

The buzz around our office this week has been all about the debut of The Essential Guide to 3D in Flash, a book written with the help of one of our senior Flash Developer, Peter Kapelyan. The book teaches ActionScript developers and designers the core Flash 3D concepts using the Away3D engine as a primary tool. It hit the shelves of Barnes & Noble yesterday and is available online at

Peter has been working in Flash and 3D for over 14 years. In addition to his role at Fusebox, he is part of the 10-person team behind Away3D, an open source 3D Flash engine. Away3D enables users to produce complex, professional quality 3D models in Flash. It holds a wide variety of potential applications, and has been used successfully to create 3D games, architecture floorplans, [need another example], examples of which can be viewed at Away3D.

Peter contributed his expertise in 3D modeling throughout the book, and specifically to the chapter called “Optimization Tips & Tricks.” When asked about the project he explained that, “For about two years, the team talked about writing a book. Then Richard Olsson showed up about a year ago offering to write it, and the idea started to become a reality. We all pitched in, we’re really excited about it.”

We’ve got a few copies already en route to our office, you can get hold of one at

Fusebox has long been an advocate of Flash’s 3-D capabilities and have recently done some showcase work for JPMorgan Chase utilizing Google’s KML language overlaid with a interactive 3-D Globe to let the user experience their global presence.

If you have projects that require Flash 3D for gaming or application development you need to work with us. Please contact Laura Michaels for business development inquiries.