Reading #21: Teddy: A Sketching Interface for 3D Freeform Design

Comments:
Geroge!

Summary:
Teddy affords the user drawing 2D strokes and then automatically constructs potential 3D polygonal surfaces based on said strokes. Users interactively specify the silhouettes of objects, and the system attempts to create a 3D model which would match that silhouette (yes, I kind of just repeated myself). Keep in mind that Teddy was designed for rapid approximations. And the authors succeeded in this goal! Users were able to create basic models after as little as 10 minutes of getting used to the system.Once Teddy generates an initial 3D shape, users are able to view their model from different angles and can modify with various gestures (as shown above). The rest of the paper focuses on the algorithms used to perform the various modeling operations.

Discussion:
What I took away from this paper is that basic gestures can be used to perform some heavy back-end stuff... and to create animations that kids would love. What is important about this work is that the gestures seem intuitive, and thus users can understand how everything works quickly. I recommend that you look at the examples of each action for yourself, as this paper is filled with great screenshots of gestures in action.

Reading #20: MathPad2: A System for the Creation and Exploration of Mathematical Sketches

Comments:
Sam

Summary:
MathPad lets users draw mathematical expressions, symbols, and diagrams. Some diagrams can even be animated by the system! The authors suggest that allowing users to visualize their problems (which they will naturally do on pen and paper anyway) can help them in their sensemaking tasks.

As shown in the table above, basic gestures are used to inform MathPad of user intent for various items. Expressions are further converted to strings that can be evaluated by MatLab. MathPad also includes a nice set of computational functions that can aide users. And you can change your stroke color to help organize your work. It's the little things that count.

Discussion:
I did not go into much detail about MathPad, but do not let that in any way discredit it. The system sounds great! It also reminded me of Mechanix because the authors gave some consideration to usability and the aides they could provide to users. I think Mechanix takes it a step further in terms of features and freedom of drawing (recognition is off the chain), but MathPad sounds like a very smartly designed system all around. Good show.

Reading #19: Diagram Structure Recognition by Bayesian Conditional Random Fields

Comments:
Jonathan

Summary:
The recognition method discussed in this paper is based on Bayesian conditional random fields (BCRFs). BCRFs consider both spatial and temporal information, and can correlate features. CRFs are prone to overfitting, meaning that they do awesome for training data and horrible for new data. You could simulate this failnomenon by using the same training files over and over and over when building your feature set.

The authors are interested in discriminating between the containers and connectors in organization charts (see the figure at the top). They had 17 participants draw the chart shown, and ran 5 different algorithms to test the classification. The BCRFs proved to have the best recognition rates.

Discussion:
This paper was out of my league. Do you ever read something that makes you feel like you don't actually know anything about a given field? That was this paper. A little bit of side Googling returned some helpful links on BCRFs, etc., but I still felt lost. The results section basically showed me that everything they did was awesome and that it worked.

Reading #18: Spatial Recognition and Grouping of Text and Graphics

Comments:
Sam

Summary:
This paper discusses a spatial approach to the grouping and recognition of sketches. The process, as shown in the stolen image below, can be done in real-time. Strokes near each other are shown with the labeled graph in (b). Shapes are computed and matched to templates in (d). Templates return potential scores (0 to 1) which are used to determine best overall classification for the user's strokes.

I know what you're thinking... isn't speed an issue here?! The neighborhood graph in (b) helps to eliminate possible classifications based on vertex count and proximity. The authors also discredit potential strokes consisting of K components, where K is the number of strokes in the current largest template. Oh, and everything is based on machine learning (including the A* search). A user need only provide examples.

Discussion:
AdaBoost sounds like a deliciously nerdy energy drink. As the authors discuss in their... Discussion... an off the shelf system that is both efficient and accurate would be boss. If this work could be furthered to achieve similar results with fewer templates, then Rubine himself might rejoice and raise an AdaBoost toast to designer-accessible sketch recognition plug-ins.

Reading #17: Distinguishing Text from Graphics in On-line Handwritten Ink

Comments:
Kim

Summary:
What is the probability that a system can interpret your text from graphics when you draw with a stylus? Such is the question behind the work in this paper. The system described is broken into three main approaches.

1. Independent Strokes: Sequences of points between pen-down and pen-up events are taken to be strokes. 11 features are computed for each stroke. A multilayer perceptron is used to train a classifier as to which feature vectors correspond to either text or graphics.
2. Hidden Markov Model (HMM): The order of strokes can lend a clue as to what they should be classified as (unless the user jumps between a letter and a shape because they are weird). By looking at overall classification patterns, the HMM can be used to predict the current stroke givent he last stroke.
3. Bi-partite HMM: The gaps between strokes can lend additional information. A user will employ a different sized graph between two text strokes, two graphics, or a mixture therein.
   

Discussion:
I think I read this paper before... anyway! I did not like the way that they presented their results. Call me old fashioned, but I think you should always put your accuracies in plain old X (where X is your written language used). And was that plot drawn in paint? I felt like I was interpreting their findings rather than reading about them! Besides that, I thought the paper was very interesting.

Reading #16: An Efficient Graph-Based Symbol Recognizer

Comments:
Geroge

Summary:
This paper discusses (you guessed it) an efficient graph-based symbol recognizer. Using an Attributed Relational Graph (ARG), the authors can describe symbols in terms of their geometry and topology. A symbols geometry would include its primitive shapes and structures, which are treated as nodes. As the authors state, recognizing sketches involves matching graphs. And FYI, this can be hard as hell. So your user forgot to draw a primitive shape like a circle? There goes a node and a couple of edges. Although representing a symbol in terms of its topology allows for rotations and scaling to be matching, it cannot help the issue of missing components.

ARG for a perfect square.

Given training examples of each symbol class, the system constructs an "average ARG" for that class. To improve the average, the authors maintain stroke order and orientation across all examples. In testing, the system was proven to return the correct symbol in a top 3 list with over 93% accuracy.

Discussion:
I like how complex the symbols are that the authors tested in this paper. Some of them remind me of good old Civil Sketch. The trade-off between accuracy and speed of calculation is important, although even the longest recognition time only took 67.8 ms. Not too bad, but the fastest took 2.0 ms... but with accuracy around 79%. Accuracy vs. Time is an epic battle.

Reading #15: An Image-Based Trainable Symbol Recognizer for Sketch-Based Interfaces

Comments:
JJ

Summary:
Image-based recognition with only a single provided template. Such is the boast of the system designed by Kara and Stahovich. This paper also outlines the unique and low-cost polar coordinate analysis which is used to achieve rotation invariance. A three step process that begins with rotational checks is used to prune possible templates for any given sketch. Each sketch is treated as a 48x48 bitmap image which preserves the input's aspect ratio. Template matching is then carried out through the use of four different techniques. Results of these four techniques are then "parallelized" and "normalized", resulting in values between 0 and 1 which are used to determine how close an input sketch is to the different templates. Through a series of tests, the authors proved that their system was able to recognize the sketches of amateurs using only one or two templates with an accuracy of over 90%.

Discussion:
Though I brushed over it in the summary, a great new idea coming out of this paper is the polar transformation used to handle rotations. It is very constricting to require users to always draw with the exact same rotation. You could always create templates for different rotations of the same gesture... but why waste the time when you can use something as efficient as the transformer presented here?