Reading: "Motion Doodles: An Interface for Sketching Character Motion"

Abstract

• Present a novel system for sketching the motion of a character
  • Sketch a character
  • Draw a continuous sequence of lines, arcs, and loops
  • Parsed and mapped to a parameterized set of output motions that further reflect the location and timing of the input sketch

Contributions

• Present the design of a set of continuous (cursive) gestures for sketching a significant variety of motions, their locations, and their timing. These gestures are implemented in a sketchbased animation system and are demonstrated on a variety of display devices.
• Present a system that allows novices to sketch a 2D character and then draw a variety of animated motion for it, all within tens of seconds.

Strength and Weakness

• Strong in its use of a cursive motion specification, its ability to allow for fast experimentation, and its ease of use for non-experts.
• Focus on how 2D stylus input can be used to drive stylized 2D and 3D character motion
• Provide a written notation system that is much easier to learn and use, foregoing much of the detailed control that a general motion notation system can provide.
• Unlike actingbased interfaces, the motion gestures provide a meaningful visual record of the motion, as well as allowing for “superhuman” unphysical and exaggerated motions.
• Doesn't support the complete gesture vocabulary in 3D

Ideas for Expansion

• We could extend the library of motions so that it captures not only human-like models but also different kind of characters such as animals
• We could add functions such that motions depend on surrounding environments (not only the path that users draw.) I.e. If there is a tunnel-like object, whose height is low relative to the height of the character, and if our path touches that tunnel, then the character moves as if it passes through the tunnel like a toddler.

Open Questions

• What other gesture identification can we think of for creating different kinds of motions?
• If we apply this system to the areas mentioned in the paper (film storyboarding, theatre staging, choreography for dance and sports, and interactive games), how faster/more efficient would this be? (need more user studies)

Reading: "Multiperspective Panoramas for Cel Animation"

Contributions

• Introduce a new approach for simulating apparent camera motion through a 3D environment

(How multiperspective panoramas can be created from 3D models and camera paths)

• Propose applications in which predefined camera moves are applied to 3D scenes, including virtual reality fly-throughs, computer games, and architectural walk-throughs.

Strength and Weakness

• Leverages the strengths of both the computer and the artist. The computer permits the use of much more complex camera paths than can be created by hand
• Also allows easier experimentation in designing them.
• In many internet-based applications, the flat design made by this model may be significantly faster to download and interact with than true 3D models

Ideas for Extension

• This system can be used for training of filmmaking and practicing camera work for newcomers.
• Can be used for artistic expressions

Open Questions

• What other camera moves can we think other than pan, tilt-pan, zoom, and truck? ---How about rotate? Although the move of rotation requires to have 360 degree of views, can we approximate it by combining pan and tilt-pan?

Reading: "Image Warps for Artistic Perspective Manipulation"

Abstract

• Introduces a tool that gives users the ability to manipulate perspective in photographs using image space controls (a 2D warp guided by constraints based on projective geometry) similar to those used by artists.

Contributions

• Propose an interface that allows users to manipulate perspective in photographs using controls similar to those used by illustrators in the construction of perspective images, namely vanishing points and lines.
• Propose an optimization-based warping model which provides a means to plausibly simulate moderate changes in perspective induced by these controls

Strength and Weakness

• similar in that their approach optimizes a mapping over a meshed image, but it also uses constraints designed to give high level control over perspective
• does not rely on access to 3D scene geometry, so thier approach is simpler and more computationally tractable
• It will not generally produce a rectangular image; most results need to be manually cropped
• Rendering may introduce sampling artifacts, especially in areas undergoing a scale increase.
• The interface of our tool has a learning curve, and a user needs to understand the basic principles of perspective construction

Ideas for Extention

• This might be a good system for animators, illustrators, or comic artists. When they work on background images, sometimes they draw based on pictures of real scenes; By adjusting perspectives of the pictures that they have taken by this system, they will be able to find the most suitable composite for their work.
• The same thing can be said for architecture design.

Open Question

• How can we extend this system to motion videos(or animations) from photographs?

Reading: "Artistic Multiprojection Rendering"

Absract

• Introduce a tool for creating such multiprojection images and animations
  • a multiprojection rendering algorithm
  • an interactive interface for attaching local cameras to the scene geometry

• The most common method for creating a multiprojection image requires a combination of 3D rendering and 2D image compositing.

Contribution

• propose an algorithm that automatically resolves visibility for most practical cases and allows user adjustments when the automatically computed visibility is not satisfactory
• simple and intuitive set of camera constraints allowing the user to choose appropriate projections for a variety of artistic effects
• leverage multipass hardware rendering to achieve interactive rendering

Strength and Weakness

• adequately addresses the problem of resolving visibility in the multiprojection setting
• use a master camera to specify visibility ordering while maintaining shape distortions due to local cameras attached to each object
• provide several novel camera constraints that allow the user to obtain commonly desired effects with relative ease
• In certain cases, occlusion cycles can pose a problem in determining visibility ordering in an image
• difficulty in dealing with lighting effects

Ideas for Extention

• This system seems very suitable for architecture design. Drawings such as multi-sectional views on a single page and a large building are good examples for which users can utilize this system.

Open Questions

• What other uses can we think other than artistic expression, representation, and visualization?

Project Overview

Goal – What are we going to do?

The importance of statistical learning methods in computer graphics research has increased tremendously over the last decade due to the rise of powerful computational resources and the massive accumulation of image data on the internet. As we have seen in the class, recapturing classic problems in computer graphics in terms of statistical learning has brought us novel insights. For example, Automatic Photo Pop Up uses decision trees to classify geometric cues in an image, which allows this system to reconstruct 3D image based on a single picture instead of multiple images. ShadowDraw hugely relies on Content Based Image Reitreival, which utilizes a number of statistical algorithms such as nearest neighbor matching. The main idea behind the paper "HelpingHand: Example Based Stylization" was non-parametric modeling, which is used to selectively copy portions of a library, generating new data that maintains a similar statistical distribution with the given image data.

In "Sketch Classification and Classification-driven Analysis using Fisher Vectors," they use Fisher Vectors to characterize a sample image from a distribution by its deviation from the generative model. In this project, we will focus on Bag-of-Visual-Words---another method for image classification that is one of the most widely-used techniques for image classification as the paper mentions in the section 2.1 Related Work: Image Classification. Specifically, we will develop a content based image retrieval system by using Bag-of-Visual-Words to retrieve similar images given an input image data.

Previous Work – What related work has been done?

• As described above ShadowDraw, Automatic Photo Pop Up, HelpingHand, and Sketch Classification.

Others:

• "How do humans sketch objects?" (EITZ, M., HAYS, J., AND ALEXA, M. 2012.): develop a bag-of-features sketch

representation and use multi-class support vector machines, trained on thier sketch dataset, to classify sketches

• "Sketch Based Object Recognition" (ZHU,B., QUIGLEY, E. Stanford CS231a)
• "Visual Categorization with Bags of Keypoints" (CSURKA.G, DANCE.C, 2004)
• "A symmetry-aware flip invariant sketch histogram shape descriptor" (CAO, X., ZHANG, H., LIU, S., GUO, X., AND LIN, L. 2013.)

Approach – What approach are we going to try? Why do we think it will work well?

• Clean the data set for the sake of efficiency (Delete some images that might confuse the classification algorithm)
• Extract SURF features from the data set
• Cluster each SURF vector (corresponding to each image) into k categories via K-means
• Calculate Visual-Words for each cluster
• Transform each image into histogram of Visual-Word
• Use the histogram above to search a similar image
• Experiments. Observe the results. Comment for future work.

Methodology – What steps (task list) are required? Which of these steps in particular is hard? Contingencies– what to do if the hard steps don’t work out?

• Understand the learning algorithm in detail to the point where we can actually implment.
• Learn OpenCV as fast as possible (while doing the project)
• Need to find a moderate amount of image data set so that we can perform multiple tests and experiments in a short amount of time
• If we have time, we could experiment a few different ways of measuring similarity to find which is the best; But for now, we will use Histogram Intersection for our similarity measure function
• The hardest part would be implmentation. Contingencies: Will try to fix bugs as much as possible.

Metrics – How we will we know when we’re done? How will we know whether we have succeeded?

• The goal is to finish implementing the entire search system from scratch.
• When the search system outputs reasonable results, it means we have succeeded.

Future Work
We have limited our work in just creating a search engine for the sake of completing the project in a limited time. Often times, this sort of search engine is used in many computer graphics research. For future work, we hope to develop this system to use in an actual research relating to image classification.

Summary – What we will learn by doing this project?

• We will learn the role of statistical learning, particularly in content image based retreival through OpenCV implementation of image classification. We hope that the knowledge and the experience we will get through this project would benefit us in our future exploration in a wide range of research areas including computer graphics, computer vision, and machine learning.

SandCanvas: A Multi-touch Art Medium Inspired by Sand Animation

Abstract

• SandCanvas is a new multi-touch digital artistic medium that simplifies the creation of sand animations

Contributions

• Introduces a new digital artistic medium that helps the expressiveness of hand gestures on a multi-touch platform to provide a visual experience that goes beyond physical sand animation
• Analyze the sand animation process, highlighting common pouring and manipulation techniques and developing a taxonomy of hand gestures
• Present SandCanvas’s intuitive UI design and its enhanced digital capabilities
• Describe our implementation, which employs new techniques for performing real time sand simulation in response to gestural input
• Evaluate SandCanvas with 1 professional artist, 4 amateur artists and 2 novice users to gain insight into the importance and unique affordances of this medium

Strength and Weakness

• Captures rich human hand gestures in multiple areas instead of multiple points
• Preserves the expressive and playful nature of sand animation and adds new capabilities that go beyond traditional sand animation, such as recorded gestures and video mixing
• The system is also dependent on surrounding ambient light.

Ideas for Extension

• SandArt needs special settings and special efforts to create sophisticated artwork; How about some other forms? WaterArt...uses water as a canvas and use ink to create an image; should be Water Animation Gesture as in Sand Animation Gesture such as dropping ink, powering ink to manipulate the image.

Open Questions

• What would be applications of this system? The detailed model of sand drawing might be applicable to the system that deals with other granulated objects.

Kitty: Sketching Dynamic and Interactive Illustrations

Abstract
Kitty is a sketch-based tool for authoring dynamic and interactive illustrations. The application of this includes: technical illustrations, scientific explanation, infographics, medical illustrations, children's e-books, and cartoon strips.

Contribution

• Provide an easy-to-use, yet expressive tool for authoring interactive dynamic illustrations through freeform sketching
• Relational graph structure enables users/artists to produce intuitive interaction in their work without requiring any prior programming experience.
• the first system that allows authoring of interactive illustrations and functional relationships between objects through direct manipulation

Related Works in crafting Dynamic Illustrations

• 1. Authoring animated illustrations: e.g. Flash, Scratch etc. -> entities’ parameters

can be accessed as variables in scripts that enable
coordination between them....rather reply on pre-defined models or programming; Powerpoint etc -> interactivity is limited in linear global time; Kitty rather capitalizes on free-form sketching

• 2. Explanatory illustrations:
  • effective for communicating design ideas, technical assembly and

scientific concepts.

• • often employ a graph model to represent the dynamics of a system
  • Kitty also uses a similar model, in which artists can directly manipulate through sketching
• 3. Sketch-based user interfaces
  • sketch:powerful tool for rapid design and visual thinking
  • Kitty is build on Draco and K-Sketch

Strength and Weakness

• doesn't support the defition of groups
• doesn't support sophisticated effects such as inter-object interactions, temporal relationships and underlying physics (momentum, gravity),

Ideas for Extention

• Incorporate multi-touch screen
• Use computer vision approach to automatically categorize each element/object
• Or users can define each category by themselves too

Open Questions

• How much impact does this system have as a whole? Although the paper says that the system is applicable for a variety of purposes, how many of them will be benefited by this system