This paper presents a novel method that interactively recognizes and vectorizes hand-drawn strokes in front of a commodity webcam. Compared to existing methods, which recognize strokes on a completed drawing, our method captures both spatial and temporal information of the strokes, and faithfully vectorizes them with timestamps. By this, we can avoid various stroke recognition ambiguities, enhance the vectorization quality, and recover the stroke drawing order. This is a challenging problem, requiring robust tracking of pencil tip, accurate modeling of pen-paper contact, handling pen-paper and hand-paper occlusion, while achieving interactive performance. To address these issues, we develop the following novel techniques. First, we perform robust spatio-temporal tracking of pencil tip by extracting discriminable features, which can be classified with a fast cascade of classifiers. Second, we model the pen-paper contact by analyzing the edge-profile of the acquired trajectory and extracting the portions related to individual strokes. Lastly, we propose a spatio-temporal method to reconstruct meaningful strokes, which are coherent to the stroke drawing continuity and drawing order. By integrating these techniques, our method can support interactive recognition and vectorization of drawing strokes that are faithful to the actual strokes drawn by the user, and facilitate the development of various multimedia applications such as video scribing, cartoon production, and pen input interface.