An Automated Approach to Assessing an Application Tutorial’s Difficulty

Online step-by-step text and video tutorials play an integral role in learning feature-rich software applications. However, when searching, users can find it difficult to assess whether a tutorial is designed for their level of software expertise. Novice users can struggle when a tutorial is out of their reach, whereas more advanced users can end up wasting time with overly simple, first-principles instruction. To assist users in selecting tutorials, we investigate the feasibility of using machine-learning techniques to automatically assess a tutorial’s difficulty. Using Photoshop as our primary testbed, we develop a set of distinguishable tutorial features, and use these features to train a classifier that can label a tutorial as either Beginner or Advanced with 85% accuracy. To illustrate a potential application, we developed a tutorial browsing interface called TutVis. Our initial user evaluation provides insight into TutVis’s ability to support users in a range of tutorial selection scenarios.