[POSTER] Planning-Based Workflow Modeling for AR-enabled Automated Task Guidance

In this paper, we implemented and validated a workflow modeling approach that is able to model a sequence of procedures to achieve a complex task to enable an AR-based automated task guidance system. We formulated automated task guidance as a decision making problem, based upon the general Partially Observable Markov Decision Processes (POMDP) paradigm as the foundation. Our approach is able to provide actionable information to actively instruct users through a complex multi-step task. Our method can also plan ahead an action sequence that is optimal in the long term, while maintaining flexibility to deal with changes in an uncertain environment. We validated our approach in the applications of copy machine inspection and compressor startup guidance. The experimental results have shown the effectiveness of our planning-based workflow models in real-world applications.