Recent advancements in aircraft controllers paired with increasingly flexible aircraft designs create the need for adaptive and intelligent control systems. To correctly capture the motion of a flexible aircraft wing and provide feedback to the controller, a large number of states (nodes along the span) must be monitored in real-time. Visual sensing methods carry the promise of flexibility needed for this type of smart sensing and control. However, visual sensing requires capturing and tracking keypoint features (marker tracking), while detecting thereof from a feature-rich image can be a computationally intensive task. The computational effort significantly increases with image size or when an image stereo pair is used to find matching keypoints. In this study, a parallel approach is presented with Threading Building Blocks (TBB), using sub-matrix computations, for extraction of corresponding keypoints from an image-stereo pair, and triangulation with the Direct Linear Transform (DLT) method to reconstruct the 3D position of the object in space. Additional robustness is investigated by implementing a Kalman filter for tracking prediction during the domain transition between the sub-matrices. Furthermore, a flexible simulation framework is set up for smart sensing with a coupled unsteady aeroservoelastic model of a 3D wing and a visual model to test the method for intelligent control feedback in a simulation environment. The methodology is tested in a laboratory environment with a stereo camera setup, and in a virtual environment, where the virtual camera parameters are reconstructed to meet a stereo setup. The proposed approach aims to advance the state-of-the-art in smart sensing, particularly in the context of real-time state estimation of aeroelastic structures and control feedback. The parallel approach shows a significant improvement of speed and efficiency, allowing real-time computation from a live image stream at 50 fps.
[1]
Michael I. Jordan,et al.
Real-Time Machine Learning: The Missing Pieces
,
2017,
HotOS.
[2]
Shane Legg,et al.
Human-level control through deep reinforcement learning
,
2015,
Nature.
[3]
A. W. Burner,et al.
Videogrammetric Model Deformation Measurement Technique
,
2001
.
[4]
Roeland De Breuker,et al.
Combined Active and Passive Loads Alleviation through Aeroelastic Tailoring and Control Surface/Control System Optimization
,
2018
.
[5]
Geert Lombaert,et al.
An augmented Kalman filter for force identification in structural dynamics
,
2012
.
[6]
Zhiyuan Xu,et al.
Model-free control for distributed stream data processing using deep reinforcement learning
,
2018,
VLDB 2018.
[7]
C. Papadimitriou,et al.
A dual Kalman filter approach for state estimation via output-only acceleration measurements
,
2015
.
[8]
Peter Corke,et al.
VISUAL CONTROL OF ROBOT MANIPULATORS – A REVIEW
,
1993
.
[9]
Gary R. Bradski,et al.
ORB: An efficient alternative to SIFT or SURF
,
2011,
2011 International Conference on Computer Vision.
[10]
J. G. Leishman,et al.
Subsonic unsteady aerodynamics caused by gusts using the indicial method
,
1996
.
[11]
Xiaogang Wang,et al.
Intelligent multi-camera video surveillance: A review
,
2013,
Pattern Recognit. Lett..
[12]
Alex Graves,et al.
Asynchronous Methods for Deep Reinforcement Learning
,
2016,
ICML.
[13]
Zhengyou Zhang,et al.
A Flexible New Technique for Camera Calibration
,
2000,
IEEE Trans. Pattern Anal. Mach. Intell..