Optimizing age of information on real-life TCP/IP connections through reinforcement learning

Age of Information (AoI) has emerged as a performance metric capturing the freshness of data for status-update based applications (e.g., remote monitoring) as a more suitable alternative to classical network performance indicators such as throughput or delay. Optimizing AoI often requires distinctly novel and sometimes counter-intuitive networking policies that adapt the rate of update transmissions to the randomness in network resources. However, almost all previous work on AoI to data has been theoretical, assuming idealized networking models, and known delay and service time distributions. It is difficult to obtain these statistics and optimize for them in a real-life network as there are many interacting phenomena in different networking layers (e.g., consider an end-to-end IoT application running over the Internet). With this work we introduce a deep reinforcement learning-based approach that can learn to minimize the AoI with no prior assumptions about network topology. After evaluating the learning model on an emulated network, we have shown that the method can be scaled up to any realistic network with unknown delay distribution.