Systems AI: A Declarative Learning Based Programming Perspective

Data-driven approaches are becoming dominant problem-solving techniques in many areas of research and industry. Unfortunately, current technologies do not make such techniques easy to use for application experts who are not fluent in machine learning nor for machine learning experts who aim at testing ideas on real-world data and need to evaluate those as a part of an end-to-end system. We review key efforts made by various AI communities to provide languages for high-level abstractions over learning and reasoning techniques needed for designing complex AI systems. We classify the existing frameworks based on the type of techniques as well as the data and knowledge representations they use, provide a comparative study of the way they address the challenges of programming real-world applications, and highlight some shortcomings and future directions.

[1]  Fan Yang,et al.  TensorLog: Deep Learning Meets Probabilistic DBs , 2017, ArXiv.

[2]  Bartosz Broda,et al.  Fextor: A Feature Extraction Framework for Natural Language Processing: A Case Study in Word Sense Disambiguation, Relation Recognition and Anaphora Resolution , 2013, Computational Linguistics - Applications.

[3]  Kevin Leyton-Brown,et al.  Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms , 2012, KDD.

[4]  Dan Roth,et al.  Learning Based Java for Rapid Development of NLP Systems , 2010, LREC.

[5]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[6]  Emir Pasalic,et al.  Design and Implementation of the LogicBlox System , 2015, SIGMOD Conference.

[7]  W. Marsden.  I and J , 2012 .

[8]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[9]  Dan Roth,et al.  Incidental Supervision: Moving beyond Supervised Learning , 2017, AAAI.

[10]  Avi Pfeffer,et al.  Structured Factored Inference: A Framework for Automated Reasoning in Probabilistic Programming Languages , 2016, ArXiv.

[11]  Frederick Reiss,et al.  SystemT: a system for declarative information extraction , 2009, SGMD.

[12]  Dan Roth,et al.  On Kernel Methods for Relational Learning , 2003, ICML.

[13]  Luc De Raedt,et al.  kLog: A Language for Logical and Relational Learning with Kernels (Extended Abstract) , 2012, IJCAI.

[14]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[15]  Taisuke Sato,et al.  PRISM: A Language for Symbolic-Statistical Modeling , 1997, IJCAI.

[16]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[17]  Johann Schumann,et al.  Under Consideration for Publication in J. Functional Programming Autobayes: a System for Generating Data Analysis Programs from Statistical Models , 2022 .

[18]  Parisa Kordjamshidi,et al.  EDISON: Feature Extraction for NLP, Simplified , 2016, LREC.

[19]  Christopher De Sa,et al.  DeepDive: Declarative Knowledge Base Construction , 2016, SGMD.

[20]  Randy H. Katz,et al.  A Berkeley View of Systems Challenges for AI , 2017, ArXiv.

[21]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[22]  A. Macdonald.  A Statistician , 1921 .

[23]  Letizia Tanca,et al.  What you Always Wanted to Know About Datalog (And Never Dared to Ask) , 1989, IEEE Trans. Knowl. Data Eng..

[24]  Xinlei Chen,et al.  Never-Ending Learning , 2012, ECAI.

[25]  Irene Stahl,et al.  The appropriateness of predicate invention as bias shift operation in ILP , 1995, Machine Learning.

[26]  Lise Getoor,et al.  Probabilistic Similarity Logic , 2010, UAI.

[27]  Ming-Wei Chang,et al.  Structured learning with constrained conditional models , 2012, Machine Learning.

[28]  Yura N. Perov,et al.  Venture: a higher-order probabilistic programming platform with programmable inference , 2014, ArXiv.

[29]  Luc De Raedt,et al.  Statistical Relational Artificial Intelligence: Logic, Probability, and Computation , 2016, Statistical Relational Artificial Intelligence.

[30]  Vivek Srikumar,et al.  WOLFE: Strength Reduction and Approximate Programming for Probabilistic Programming , 2014, AAAI Workshop: Statistical Relational Artificial Intelligence.

[31]  Carina Silberer,et al.  Proceedings of the International Conference on Language Resources and Evaluation (LREC) , 2008 .

[32]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[33]  John Langford,et al.  Efficient programmable learning to search , 2014, ArXiv.

[34]  Kristian Kersting,et al.  Relational linear programming , 2017, Artif. Intell..

[35]  William Yang Wang,et al.  ProPPR: Efficient First-Order Probabilistic Logic Programming for Structure Discovery, Parameter Learning, and Scalable Inference , 2014, StarAI@AAAI.

[36]  Parisa Kordjamshidi,et al.  Better call Saul: Flexible Programming for Learning and Inference in NLP , 2016, COLING.

[37]  Tim Rocktäschel,et al.  Programming with a Differentiable Forth Interpreter , 2016, ICML.

[38]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[39]  Christopher Ré,et al.  Probabilistic databases , 2011, SIGA.

[40]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[41]  Ben Taskar,et al.  Markov Logic: A Unifying Framework for Statistical Relational Learning , 2007 .

[42]  Pedro M. Domingos.  1 Markov Logic: A Unifying Framework for Statistical Relational Learning , 2010 .

[43]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[44]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[45]  Avi Pfeffer,et al.  Practical Probabilistic Programming , 2016, ILP.

[46]  Dan Roth.  Learning Based Programming , 1999 .

[47]  Parisa Kordjamshidi,et al.  Relational Learning and Feature Extraction by Querying over Heterogeneous Information Networks , 2017, ArXiv.

[48]  Parisa Kordjamshidi,et al.  Saul: Towards Declarative Learning Based Programming , 2015, IJCAI.

[49]  Stuart J. Russell,et al.  BLOG: Probabilistic Models with Unknown Objects , 2005, IJCAI.