Declarative Learning-Based Programming as an Interface to AI Systems

Data-driven approaches are becoming more common as problem-solving techniques in many areas of research and industry. In most cases, machine learning models are the key component of these solutions, but a solution involves multiple such models, along with significant levels of reasoning with the models' output and input. Current technologies do not make such techniques easy to use for application experts who are not fluent in machine learning nor for machine learning experts who aim at testing ideas and models on real-world data in the context of the overall AI system. We review key efforts made by various AI communities to provide languages for high-level abstractions over learning and reasoning techniques needed for designing complex AI systems. We classify the existing frameworks based on the type of techniques and the data and knowledge representations they use, provide a comparative study of the way they address the challenges of programming real-world applications, and highlight some shortcomings and future directions.

[1]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[2]  Dan Klein,et al.  Neural Module Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Parisa Kordjamshidi,et al.  Saul: Towards Declarative Learning Based Programming , 2015, IJCAI.

[4]  Kevin Leyton-Brown,et al.  Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms , 2012, KDD.

[5]  Dan Roth,et al.  Learning Based Java for Rapid Development of NLP Systems , 2010, LREC.

[6]  William Yang Wang,et al.  ProPPR: Efficient First-Order Probabilistic Logic Programming for Structure Discovery, Parameter Learning, and Scalable Inference , 2014, StarAI@AAAI.

[7]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[8]  Xinlei Chen,et al.  Never-Ending Learning , 2012, ECAI.

[9]  Dan Suciu,et al.  Probabilistic databases , 2011, SIGA.

[10]  Lise Getoor,et al.  Probabilistic Similarity Logic , 2010, UAI.

[11]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[12]  Tim Rocktäschel,et al.  Programming with a Differentiable Forth Interpreter , 2016, ICML.

[13]  Joshua B. Tenenbaum,et al.  Church: a language for generative models , 2008, UAI.

[14]  Dan Roth,et al.  On Kernel Methods for Relational Learning , 2003, ICML.

[15]  Letizia Tanca,et al.  What you Always Wanted to Know About Datalog (And Never Dared to Ask) , 1989, IEEE Trans. Knowl. Data Eng..

[16]  Ming-Wei Chang,et al.  Structured learning with constrained conditional models , 2012, Machine Learning.

[17]  Parisa Kordjamshidi,et al.  Better call Saul: Flexible Programming for Learning and Inference in NLP , 2016, COLING.

[18]  Dan Roth,et al.  A Linear Programming Formulation for Global Inference in Natural Language Tasks , 2004, CoNLL.

[19]  Johann Schumann,et al.  Under Consideration for Publication in J. Functional Programming Autobayes: a System for Generating Data Analysis Programs from Statistical Models , 2022 .

[20]  Guy Van den Broeck,et al.  Query Processing on Probabilistic Data: A Survey , 2017, Found. Trends Databases.

[21]  Balder ten Cate,et al.  Declarative Probabilistic Programming with Datalog , 2016, ICDT.

[22]  Christopher De Sa,et al.  DeepDive: Declarative Knowledge Base Construction , 2016, SGMD.

[23]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[24]  Fan Yang,et al.  TensorLog: Deep Learning Meets Probabilistic DBs , 2017, ArXiv.

[25]  Emir Pasalic,et al.  Design and Implementation of the LogicBlox System , 2015, SIGMOD Conference.

[26]  Kristian Kersting,et al.  Boosted Statistical Relational Learners: From Benchmarks to Data-Driven Medicine , 2015 .

[27]  David A. Ferrucci,et al.  UIMA: an architectural approach to unstructured information processing in the corporate research environment , 2004, Natural Language Engineering.

[28]  John Langford,et al.  Efficient programmable learning to search , 2014, ArXiv.

[29]  Irene Stahl,et al.  The appropriateness of predicate invention as bias shift operation in ILP , 1995, Machine Learning.

[30]  Kristian Kersting,et al.  Relational linear programming , 2017, Artif. Intell..

[31]  Dan Roth,et al.  Lifted First-Order Probabilistic Inference , 2005, IJCAI.

[32]  Randy H. Katz,et al.  A Berkeley View of Systems Challenges for AI , 2017, ArXiv.

[33]  Luc De Raedt,et al.  kLog: A Language for Logical and Relational Learning with Kernels (Extended Abstract) , 2012, IJCAI.

[34]  Ian H. Witten,et al.  Weka: Practical machine learning tools and techniques with Java implementations , 1999 .

[35]  Pedro M. Domingos 1 Markov Logic: A Unifying Framework for Statistical Relational Learning , 2010 .

[36]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[37]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[38]  Taisuke Sato,et al.  PRISM: A Language for Symbolic-Statistical Modeling , 1997, IJCAI.

[39]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[40]  Parisa Kordjamshidi,et al.  EDISON: Feature Extraction for NLP, Simplified , 2016, LREC.

[41]  Avi Pfeffer,et al.  Practical Probabilistic Programming , 2016, ILP.

[42]  Dan Roth Learning Based Programming , 1999 .

[43]  Ben Taskar,et al.  BLOG: Probabilistic Models with Unknown Objects , 2007 .

[44]  Yura N. Perov,et al.  Venture: a higher-order probabilistic programming platform with programmable inference , 2014, ArXiv.

[45]  Luc De Raedt,et al.  Statistical Relational Artificial Intelligence: Logic, Probability, and Computation , 2016, Statistical Relational Artificial Intelligence.

[46]  Vivek Srikumar,et al.  WOLFE: Strength Reduction and Approximate Programming for Probabilistic Programming , 2014, AAAI Workshop: Statistical Relational Artificial Intelligence.

[47]  Lluís A. Belanche Muñoz,et al.  Feature selection algorithms: a survey and experimental evaluation , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[48]  Frederick Reiss,et al.  SystemT: a system for declarative information extraction , 2009, SGMD.

[49]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[50]  Dan Roth,et al.  Incidental Supervision: Moving beyond Supervised Learning , 2017, AAAI.

[51]  Parisa Kordjamshidi,et al.  Relational Learning and Feature Extraction by Querying over Heterogeneous Information Networks , 2017, ArXiv.

[52]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[53]  Ben Taskar,et al.  Markov Logic: A Unifying Framework for Statistical Relational Learning , 2007 .

[54]  Bartosz Broda,et al.  Fextor: A Feature Extraction Framework for Natural Language Processing: A Case Study in Word Sense Disambiguation, Relation Recognition and Anaphora Resolution , 2013, Computational Linguistics - Applications.

[55]  Avi Pfeffer,et al.  Structured Factored Inference: A Framework for Automated Reasoning in Probabilistic Programming Languages , 2016, ArXiv.