Deep reinforcement learning, symbolic learning and the road to AGI by Jeremie Harris

In Section 2, we categorize the different methods of neural-symbolic learning systems. Section 3 introduces the main technologies of neural-symbolic learning systems. We summarize the main applications of neural-symbolic learning systems in Section 4.

Finally, this review identifies promising directions and challenges for the next decade of AI research from the perspective of neurosymbolic computing, commonsense reasoning and causal explanation. Knowledge graph reasoning is the fundamental component to support machine learning applications such as information extraction, information retrieval, and recommendation. Since knowledge graphs can be viewed as the discrete symbolic representations of knowledge, reasoning on knowledge graphs can naturally leverage the symbolic techniques. On the contrary, the recent advances of deep learning have promoted neural reasoning on knowledge graphs, which is robust to the ambiguous and noisy data, but lacks interpretability compared to symbolic reasoning. Considering the advantages and disadvantages of both methodologies, recent efforts have been made on combining the two reasoning methods. In this survey, we take a thorough look at the development of the symbolic, neural and hybrid reasoning on knowledge graphs.

Hinge-loss markov random fields and probabilistic soft logic

New deep learning approaches based on Transformer models have now eclipsed these earlier symbolic AI approaches and attained state-of-the-art performance in natural language processing. However, Transformer models are opaque and do not yet produce human-interpretable semantic representations for sentences and documents. Instead, they produce task-specific vectors where the meaning of the vector components is opaque. (2) We provide a comprehensive overview of neural-symbolic techniques, along with types and representations of symbols such as logic knowledge and knowledge graphs.

This directed mapping helps the system to use high-dimensional algebraic operations for richer object manipulations, such as variable binding — an open problem in neural networks. When these “structured” mappings are stored in the AI’s memory (referred to as explicit memory), they help the system learn—and learn not only fast but also all the time. The ability to rapidly learn new objects from a few training examples of never-before-seen data is known as few-shot learning. Similar to the problems in handling dynamic domains, common-sense reasoning is also difficult to capture in formal reasoning. Examples of common-sense reasoning include implicit reasoning about how people think or general knowledge of day-to-day events, objects, and living creatures. Common-sense reasoning is an open area of research and challenging both for symbolic systems (e.g., Cyc has attempted to capture key parts of this knowledge over more than a decade) and neural systems (e.g., self-driving cars that do not know not to drive into cones or not to hit pedestrians walking a bicycle).

Title:SymbolicAI: A framework for logic-based approaches combining generative models and solvers

Sections on Machine Learning and Uncertain Reasoning are covered earlier in the history section. Early work covered both applications of formal reasoning emphasizing first-order logic, along with attempts to handle common-sense reasoning in a less formal manner. However, to be fair, such is the case with any standard learning model, such as SVMs or tree ensembles, which are essentially propositional, too. A similar problem, called the Qualification Problem, occurs in trying to enumerate the preconditions for an action to succeed. An infinite number of pathological conditions can be imagined, e.g., a banana in a tailpipe could prevent a car from operating correctly.

ArXiv is committed to these values and only works with partners that adhere to them. Similar axioms would be required for other domain actions to specify what did not change. Limitations were discovered in using simple first-order logic to reason about dynamic domains.

One of the most successful neural network architectures have been the Convolutional Neural Networks (CNNs) [3]⁴ (tracing back to 1982’s Neocognitron [5]). The distinguishing features introduced in CNNs were the use of shared weights and the idea of pooling. However, there have also been some major disadvantages including computational complexity, inability to capture real-world noisy problems, numerical values, and uncertainty.

Prolog provided a built-in store of facts and clauses that could be queried by a read-eval-print loop. The store could act as a knowledge base and the clauses could act as rules or a restricted form of logic. As a subset of first-order logic Prolog was based on Horn clauses with a closed-world assumption—any facts not known were considered false—and a unique name assumption for primitive terms—e.g., the identifier barack_obama was considered to refer to exactly one object. At the height of the AI boom, companies such as Symbolics, LMI, and Texas Instruments were selling LISP machines specifically targeted to accelerate the development of AI applications and research.

An AI model trained on data that looks real but won’t leak personal information

To provide a comprehensive understanding, the survey initially outlines key characteristics of symbolic systems and neural systems (refer to Table 1), including processing methods, knowledge representation, etc. Analysis of Table 1 reveals that symbolic systems and neural systems exhibit complementary features across various aspects. For instance, symbolic systems may possess limited robustness, whereas neural systems demonstrate robustness. Consequently, neural-symbolic learning systems emerge as a means to compensate for the shortcomings inherent in individual systems.

• Due to these problems, most of the symbolic AI approaches remained in their elegant theoretical forms, and never really saw any larger practical adoption in applications (as compared to what we see today).
• McCarthy’s Advice Taker can be viewed as an inspiration here, as it could incorporate new knowledge provided by a human in the form of assertions or rules.
• While we cannot give the whole neuro-symbolic AI field due recognition in a brief overview, we have attempted to identify the major current research directions based on our survey of recent literature, and we present them below.
• For instance, when confronted with unseen situations during training, machines may struggle to make accurate decisions in medical diagnosis.

Complex problem solving through coupling of deep learning and symbolic components. Coupled neuro-symbolic systems are increasingly used to solve complex problems such as game playing or scene, word, sentence interpretation. In a different line of symbolic learning work, logic tensor networks in particular have been designed to capture logical background knowledge to improve image interpretation, and neural theorem provers can provide natural language reasoning by also taking knowledge bases into account.

Supplementary data

The first one comes from the field of cognitive science, a highly interdisciplinary field that studies the human mind. In that context, we can understand artificial neural networks as an abstraction of the physical workings of the brain, while we can understand formal logic as an abstraction of what we perceive, through introspection, when contemplating explicit cognitive reasoning. In order to advance the understanding of the human mind, it therefore appears to be a natural question to ask how these two abstractions can be related or even unified, or how symbol manipulation can arise from a neural substrate [1].