
LogicENN: A Neural Based Knowledge Graphs Embedding Model with Logical Rules - sel1
https://arxiv.org/abs/1908.07141
======
MLres
Knowledge Representation: Knowledge has played a pivotal role in the
development of civilizations. The 21st century has become the century of
explosion of humans knowledge thanks to the development of sophisticated
communication systems. This indeed provides a great opportunity in front of
the human being. However, proper representation and management of such a huge
amount of knowledge is a challenging task. The other challenge is to reason
over such a knowledge to get new knowledge. Knowledge Graph (KG) properly
addresses the challenge of representation of the knowledge obtained by human.
The KG represents knowledge in the form of multi-relational graph where
entities are connected by different relations. This indeed more consistent
with the essence of real world knowledge where the things are connected to
each others.

~~~
MLres
Learning over Symbolic Data: Knowledge graphs mostly contain symbolic data
which are connected. Learning over such a data is indeed a challenging task in
Machine Learning (ML) where ML models mostly work on vectors. Knowledge Graph
Embedding properly addresses this challenge. The core idea is that there is an
equivalent vector space for the symbolic KG space that follows the underlying
structure. KGE aims to map symbolic KG to a vector space.

~~~
MLres
Neural Based Embedding: Neural Networks (NNs) have been widely used to learn
over data represented in the form of vectors. NNs can be used to learn over
symbolic connected data (e.g., KG). They can be used to learn knowledge graph
representation and provide a mapping between symbolic KG space and the
corresponding vector space. In other words, the NNs embeds a symbolic KG to a
vector space by preserving the underlying structure.

~~~
MLres
Reasoning by Learning:When the Neural Network maps the KG to a vector space
and learns the KG representation, it can do reasoning: the NN gets a triple
(e.g., (Bonn, IsAcityIn, Germany)) and determines if this is true or not. This
prediction is resulted by a reasoning over the KG in the vector space.

~~~
MLres
From Logic to Algebra: After embedding a KG in its corresponding vector space,
each symbolic elements (entities and relations) will be assigned a vector.
Moving from symbols to vectors is equivalent with moving from logic to
algebra. In other words, after mapping a KG to a vector space, logical rules
can be converted to their corresponding algebraic formula. More concretely,
considering the target vector space provided for the KG, one can derive the
formula for each logical rules. Therefore, as there is an equivalent vector
for each of the symbols, there will be an equivalent algebraic formula for
each of the logical rules.

~~~
MLres
Capability vs practical rule injection:

The question is that if the NNs are capable of encoding logical rules? The
paper proposes a neural network and proves that the network is capable of
expressing any ground truth over encoded rules in the KG. The paper focuses on
one class NNs as well as one class of logical rules. Investigation of
capability of different class of NNs in encoding different class of logical
rules is indeed an important problem which can open a new window in front of
ML.

When one is sure that the KGE model has enough capability to encode a class of
logical rules, deriving a formula for each of logical rules is essential. The
derived formula can be used to guide the learning process.

The paper reported that injection of rules in the learning process
significantly improves performance the learner (KGE model). Therefore, the
prior knowledge which is encoded as logical rules significantly improves the
performance of the NN.

------
hadsed
How are people using knowledge graphs at companies? Immediate examples that
come to mind are product categorisation and perhaps trivia questions. What
else?

~~~
geraltofrivia
A variety of ways actually. A common use case is custom built ontologies and
knowledge graphs for linking, and annotating industrial data. Another
interesting way is to impart a degree of common sense to dialogue/qa systems.
Here's an excellent blog post on the topic - [https://medium.com/@mgalkin/the-
mushroom-effect-or-why-you-n...](https://medium.com/@mgalkin/the-mushroom-
effect-or-why-you-need-knowledge-graphs-for-dialog-systems-b7894f74cf86)

------
mark_l_watson
Thanks for linking this paper. I have professional experience working with
both knowledge graphs (as a contractor at Google) and deep learning (Capital
One). Adding logical rules as prior knowledge (an analog might be structure in
our brains that we are born with, not learned) with KG embedding is a really
interesting idea.

~~~
MLres
Thanks for the comment, Developing a learning model which is more close to the
learning done by human brain is indeed an interesting problem. That there
might be a prior knowledge in brain which is not learned, but affects (guides)
the learning process, inference and reasoning is really interesting. The
intuition is that the real world is a big multi-modal knowledge graph with an
underlying ontology, axioms and rules (e.g., principals in classic mechanics
etc). With this intuition, humans learn new facts from the big real world
multi modal knowledge graph by incorporating their (Aristotelian) senses as
well as five (inward/outward) wits. One prior knowledge might be intrinsic.
The nice point is that AI experts can map the knowledge graph and its elements
(axioms, rules, symbols, facts etc) to a target vector space and find
equivalent of the elements for the vector space (e.g., logical formula are
mapped to their corresponding equivalent algebraic formula). Actually, this is
multidisciplinary topic and I would invite people from different backgrounds
and fields to add comments. It would be appreciated

