Knowledge graphs (KGs) are structured representations of factual knowledge organized as directed labeled graphs, where nodes represent entities and edges represent relations between them. Each fact is stored as a triple (head entity, relation, tail entity) -- for example, (Paris, capital_of, France). Large-scale knowledge graphs such as Wikidata, DBpedia, Freebase, and YAGO contain billions of triples covering millions of entities, providing the structured world knowledge that NLP systems need for question answering, entity linking, fact verification, and commonsense reasoning.
Knowledge Graph Embeddings
DistMult: score(h, r, t) = e_h^T diag(e_r) e_t
ComplEx: score(h, r, t) = Re(e_h^T diag(e_r) ē_t)
RotatE: score(h, r, t) = −||e_h ∘ e_r − e_t||
Link prediction: rank t* for (h, r, ?) by score
Knowledge graph embedding models learn low-dimensional vector representations of entities and relations such that the geometric relationship between entity vectors encodes the semantic relationship. TransE (Bordes et al., 2013) models relations as translations: for a true triple (h, r, t), the embedding of h plus the relation vector r should be close to the embedding of t. DistMult uses bilinear scoring, ComplEx extends this to complex-valued embeddings to handle asymmetric relations, and RotatE models relations as rotations in complex space. These embeddings enable link prediction (inferring missing triples) and knowledge base completion.
Knowledge Graph Construction
Building knowledge graphs from text involves several NLP subtasks: named entity recognition identifies entity mentions, entity linking maps mentions to knowledge base entries, relation extraction identifies the relations between entity pairs, and knowledge base population integrates new facts into the existing graph. Modern approaches use neural models for each component, with joint models that simultaneously extract entities and relations from text achieving the best results. Distant supervision, which automatically labels training data by aligning text with existing KG triples, reduces the need for manual annotation.
KGQA systems answer natural language questions by retrieving and reasoning over knowledge graph triples. Simple questions ("What is the capital of France?") require identifying the topic entity and relation, then looking up the answer in the KG. Complex questions require multi-hop reasoning ("Who is the spouse of the president of France?"), aggregation ("How many countries are in Europe?"), or temporal reasoning ("Who was president of the US in 1990?"). Approaches range from semantic parsing into SPARQL queries to embedding-based methods that score candidate answers in a shared vector space.
Integration with Language Models
A major research direction is integrating knowledge graphs with pre-trained language models. ERNIE and KnowBERT inject entity embeddings from knowledge graphs into the Transformer architecture during pre-training, improving performance on knowledge-intensive tasks. Retrieval-augmented generation (RAG) uses KG-based retrieval to provide relevant facts as context for language model generation, reducing hallucination and improving factual accuracy. Conversely, language models can be used to improve knowledge graphs: GPT-based models can generate relation descriptions, and BERT-based models improve entity linking and relation extraction.
Challenges in knowledge graph research include incompleteness (even the largest KGs are far from comprehensive), temporal dynamics (facts change over time), commonsense knowledge (encoding implicit knowledge like "birds can fly" with exceptions), and multimodal knowledge (integrating visual, textual, and structured information). The convergence of symbolic knowledge representation in KGs with neural language understanding in large language models represents one of the most active frontiers in NLP, with the goal of building systems that combine the reliability of structured knowledge with the flexibility of natural language understanding.