plenz commited on
Commit
e30d1cf
1 Parent(s): 536e81b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -11,7 +11,7 @@ Paper abstract: <br>
11
  > *While Language Models (LMs) are the workhorses of NLP, their interplay with structured knowledge graphs (KGs) is still actively researched. Current methods for encoding such graphs typically either (i) linearize them for embedding with LMs – which underutilize structural information, or (ii) use Graph Neural Networks (GNNs) to preserve the graph structure – but GNNs cannot represent text features as well as pretrained LMs. In our work we introduce a novel LM type, the Graph Language Model (GLM), that integrates the strengths of both approaches and mitigates their weaknesses. The GLM parameters are initialized from a pretrained LM to enhance understanding of individual graph concepts and triplets. Simultaneously, we design the GLM’s architecture to incorporate graph biases, thereby promoting effective knowledge distribution within the graph. This enables GLMs to process graphs, texts, and interleaved inputs of both. Empirical evaluations on relation classification tasks show that GLM embeddings surpass both LM- and GNN-based baselines in supervised and zero-shot setting, demonstrating their versatility.*
12
 
13
 
14
- ## Uses
15
 
16
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
17
  In the paper we evaluate the model as a graph (and text) encoder for (text-guided) relation classification on ConceptNet and WikiData subgraphs. However, the model can be used for any task that requires encoding text-attributed graphs, texts, or interleaved inputs of both. See [Encoding Graphs and Texts](#encoding-graphs-and-texts) for an example implementation.
 
11
  > *While Language Models (LMs) are the workhorses of NLP, their interplay with structured knowledge graphs (KGs) is still actively researched. Current methods for encoding such graphs typically either (i) linearize them for embedding with LMs – which underutilize structural information, or (ii) use Graph Neural Networks (GNNs) to preserve the graph structure – but GNNs cannot represent text features as well as pretrained LMs. In our work we introduce a novel LM type, the Graph Language Model (GLM), that integrates the strengths of both approaches and mitigates their weaknesses. The GLM parameters are initialized from a pretrained LM to enhance understanding of individual graph concepts and triplets. Simultaneously, we design the GLM’s architecture to incorporate graph biases, thereby promoting effective knowledge distribution within the graph. This enables GLMs to process graphs, texts, and interleaved inputs of both. Empirical evaluations on relation classification tasks show that GLM embeddings surpass both LM- and GNN-based baselines in supervised and zero-shot setting, demonstrating their versatility.*
12
 
13
 
14
+ ## Usage
15
 
16
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
17
  In the paper we evaluate the model as a graph (and text) encoder for (text-guided) relation classification on ConceptNet and WikiData subgraphs. However, the model can be used for any task that requires encoding text-attributed graphs, texts, or interleaved inputs of both. See [Encoding Graphs and Texts](#encoding-graphs-and-texts) for an example implementation.