Update README.md
Browse files
README.md
CHANGED
@@ -6,4 +6,40 @@ library_name: paddlenlp
|
|
6 |
tags:
|
7 |
- conversational
|
8 |
---
|
9 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
tags:
|
7 |
- conversational
|
8 |
---
|
9 |
+
|
10 |
+
[![paddlenlp-banner](https://user-images.githubusercontent.com/1371212/175816733-8ec25eb0-9af3-4380-9218-27c154518258.png)](https://github.com/PaddlePaddle/PaddleNLP)
|
11 |
+
|
12 |
+
# PaddlePaddle/plato-mini
|
13 |
+
|
14 |
+
## Introduction
|
15 |
+
|
16 |
+
Pre-training models have been proved effective for a wide range of natural language processing tasks.
|
17 |
+
Inspired by this, we propose a novel dialogue generation pre-training framework to support various kinds of conversations,
|
18 |
+
including chit-chat, knowledge grounded dialogues, and conversational question answering. In this framework, we adopt flexible
|
19 |
+
attention mechanisms to fully leverage the bi-directional context and the uni-directional characteristic of language generation.
|
20 |
+
We also introduce discrete latent variables to tackle the inherent one-to-many mapping problem in response generation.
|
21 |
+
Two reciprocal tasks of response generation and latent act recognition are designed and carried out simultaneously within a shared network.
|
22 |
+
Comprehensive experiments on three publicly available datasets verify the effectiveness and superiority of the proposed framework.
|
23 |
+
|
24 |
+
More detail: https://arxiv.org/abs/1910.07931
|
25 |
+
|
26 |
+
## Available Models
|
27 |
+
|
28 |
+
- **plato-mini**, *12 layer, 12 heads, 768 hidden size, pretrained model*
|
29 |
+
|
30 |
+
## How to Use?
|
31 |
+
|
32 |
+
Click on the *Use in paddlenlp* button on the top right!
|
33 |
+
|
34 |
+
## Citation Info
|
35 |
+
|
36 |
+
```text
|
37 |
+
@article{ernie2.0,
|
38 |
+
title = {PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable},
|
39 |
+
author = {Bao, Siqi and He, Huang and Wang, Fan and Wu, Hua and Wang, Haifeng},
|
40 |
+
journal={arXiv preprint arXiv:1910.07931},
|
41 |
+
year = {2019},
|
42 |
+
}
|
43 |
+
```
|
44 |
+
|
45 |
+
|