juyongjiang
commited on
Commit
•
a5d6bee
1
Parent(s):
799f8f0
upgrade images
Browse files- README.md +2 -7
- assets/Logo.jpg +2 -2
- assets/PL_Filter.jpg +3 -0
README.md
CHANGED
@@ -7,12 +7,11 @@ tags:
|
|
7 |
- multilingual-code-generation
|
8 |
---
|
9 |
|
10 |
-
# CodeUp: A Multilingual Code Generation Llama2 Model with Parameter-Efficient Instruction-Tuning on a Single RTX 3090
|
11 |
-
|
12 |
<!-- <p align="center" width="70%">
|
13 |
<img src="assets/Logo.jpg" alt="HKUST CodeUp" style="width: 50%; min-width: 250px; display: block; margin: auto;">
|
14 |
</p> -->
|
15 |
![HKUST CodeUp](assets/Logo.jpg)
|
|
|
16 |
|
17 |
## Description
|
18 |
In recent years, large language models (LLMs) have shown exceptional capabilities in a wide range of applications due to their fantastic emergence ability. To align with human preference, instruction-tuning and reinforcement learning from human feedback (RLHF) are proposed for Chat-based LLMs (e.g., ChatGPT, GPT-4). However, these LLMs (except for Codex) primarily focus on the general domain and are not specifically designed for the code domain. Although Codex provides an alternative choice, it is a closed-source model developed by OpenAI. Hence, it is imperative to develop open-source instruction-following LLMs for the code domain.
|
@@ -45,11 +44,7 @@ This way, we gain the 19K high-quality instruction data of code generation. The
|
|
45 |
| -- | -- |
|
46 |
| <center><img src="assets/PL_Raw.png" width="100%"></center> | <center><img src="assets/PL_Clean.png" width="92%"></center> | -->
|
47 |
|
48 |
-
|
49 |
-
![Raw Data (20K + 4K)](assets/PL_Raw.png)
|
50 |
-
|
51 |
-
**Filtered Data (19K)**
|
52 |
-
![Filtered Data (19K)](assets/PL_Clean.png)
|
53 |
|
54 |
|
55 |
## Training & Inference
|
|
|
7 |
- multilingual-code-generation
|
8 |
---
|
9 |
|
|
|
|
|
10 |
<!-- <p align="center" width="70%">
|
11 |
<img src="assets/Logo.jpg" alt="HKUST CodeUp" style="width: 50%; min-width: 250px; display: block; margin: auto;">
|
12 |
</p> -->
|
13 |
![HKUST CodeUp](assets/Logo.jpg)
|
14 |
+
# CodeUp: A Multilingual Code Generation Llama2 Model with Parameter-Efficient Instruction-Tuning on a Single RTX 3090
|
15 |
|
16 |
## Description
|
17 |
In recent years, large language models (LLMs) have shown exceptional capabilities in a wide range of applications due to their fantastic emergence ability. To align with human preference, instruction-tuning and reinforcement learning from human feedback (RLHF) are proposed for Chat-based LLMs (e.g., ChatGPT, GPT-4). However, these LLMs (except for Codex) primarily focus on the general domain and are not specifically designed for the code domain. Although Codex provides an alternative choice, it is a closed-source model developed by OpenAI. Hence, it is imperative to develop open-source instruction-following LLMs for the code domain.
|
|
|
44 |
| -- | -- |
|
45 |
| <center><img src="assets/PL_Raw.png" width="100%"></center> | <center><img src="assets/PL_Clean.png" width="92%"></center> | -->
|
46 |
|
47 |
+
![PL Data Filtering)](assets/PL_Filter.jpg)
|
|
|
|
|
|
|
|
|
48 |
|
49 |
|
50 |
## Training & Inference
|
assets/Logo.jpg
CHANGED
Git LFS Details
|
Git LFS Details
|
assets/PL_Filter.jpg
ADDED
Git LFS Details
|