Update README.md
Browse files
README.md
CHANGED
@@ -2604,9 +2604,9 @@ language:
|
|
2604 |
license: mit
|
2605 |
---
|
2606 |
|
2607 |
-
# E5-base-
|
2608 |
|
2609 |
-
[LongEmbed: Extending Embedding Models for Long Context Retrieval](). Dawei Zhu, Liang Wang, Nan Yang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li, arxiv 2024. Github Repo for LongEmbed: https://github.com/dwzhu-pku/LongEmbed.
|
2610 |
|
2611 |
This model has 12 layers and the embedding size is 768.
|
2612 |
|
@@ -2664,7 +2664,7 @@ print(scores.tolist())
|
|
2664 |
|
2665 |
## Training Details
|
2666 |
|
2667 |
-
Please refer to our paper at [https://arxiv.org/
|
2668 |
|
2669 |
## Benchmark Evaluation
|
2670 |
|
@@ -2676,10 +2676,10 @@ on the [BEIR](https://arxiv.org/abs/2104.08663) and [MTEB benchmark](https://arx
|
|
2676 |
If you find our paper or models helpful, please consider cite as follows:
|
2677 |
|
2678 |
```
|
2679 |
-
@article{
|
2680 |
-
title={
|
2681 |
-
author={
|
2682 |
-
journal={arXiv preprint arXiv:
|
2683 |
-
year={
|
2684 |
}
|
2685 |
```
|
|
|
2604 |
license: mit
|
2605 |
---
|
2606 |
|
2607 |
+
# E5-base-4k
|
2608 |
|
2609 |
+
[LongEmbed: Extending Embedding Models for Long Context Retrieval](https://arxiv.org/abs/2404.12096). Dawei Zhu, Liang Wang, Nan Yang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li, arxiv 2024. Github Repo for LongEmbed: https://github.com/dwzhu-pku/LongEmbed.
|
2610 |
|
2611 |
This model has 12 layers and the embedding size is 768.
|
2612 |
|
|
|
2664 |
|
2665 |
## Training Details
|
2666 |
|
2667 |
+
Please refer to our paper at [https://arxiv.org/abs/2404.12096.pdf](https://arxiv.org/abs/2404.12096.pdf). Note that E5-Base-4k simply expands the position embedding matrix to allow for 4,096 position ids. The embedding vectors for the original pids {0,1,2,...,511} is mapped to represent {0,8,16,...,4088}. Embedding vectors for other pids are trained. So for inputs not exceeding 512 tokens, please multiply the position ids by 8 to maintain the original behavior, as shown in the code above.
|
2668 |
|
2669 |
## Benchmark Evaluation
|
2670 |
|
|
|
2676 |
If you find our paper or models helpful, please consider cite as follows:
|
2677 |
|
2678 |
```
|
2679 |
+
@article{zhu2024longembed,
|
2680 |
+
title={LongEmbed: Extending Embedding Models for Long Context Retrieval},
|
2681 |
+
author={Zhu, Dawei and Wang, Liang and Yang, Nan and Song, Yifan and Wu, Wenhao and Wei, Furu and Li, Sujian},
|
2682 |
+
journal={arXiv preprint arXiv:2404.12096},
|
2683 |
+
year={2024}
|
2684 |
}
|
2685 |
```
|