Elron commited on
Commit
a1edebf
β€’
1 Parent(s): 50bcc2d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -73
README.md CHANGED
@@ -10,76 +10,3 @@ pinned: false
10
  <div align="center">
11
  <img src="https://raw.githubusercontent.com/IBM/unitxt/main/assets/banner.png" alt="Image Description" width="100%" />
12
  </div>
13
-
14
- [![Button](https://img.shields.io/badge/Video-pink?style=for-the-badge)](https://unitxt.readthedocs.io/)
15
- [![Button](https://img.shields.io/badge/Demo-pink?style=for-the-badge)](https://unitxt.readthedocs.io/en/latest/docs/demo.html)
16
- [![Button](https://img.shields.io/badge/Tutorial-pink?style=for-the-badge)](https://unitxt.readthedocs.io/en/latest/docs/adding_dataset.html)
17
- [![Button](https://img.shields.io/badge/Paper-pink?style=for-the-badge)](https://arxiv.org/abs/2401.14019)
18
- [![Button](https://img.shields.io/badge/Documentation-pink?style=for-the-badge)](https://unitxt.readthedocs.io/en/latest/modules.html)
19
- [![Button](https://img.shields.io/badge/Catalog-pink?style=for-the-badge)](https://unitxt.readthedocs.io/en/latest/catalog/catalog.__dir__.html)
20
- [![Button](https://img.shields.io/badge/Contributors-pink?style=for-the-badge)](https://github.com/IBM/unitxt/blob/main/CONTRIBUTING.md)
21
- [![Button](https://img.shields.io/badge/PyPi-pink?style=for-the-badge)](https://pypi.org/project/unitxt/)
22
-
23
-
24
- In the dynamic landscape of generative NLP, traditional text processing pipelines limit research flexibility and reproducibility, as they are tailored to specific dataset, task, and model combinations. The escalating complexity, involving system prompts, model-specific formats, instructions, and more, calls for a shift to a structured, modular, and customizable solution.
25
-
26
- Addressing this need, we present Unitxt, an innovative library for customizable textual data preparation and evaluation tailored to generative language models. Unitxt natively integrates with common libraries like HuggingFace and LM-eval-harness and deconstructs processing flows into modular components, enabling easy customization and sharing between practitioners. These components encompass model-specific formats, task prompts, and many other comprehensive dataset processing definitions. The Unitxt-Catalog centralizes these components, fostering collaboration and exploration in modern textual data workflows. Beyond being a tool, Unitxt is a community-driven platform, empowering users to build, share, and advance their pipelines collaboratively.
27
-
28
- #
29
- [![version](https://img.shields.io/pypi/v/unitxt)](https://pypi.org/project/unitxt/)
30
- ![license](https://img.shields.io/github/license/ibm/unitxt)
31
- ![python](https://img.shields.io/badge/python-3.8%20|%203.9-blue)
32
- ![tests](https://img.shields.io/github/actions/workflow/status/ibm/unitxt/library_tests.yml?branch=main&label=tests)
33
- [![codecov](https://codecov.io/gh/IBM/unitxt/branch/main/graph/badge.svg?token=mlrWq9cwz3)](https://codecov.io/gh/IBM/unitxt)
34
- ![Read the Docs](https://img.shields.io/readthedocs/unitxt)
35
- [![downloads](https://static.pepy.tech/personalized-badge/unitxt?period=total&units=international_system&left_color=grey&right_color=green&left_text=downloads)](https://pepy.tech/project/unitxt)
36
-
37
- #
38
-
39
- https://github.com/IBM/unitxt/assets/23455264/baef9131-39d4-4164-90b2-05da52919fdf
40
-
41
- ### πŸ¦„ Currently on Unitxt Catalog
42
-
43
- ![NLP Tasks](https://img.shields.io/badge/NLP_tasks-40-blue)
44
- ![Dataset Cards](https://img.shields.io/badge/Dataset_Cards-457-blue)
45
- ![Templates](https://img.shields.io/badge/Templates-229-blue)
46
- ![Formats](https://img.shields.io/badge/Formats-18-blue)
47
- ![Metrics](https://img.shields.io/badge/Metrics-98-blue)
48
-
49
- ### πŸ¦„ Run Unitxt Exploration Dashboard
50
-
51
- To launch unitxt graphical user interface first install unitxt with ui requirements:
52
- ```
53
- pip install unitxt[ui]
54
- ```
55
- Then launch the ui by running:
56
- ```
57
- unitxt-explore
58
- ```
59
-
60
- # πŸ¦„ Contributors
61
-
62
- Please install Unitxt from source by:
63
- ```
64
- git clone git@github.com:IBM/unitxt.git
65
- cd unitxt
66
- pip install -e ".[dev]"
67
- pre-commit install
68
- ```
69
-
70
- # πŸ¦„ Citation
71
-
72
- If you use Unitxt in your research, please cite our paper:
73
-
74
- ```
75
- @misc{unitxt,
76
- title={Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI},
77
- author={Elron Bandel and Yotam Perlitz and Elad Venezian and Roni Friedman-Melamed and Ofir Arviv and Matan Orbach and Shachar Don-Yehyia and Dafna Sheinwald and Ariel Gera and Leshem Choshen and Michal Shmueli-Scheuer and Yoav Katz},
78
- year={2024},
79
- eprint={2401.14019},
80
- archivePrefix={arXiv},
81
- primaryClass={cs.CL}
82
- }
83
- ```
84
-
85
- Unitxt emoji designed by [OpenMoji](https://openmoji.org/#) - the open-source emoji and icon project. License: [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/#)
 
10
  <div align="center">
11
  <img src="https://raw.githubusercontent.com/IBM/unitxt/main/assets/banner.png" alt="Image Description" width="100%" />
12
  </div>