# TRL Model

This model is [TRL language model](https://github.com/huggingface/trl) and is a part of a study explained [here](https://github.com/Watheq9/d2qminus-repro) on document expansion using Doc2Query. The model has been fine-tuned with reinforcement learning to guide the model outputs according to a value, function, or human feedback. The model can be used for text generation. Please cite the paper below if you use it.

To use this model for inference, first install the TRL library:

```bash
pip install trl
```

You can then generate text as follows:

```python
from transformers import pipeline

generator = pipeline("text-generation", model="watheq/d2q_monoELECTRA_1400")
outputs = generator("Coffee is a beverage brewed from roasted coffee beans. Coffee has a stimulating effect on humans, primarily due to its caffeine content.")
```

If you want to use the model for training or to obtain the outputs from the value head, load the model as follows:

```python
from transformers import AutoTokenizer
from trl import AutoModelForCausalLMWithValueHead

tokenizer = AutoTokenizer.from_pretrained("watheq/d2q_monoELECTRA_1400")
model = AutoModelForCausalLMWithValueHead.from_pretrained("watheq/d2q_monoELECTRA_1400")

inputs = tokenizer("Coffee is a beverage brewed from roasted coffee beans. Coffee has a stimulating effect on humans, primarily due to its caffeine content.", return_tensors="pt")
outputs = model(**inputs, labels=inputs["input_ids"])
```



## Citation

If you used any piece of this repository, please consider citing our work :

```plaintext
@inproceedings{mansour2024revisit,
  title={Revisiting Document Expansion and Filtering for Effective First-Stage Retrieval},
  author={Mansour, Watheq and Zhuang, Shengyao and Zhuang, Guido and Mackenzie, Joel},
  booktitle = {Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year={2024},
  publisher = {Association for Computing Machinery},
  series = {SIGIR '24}
}
```


---
license: cc-by-4.0
---