Model card for CoNN Add Carry

Introduction

In paper Neural Comprehension: Language Models with Compiled Neural Networks , we introduced the integration of Compiled Neural Networks (CoNN) into the framework of language models, enabling existing language models to perform symbolic operations with perfect accuracy without the need for external tools. In this model card, we introduce the Add Carry model, which is similar to the Transformer model and can perform carry operations on a sequence of numbers added in parallel.

Install

git clone https://github.com/WENGSYX/Neural-Comprehension
cd Neural-Comprehension
pip install .

To run neural comprehension, you need to install PyTorch, Transformers, jax, and tracr.

How to Use?

from NeuralCom.CoNN.modeling_conn import CoNNModel
from NeuralCom.CoNN import Tokenizer


model = CoNNModel.from_pretrained('WENGSYX/CoNN_Add_Carry')
tokenizer = Tokenizer(model.config.input_encoding_map, model.config.output_encoding_map,model.config.max_position_embeddings)

output = model(tokenizer('2 15 3 8 10').unsqueeze(0))
print(tokenizer.decode(output.argmax(2)))

>>> [['bos', '3', '5', '3', '9', '0']]

🙏Cite🙏

If you are interested in our paper, please feel free to cite it.

@misc{weng2023neural,
      title={Neural Comprehension: Language Models with Compiled Neural Networks}, 
      author={Yixuan Weng and Minjun Zhu and Fei Xia and Bin Li and Shizhu He and Kang Liu and Jun Zhao},
      year={2023},
      eprint={2304.01665},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}