README.md

---
extra_gated_heading: You need to share contact information with Alchemab to access this model
extra_gated_prompt: >-

  ### FAbCon Terms of Use
  
  FAbCon models follow a [modified Apache 2.0 license](https://huggingface.co/alchemab/fabcon-large/blob/main/LICENSE.md)
extra_gated_fields:
  First Name: text
  Last Name: text
  Email: text
  Organization: text
  By clicking 'Submit' below, I accept the terms of the license, agree to share contact information with Alchemab: checkbox
  I agree to being contacted about future products, services, and/or partnership opportunities: checkbox
extra_gated_description: >-
  The information you provide will be collected, stored, processed, and shared in accordance with the [Alchemab Privacy Notice](https://www.alchemab.com/privacy-policy/).
extra_gated_button_content: Submit
license: other
widget:
    - text: "ḢQVQLE"
tags:
  - biology
---

## FAbCon-medium 🦅🧬

FAbCon is a generative, antibody-specific language model based on the [Falcon model](https://huggingface.co/tiiuae/falcon-7b). It is pre-trained using causal language modelling,
and is suitable for a range of tasks. FAbCon-small, FAbCon-medium, and FAbCon-large are available for non-commercial use via a modified Apache 2.0 license. For any users seeking
commercial use of our models (and license for generated antibodies from all FAbCon models), please contact us.

| Model variant | Parameters | Config | License |
| ------------- | ---------- | ------ | ------- | 
| [FAbCon-small](https://huggingface.co/alchemab/fabcon-small)  | 144M       | 24L, 12H, 768d | Modified Apache 2.0 |
| [FAbCon-medium](https://huggingface.co/alchemab/fabcon-medium) | 297M       | 28L, 16H, 1024d | Modified Apache 2.0 |
| [FAbCon-large](https://huggingface.co/alchemab/fabcon-large) | 2.4B       | 56L, 32H, 2048d | Modified Apache 2.0 |

## Usage example - generation

Generating sequences can be done using HuggingFace's built-in `model.generate` method,

```
from transformers import (
    PreTrainedTokenizerFast,
    FalconForCausalLM
)

>>> tokenizer = PreTrainedTokenizerFast.from_pretrained("alchemab/fabcon-medium")
>>> model = FalconForCausalLM.from_pretrained("alchemab/fabcon-medium")
>>> o = model.generate(
            tokenizer("Ḣ", return_tensors='pt')['input_ids'][:, :-1],
            max_new_tokens=...,
            top_k = ...,
            temperature = ...
    )
>>> decoded_seq = tokenizer.batch_decode(o)
```

## Usage example - sequence property prediction

Use the `transformers` built-in SequenceClassification classes

```
from transformers import (
    PreTrainedTokenizerFast,
    FalconForSequenceClassification
)

>>> tokenizer = PreTrainedTokenizerFast.from_pretrained("alchemab/fabcon-medium")
>>> model = FalconForSequenceClassification.from_pretrained("alchemab/fabcon-medium")
>>> o = model(input_ids=tokenizer("Ḣ", return_tensors='pt')['input_ids'],
              attention_mask=tokenizer("Ḣ", return_tensors='pt')['attention_mask'])
```