1-800-BAD-CODE
commited on
Commit
•
8dcef59
1
Parent(s):
b21595d
Update README.md
Browse files
README.md
CHANGED
@@ -70,12 +70,6 @@ Therefore, language tags do not need to be used and a single batch can contain m
|
|
70 |
# Model Inputs and Outputs
|
71 |
The model inputs should be **punctuated** texts.
|
72 |
|
73 |
-
The inputs should be packed into a batch with shape `[B, T]` , with padding being the SPE model's `<pad>` token ID.
|
74 |
-
The `<pad>` ID is required to generate a proper attention mask.
|
75 |
-
|
76 |
-
The model was trained on a maximum sequence length of 256 (subwords), and may crash or perform poorly if a longer batch is processed.
|
77 |
-
Optimal handling of longer sequences would require some inference-time logic (wrapping/overlapping inputs and re-combining outputs).
|
78 |
-
|
79 |
For each input subword `t`, this model predicts the probability that `t` is the final token of a sentence (i.e., a sentence boundary).
|
80 |
|
81 |
# Example Usage
|
|
|
70 |
# Model Inputs and Outputs
|
71 |
The model inputs should be **punctuated** texts.
|
72 |
|
|
|
|
|
|
|
|
|
|
|
|
|
73 |
For each input subword `t`, this model predicts the probability that `t` is the final token of a sentence (i.e., a sentence boundary).
|
74 |
|
75 |
# Example Usage
|