Update README.md
Browse files
README.md
CHANGED
@@ -55,7 +55,7 @@ text='''In the heart of the bustling city, where towering skyscrapers touch the
|
|
55 |
With each passing light year, the anticipation of unraveling secrets that could alter humanity's
|
56 |
understanding of life in the universe grew ever stronger.'''
|
57 |
|
58 |
-
# chunk the text. The lower threshold is, the more chunks will be generated.
|
59 |
chunks=model.chunk_text(text, tokenizer, threshold=0)
|
60 |
|
61 |
# print chunks
|
|
|
55 |
With each passing light year, the anticipation of unraveling secrets that could alter humanity's
|
56 |
understanding of life in the universe grew ever stronger.'''
|
57 |
|
58 |
+
# chunk the text. The threshold can be (-inf, +inf). The lower threshold is, the more chunks will be generated.
|
59 |
chunks=model.chunk_text(text, tokenizer, threshold=0)
|
60 |
|
61 |
# print chunks
|