pipeline_tag: sentence-similarity
language:
- es
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
- LSG
- 4096 context
license: apache-2.0
LSG Variant of hiiamsid/sentence_similarity_spanish_es
Overview
This model is an enhanced version of hiiamsid/sentence_similarity_spanish_es, now transformed using Local Sparse Global (LSG) attention mechanism. The adaptation to LSG allows for efficient handling of longer sequences, making the model more versatile and robust in a wider range of natural language processing tasks.
This LSG adaptation enables the model to efficiently process sequences up to 4096 tokens in length.
About the LSG architecture
LSG (Local Sparse Global) attention is a cutting-edge approach designed to mitigate the limitations of the traditional self-attention mechanism in Transformer models, particularly for processing long sequences. By incorporating local, sparse, and global attention, LSG attention significantly reduces computational complexity while maintaining, and often enhancing, model performance.
Model adaptation
This LSG variant has been adapted from our original model, "hiiamsid/sentence_similarity_spanish_es", with the primary goal of extending its capabilities to efficiently handle longer text inputs. This enhancement enables the model to maintain high accuracy and efficiency, even with extended sequence lengths that were previously challenging for the base model.
Use cases
The LSG-enhanced model is particularly adept at tasks involving longer documents, where capturing the essence of extended context is crucial.
Acknowledgments
This model has been adapted by Darío Muñoz Prudant, thanks to the Hugging Face community and contributors to the LSG attention mechanism for their resources and support.