Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model
Paper
ā¢
2510.18855
ā¢
Published
ā¢
69
Jia Guo
Jiiiia
AI & ML interests
None yet
Recent Activity
authored
a paper
about 1 month ago
Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale
Thinking Model
upvoted
a
paper
about 1 month ago
Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale
Thinking Model
liked
a model
about 2 months ago
inclusionAI/Ring-1T
Organizations
authored
a
paper
about 1 month ago
upvoted
a
paper
about 1 month ago
liked
5
models
about 2 months ago
inclusionAI/Ring-1T
Text Generation
ā¢
1000B
ā¢
Updated
ā¢
223
ā¢
222
inclusionAI/Ring-mini-2.0
Text Generation
ā¢
16B
ā¢
Updated
ā¢
259
ā¢
177
inclusionAI/Ring-flash-2.0
Text Generation
ā¢
103B
ā¢
Updated
ā¢
132
ā¢
100
inclusionAI/Ring-1T-preview
Text Generation
ā¢
1000B
ā¢
Updated
ā¢
144
ā¢
269
inclusionAI/Ling-1T
Text Generation
ā¢
1000B
ā¢
Updated
ā¢
249
ā¢
ā¢
520
upvoted
a
paper
2 months ago
liked
a
model
4 months ago
upvoted
a
paper
5 months ago
liked
a
model
9 months ago
upvoted
a
paper
over 1 year ago
reacted to
SivilTaram's
post with šš„
over 1 year ago
Post
2456
āļø Sailor: A New Multilingual Open LLM for South-East Asia š
Last month we have released a new family of multilingual language models called **Sailor**, ranging from 0.5B to 7B parameters, continually pre-trained from the Qwen1.5 models. Based on our extensive benchmarking, the Sailor models demonstrate exceptional performance on South-East Asian languages, taking us one step closer to multilingual LLMs that can serve the diverse needs of the region and beyond.
Today, we're more than excited to share the key technical details behind the Sailor models! šŖ
**Key highlights**:
š Data curation: Merging short examples, document-level code-switching, aggressive data cleaning and deduplication.
š¤ Tokenization Robustness: We find that BPE dropout is really effective to deal with prompt variations.
š Optimizing Data Mixture: We propose a new approach to automatically balance capabilities across different languages!
š Recipe in Continual Pre-training: We discover a powerful metric that can help predict how well the Sailor models will perform on the original domain (e.g., English) after continual pre-training.
We are thrilled to share these technical details with the community and invite you to explore the Sailor models. We hope Sailor models take us one step closer to multilingual LLMs in the world! šāØ
To learn more, please access our research paper or reach out to our team.
š Paper: Sailor: Open Language Models for South-East Asia (2404.03608)
š§© Model: sail/sailor-language-models-65e19a749f978976f1959825
š» Code: https://github.com/sail-sg/sailor-llm
Last month we have released a new family of multilingual language models called **Sailor**, ranging from 0.5B to 7B parameters, continually pre-trained from the Qwen1.5 models. Based on our extensive benchmarking, the Sailor models demonstrate exceptional performance on South-East Asian languages, taking us one step closer to multilingual LLMs that can serve the diverse needs of the region and beyond.
Today, we're more than excited to share the key technical details behind the Sailor models! šŖ
**Key highlights**:
š Data curation: Merging short examples, document-level code-switching, aggressive data cleaning and deduplication.
š¤ Tokenization Robustness: We find that BPE dropout is really effective to deal with prompt variations.
š Optimizing Data Mixture: We propose a new approach to automatically balance capabilities across different languages!
š Recipe in Continual Pre-training: We discover a powerful metric that can help predict how well the Sailor models will perform on the original domain (e.g., English) after continual pre-training.
We are thrilled to share these technical details with the community and invite you to explore the Sailor models. We hope Sailor models take us one step closer to multilingual LLMs in the world! šāØ
To learn more, please access our research paper or reach out to our team.
š Paper: Sailor: Open Language Models for South-East Asia (2404.03608)
š§© Model: sail/sailor-language-models-65e19a749f978976f1959825
š» Code: https://github.com/sail-sg/sailor-llm
liked
4
models
over 1 year ago