metadata
license: apache-2.0
datasets:
- Shuu12121/rust-codesearch-dataset-open
- Shuu12121/java-codesearch-dataset-open
- code-search-net/code_search_net
- google/code_x_glue_ct_code_to_text
language:
- en
pipeline_tag: fill-mask
tags:
- code
- code-search
- retrieval
- sentence-similarity
- bert
- transformers
- deep-learning
- machine-learning
- nlp
- programming
- multi-language
- rust
- python
- java
- javascript
- php
- ruby
- go
new_version: Shuu12121/CodeModernBERT-Owl-v3-len2048
🦉CodeModernBERT-Owl
旧版のモデルです。こちらのモデルなど最新版をお使いください！！
** モデルパラメータ / Model Parameters**
| パラメータ / Parameter | 値 / Value |
|---|---|
| vocab_size | 50,004 |
| hidden_size | 768 |
| num_hidden_layers | 12 |
| num_attention_heads | 12 |
| intermediate_size | 3,072 |
| max_position_embeddings | 2,048 |
| type_vocab_size | 2 |
| hidden_dropout_prob | 0.1 |
| attention_probs_dropout_prob | 0.1 |
| local_attention_window | 128 |
| rope_theta | 160,000 |
| local_attention_rope_theta | 10,000 |
ライセンス / License
Apache-2.0