---
license: apache-2.0
datasets:
- Shuu12121/rust-codesearch-dataset-open
- Shuu12121/java-codesearch-dataset-open
- code-search-net/code_search_net
- google/code_x_glue_ct_code_to_text
language:
- en
pipeline_tag: fill-mask
tags:
- code
- code-search
- retrieval
- sentence-similarity
- bert
- transformers
- deep-learning
- machine-learning
- nlp
- programming
- multi-language
- rust
- python
- java
- javascript
- php
- ruby
- go
new_version: Shuu12121/CodeModernBERT-Owl-v3-len2048
---

# **🦉CodeModernBERT-Owl**

旧版のモデルです。[こちらのモデル](https://huggingface.co/Shuu12121/CodeModernBERT-Owl-v3-len2048)など最新版をお使いください！！


## ** モデルパラメータ / Model Parameters**
| パラメータ / Parameter   | 値 / Value |
|-------------------------|------------|
| **vocab_size**         | 50,004      |
| **hidden_size**        | 768        |
| **num_hidden_layers**  | 12         |
| **num_attention_heads**| 12         |
| **intermediate_size**  | 3,072      |
| **max_position_embeddings** | 2,048 |
| **type_vocab_size**    | 2          |
| **hidden_dropout_prob**| 0.1        |
| **attention_probs_dropout_prob** | 0.1 |
| **local_attention_window** | 128 |
| **rope_theta**         | 160,000    |
| **local_attention_rope_theta** | 10,000 |

---

## **ライセンス / License**
**Apache-2.0**