---
license: apache-2.0
language:
- code
- en
datasets:
- saridormi/commit-chronicle
tags:
- code
- commit_message_generation
pipeline_tag: text2text-generation
---

# CMG/CMC: CodeT5 (without history)

This is the checkpoint for [CodeT5](https://huggingface.co/Salesforce/codet5-base) model, fine-tuned for the commit message generation (and/or completion) task as part of the paper "From Commit Message Generation to History-Aware Commit Message Completion", ASE 2023.

## Details

> 🔍 For further details, please refer to:
> * **Paper**: TODO
> * **Repository**: [https://github.com/JetBrains-Research/commit_message_generation](https://github.com/JetBrains-Research/commit_message_generation)


* This model is based on [`Salesforce/codet5-base`](https://huggingface.co/Salesforce/codet5-base) checkpoint from 📜 [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://aclanthology.org/2021.emnlp-main.685/).
* This model was trained with commit diffs, WITHOUT commit message history.
* This model was trained on the CommitChronicle dataset introduced in our study. 
* Our hyperparameter setting is mostly based on 📜 [RACE: Retrieval-augmented Commit Message Generation](https://aclanthology.org/2022.emnlp-main.372/).
 The exact values are provided below:
    
    |       Hyperparameter       |                                                                    Value                                                                    |
    |:--------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------:|
    | Encoder context max length |                                                                     512                                                                     |
    | Decoder context max length |                                                                     512                                                                     |
    | Number of training epochs  |                                                                      1                                                                      |
    |         Batch size         |                                                                     32                                                                      |
    |         Optimizer          |                  [AdamW](https://pytorch.org/docs/1.12/generated/torch.optim.AdamW.html?highlight=adamw#torch.optim.AdamW)                  |
    |           Warmup           | [Linear](https://huggingface.co/docs/transformers/v4.21.3/en/main_classes/optimizer_schedules#transformers.get_linear_schedule_with_warmup) |
    |   Number of warmup steps   |                                                                     100                                                                     |
    |     Peak learning rate     |                                                                   0.00002                                                                   |


## Available checkpoints

We also released checkpoints for other models fine-tuned as part of our study.

* Models trained *with commit message history*:
  * **CodeT5:** 🤗 [`JetBrains-Research/cmg-codet5-with-history`](https://huggingface.co/JetBrains-Research/cmg-codet5-with-history)
  * **CodeReviewer:** 🤗 [`JetBrains-Research/cmg-codereviewer-with-history`](https://huggingface.co/JetBrains-Research/cmg-codereviewer-with-history) 
  * **RACE:** 🤗 [`JetBrains-Research/cmg-race-with-history`](https://huggingface.co/JetBrains-Research/cmg-race-with-history)
* Models trained *without commit message history*:
  * **CodeT5:** 🤗 [`JetBrains-Research/cmg-codet5-without-history`](https://huggingface.co/JetBrains-Research/cmg-codet5-without-history) (this model)
  * **CodeReviewer:** 🤗 [`JetBrains-Research/cmg-codereviewer-without-history`](https://huggingface.co/JetBrains-Research/cmg-codereviewer-without-history)
  * **RACE:** 🤗 [`JetBrains-Research/cmg-race-without-history`](https://huggingface.co/JetBrains-Research/cmg-race-without-history)

## Citation

```
TODO
```