

TITLE
OpenAI's Jukebox for music generation 

COMMENTS
0

REACTIONS
+1: 0
-1: 0
laugh: 0
hooray: 0
heart: 5
rocket: 0
eyes: 0

BODY
### Model description

**[Jukebox](https://openai.com/blog/jukebox/)** is an autoregressive model for music generation published by OpenAI in 2020. It is based on a [hierarchical VQ-VAE](https://arxiv.org/abs/1906.00446) and Scalable Transformers (based on [Sparse Transformers](https://arxiv.org/abs/1904.10509) ) to create long music samples that can be conditioned on Genres, Artists, Timing and Lyrics. 

3 sampling strategies will be made available : 

- Ancestral sampling : tokens are generated in an autoregressive fashion and are then upsampled
- Windowed sampling :  in order to generate long sequences, samples are repeatedly produced from overlapping windows using the previous codes as context. 
- Primed sampling : a continuation of a previous audio is obtained using the VQ-VAE encoding of the audio as initial tokens for the ancestral sampling

The generated tokens are then passed through the VQ-VAE decoder to obtain the final audio. 

The Lyric conditional informations are obtained using a Lyric Transformer model. 

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The code is available at https://github.com/openai/jukebox, and weights are available at https://openaipublic.azureedge.net/jukebox/models/. 

Authors : 

- [prafullasd](https://github.com/prafullasd)
- [heewooj](https://github.com/heewooj)
- [jongwook](https://github.com/jongwook)
- [mcleavey](https://github.com/mcleavey)
