Spaces:

codeparrot
/

code-generation-models

Running

loubnabnl HF staff commited on May 27, 2022

Commit

d1cd5a2

•

1 Parent(s): c98c98f

update data

Files changed (1) hide show

datasets/github_code.txt CHANGED Viewed

@@ -1,4 +1,4 @@
-We also released [Github code dataset](https://huggingface.co/datasets/lvwerra/github-code), a 1TB of code data from Github repositories in 32 programming languages. It was created from the public GitHub dataset on Google [BigQuery](https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code). The dataset can be loaded in a streaming mode if you don't want to download it because of memory issues, this will create an iterable dataset:
 ```python
 from datasets import load_dataset

+We also released [Github code dataset](https://huggingface.co/datasets/lvwerra/github-code), a 1TB of code data from Github repositories in 32 programming languages. It was created from the public GitHub dataset on Google [BigQuery](https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code). The dataset can be loaded in streaming mode if you don't want to download it because of memory limitations, this will create an iterable dataset:
 ```python
 from datasets import load_dataset