Filterings on top of near-dedup + line filtering:
* Comments filtering (at least 1% of the number of lines should be comments/docstrings) 
* Stars filtering (minimum of 5 stars) (on top of near-dedup + line filtering)


 |Language |Before filtering  | Stars | Comments ratio| More near-dedup| Tokenizer fertility| 
|-------|--------|-------|--------|--------|--------|
|Python | 75.61 GB| 26.56 GB |65.64 GB | 61.97 GB | 72.52 GB |
|Java | 110 GB |35.83 GB |92.7 GB | 88.42 GB | 105.47 GB |
|Javascript | 82.7 GB |20.76 GB |57.5 GB| 65.09 GB | 76.37 GB |

region:us
{}
TAGS
#region-us 

Filterings on top of near-dedup + line filtering:


* Comments filtering (at least 1% of the number of lines should be comments/docstrings)
* Stars filtering (minimum of 5 stars) (on top of near-dedup + line filtering)




TAGS
#region-us 

passage: TAGS
#region-us 
