code: public void omitLessFreq() {
          if (name == null) throw new IllegalStateException();
  
          int threshold = nWords[0] / LESS_FREQ_RATIO;
          if (threshold < MINIMUM_FREQ) threshold = MINIMUM_FREQ;
          
          Set<String> keys = freq.keySet();
          int roman = 0;
          for(Iterator<String> i = keys.iterator(); i.hasNext(); ){
              String key = i.next();
              int count = freq.get(key);
              if (count <= threshold) {
                  nWords[key.length()-1] -= count;
                  i.remove();
              } else {
                  if (key.matches("^[A-Za-z]$")) {
                      roman += count;
                  }
              }
          }
  
          // roman check
          if (roman < nWords[0] / 3) {
              Set<String> keys2 = freq.keySet();
              for(Iterator<String> i = keys2.iterator(); i.hasNext(); ){
                  String key = i.next();
                  if (key.matches(".*[A-Za-z].*")) {
                      nWords[key.length()-1] -= freq.get(key);
                      i.remove();
                  }
              }
          }
      }
docs: Removes ngrams that occur fewer times than MINIMUM_FREQ to get rid of rare ngrams.
  
  Also removes ascii ngrams if the total number of ascii ngrams is less than one third of the total.
  This is done because non-latin text (such as Chinese) often has some latin noise in between.
  
  TODO split the 2 cleaning to separate methods.
  TODO distinguish ascii/latin, currently it looks for latin only, should include characters with diacritics, eg Vietnamese.
  TODO current code counts ascii, but removes any latin. is that desired? if so then this needs documentation.