code: public static String[] tokenize(String text) {
          final int len = text.length();
          if (len == 0) {
              return ArrayUtils.EMPTY_STRING_ARRAY;
          }
          final List<String> list = new ArrayList<>();
          int i = 0;
          while (i < len) {
              char ch = text.charAt(i);
              int ch32 = text.codePointAt(i);
  
              if (isSymbolChar(ch32)) {
                  list.add(new String(Character.toChars(ch32)));
                  i += Character.isHighSurrogate(ch) ? 2 : 1;
                  continue;
              }
  
              if (isWordChar(ch)) {
                  int wordStart = i;
                  while (i < len && isWordChar(text.codePointAt(i))) {
                      i++;
                  }
  
                  list.add(text.substring(wordStart, i));
                  continue;
              }
  
              i++;
          }
  
          return list.toArray(new String[list.size()]);
      }
docs: Tokenizes a string by splitting on non-word characters. This should be equivalent to splitting on \W+ in Python.
  The meaning of \W is different in Android, Java 7, Java 8 hence the non-use of regular expressions.
  @param text the input text
  @return the string tokens