(gram, mode='any')
Decide whether to keep or discard an n-gram.
    Args:
        gram: list of tokens (length N)
        mode: Option to throw out ngram if
          'any': any single token passes filter_word
          'all': all tokens pass filter_word
          'ends': book-ended by filterable tokens
Decide whether to keep or discard an n-gram.
    Args:
        gram: list of tokens (length N)
        mode: Option to throw out ngram if
          'any': any single token passes filter_word
          'all': all tokens pass filter_word
          'ends': book-ended by filterable tokens
Decide whether to keep or discard an n - gram . Args : gram : list of tokens ( length N ) mode : Option to throw out ngram if any : any single token passes filter_word all : all tokens pass filter_word ends : book - ended by filterable tokens
def filter_ngram(gram, mode='any'):
    """Decide whether to keep or discard an n-gram.
    Args:
        gram: list of tokens (length N)
        mode: Option to throw out ngram if
          'any': any single token passes filter_word
          'all': all tokens pass filter_word
          'ends': book-ended by filterable tokens
    """
    filtered = [filter_word(w) for w in gram]
    if mode == 'any':
        return any(filtered)
    elif mode == 'all':
        return all(filtered)
    elif mode == 'ends':
        return filtered[0] or filtered[-1]
    else:
        raise ValueError('Invalid mode: %s' % mode)
def filter_ngram ( gram , mode = 'any' ) : filtered = [ filter_word ( w ) for w in gram ] if mode == 'any' : return any ( filtered ) elif mode == 'all' : return all ( filtered ) elif mode == 'ends' : return filtered [ 0 ] or filtered [ - 1 ] else : raise ValueError ( 'Invalid mode: %s' % mode )