code: def tokenize_middle_high_german_words(text):
      
  
      assert isinstance(text, str)
      # As far as I know, hyphens were never used for compounds, so the tokenizer treats all hyphens as line-breaks
      text = re.sub(r'-\n',r'-', text)
      text = re.sub(r'\n', r' ', text)
      text = re.sub(r'(?<=.)(?=[\.\";\,\:\[\]\(\)!&?])',r' ', text)
      text = re.sub(r'(?<=[\.\";\,\:\[\]\(\)!&?])(?=.)',r' ', text)
      text = re.sub(r'\s+',r' ', text)
      text = str.split(text)
  
      return text
docs: Tokenizes MHG text