acl-long.233
CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending
Zhu, Shiyi Ye, Jing Jiang, Wei Xue, Siqiao Zhang, Qi Wu, Yifan Li, Jianguo