@inproceedings{guo-etal-2023-comave,
    title = "{C}o{M}ave: Contrastive Pre-training with Multi-scale Masking for Attribute Value Extraction",
    author = "Guo, Xinnan  and
      Deng, Wentao  and
      Chen, Yongrui  and
      Li, Yang  and
      Zhou, Mengdi  and
      Qi, Guilin  and
      Wu, Tianxing  and
      Yang, Dong  and
      Wang, Liubin  and
      Pan, Yong",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.373",
    doi = "10.18653/v1/2023.findings-acl.373",
    pages = "6007--6018",
    abstract = "Attribute Value Extraction (AVE) aims to automatically obtain attribute value pairs from product descriptions to aid e-commerce. Despite the progressive performance of existing approaches in e-commerce platforms, they still suffer from two challenges: 1) difficulty in identifying values at different scales simultaneously; 2) easy confusion by some highly similar fine-grained attributes. This paper proposes a pre-training technique for AVE to address these issues. In particular, we first improve the conventional token-level masking strategy, guiding the language model to understand multi-scale values by recovering spans at the phrase and sentence level. Second, we apply clustering to build a challenging negative set for each example and design a pre-training objective based on contrastive learning to force the model to discriminate similar attributes. Comprehensive experiments show that our solution provides a significant improvement over traditional pre-trained models in the AVE task, and achieves state-of-the-art on four benchmarks.",
}

Attribute Value Extraction (AVE) aims to automatically obtain attribute value pairs from product descriptions to aid e-commerce. Despite the progressive performance of existing approaches in e-commerce platforms, they still suffer from two challenges: 1) difficulty in identifying values at different scales simultaneously; 2) easy confusion by some highly similar fine-grained attributes. This paper proposes a pre-training technique for AVE to address these issues. In particular, we first improve the conventional token-level masking strategy, guiding the language model to understand multi-scale values by recovering spans at the phrase and sentence level. Second, we apply clustering to build a challenging negative set for each example and design a pre-training objective based on contrastive learning to force the model to discriminate similar attributes. Comprehensive experiments show that our solution provides a significant improvement over traditional pre-trained models in the AVE task, and achieves state-of-the-art on four benchmarks.
Guo, Xinnan Deng, Wentao Chen, Yongrui Li, Yang Zhou, Mengdi Qi, Guilin Wu, Tianxing Yang, Dong Wang, Liubin Pan, Yong
CoMave: Contrastive Pre-training with Multi-scale Masking for Attribute Value Extraction