LIT: Label-Informed Transformers on Token-Based Classification - Laboratoire LI, équipe BDTLN
Communication Dans Un Congrès Année : 2024

LIT: Label-Informed Transformers on Token-Based Classification

Résumé

Transformer-based language models have led to the investigation of various embedding and modeling techniques for several downstream natural language processing tasks. Nevertheless, the comprehensive exploration of semantic information about the label from encoder and decoder components in these tasks is yet to be fully realized. In this paper, we propose LIT, an end-to-end pipeline architecture that integrates the transformer’s encoder-decoder mechanism with an additional label semantic to token classification tasks (i.e., historical named entity recognition (NER) and automatic term extraction (ATE)). Our findings demonstrate that LIT outperforms the benchmark in F1 with a maximal rise of 9.5% points in the historical NER task and 11.2% points in the ATE task for the gold standard excluding named entities.
Fichier principal
Vignette du fichier
TPDL___LIT__Label_Informed_Transformers_on_Token__and_Sequence_based_Classification (1).pdf (1.19 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04710036 , version 1 (26-09-2024)

Identifiants

Citer

Wenjun Sun, Hanh Thi Hong Tran, Carlos-Emiliano González-Gallardo, Mickaël Coustaty, Antoine Doucet. LIT: Label-Informed Transformers on Token-Based Classification. The 28th International Conference on Theory and Practice of Digital Libraries, Sep 2024, LJUBLJANA, Slovenia. pp.144-158, ⟨10.1007/978-3-031-72437-4_9⟩. ⟨hal-04710036⟩
19 Consultations
10 Téléchargements

Altmetric

Partager

More