Logo image
Anchor and Broadcast: An Efficient Concept Alignment Approach for Evaluation of Semantic Graphs
Conference proceeding

Anchor and Broadcast: An Efficient Concept Alignment Approach for Evaluation of Semantic Graphs

Haibo Sun and Nianwen Xue
PROCEEDINGS OF THE 2024 JOINT INTERNATIONAL CONFERENCE ON COMPUTATIONAL LINGUISTICS, LANGUAGE RESOURCES AND EVALUATION, LREC-COLING 2024, pp.1052-1062
International Conference on Computational Linguistics Language Resources and Evaluation
01/01/2024

Abstract

Computer Science Computer Science, Artificial Intelligence Computer Science, Interdisciplinary Applications Language & Linguistics Linguistics Science & Technology Social Sciences Technology
In this paper, we present AnCast, an intuitive and efficient tool for evaluating graph-based meaning representations (MR). AnCast implements evaluation metrics that are well understood in the NLP community, and they include concept F1, unlabeled relation F1, labeled relation F1, and weighted relation F1. The efficiency of the tool comes from a novel anchor broadcast alignment algorithm that is not subject to the trappings of local maxima. We show through experimental results that the AnCast score is highly correlated with the widely used Smatch score, but its computation takes only about 40% the time.

Metrics

1 Record Views

Details

Logo image