Not logged in.
Quick Search - Contribution
Contribution Details
| Type | Master's Thesis |
| Scope | Discipline-based scholarship |
| Title | Negative Sample Generation for Open-set Text-based Intent Recognition |
| Organization Unit | |
| Authors |
|
| Supervisors |
|
| Language |
|
| Institution | University of Zurich |
| Faculty | Faculty of Business, Economics and Informatics |
| Date | 2023 |
| Abstract Text | A fundamental task in many modern task-oriented dialogue systems is intent classification, in which the user's text input is mapped to a predefined intent. However, task-oriented dialog systems support a limited number of intents, and a key challenge they face is to reject unknown intents. Open-set recognition aims to solve this problem of classifying known classes correctly and rejecting the unknown. One way to train models to reject unknowns is to include representatives of unknown classes during training, called negative samples. In this thesis, we propose several approaches for synthetic negative sample generation to improve model performance on open-set recognition. We first extend the Manifold Mixup approach with different sample selection strategies and apply it to different layers of the network. We also propose using adversarial text attack samples as another source of negative samples. In addition, we apply Entropic Open-Set (EOS) loss function that was shown to improve open-set recognition performance on images. Our experiments compare these approaches with baseline approaches using Open-set Classification Rate (OSCR) curve that was proposed specifically for the open-set recognition task. Our results show that negative samples from adversarial attacks on text could be effective for open-set recognition in certain scenarios. On the other hand, Manifold Mixup-based approaches, including a state-of-the-art approach, are on par with the baselines considering the trade-off between correctly classifying known samples and rejecting unknown samples. |
| PDF File |
Download
|
| Export |
BibTeX
|