Version 1
: Received: 21 October 2024 / Approved: 28 October 2024 / Online: 28 October 2024 (12:02:24 CET)
How to cite:
Jimale, E. L.; Wenyu, C.; Al-antari, M. A.; Gu, Y. H.; Agbesi, V. K.; Feroze, W. DecoStrat: Leveraging the Capabilities of Language Models in D2T Generation via Decoding Framework. Preprints2024, 2024102163. https://doi.org/10.20944/preprints202410.2163.v1
Jimale, E. L.; Wenyu, C.; Al-antari, M. A.; Gu, Y. H.; Agbesi, V. K.; Feroze, W. DecoStrat: Leveraging the Capabilities of Language Models in D2T Generation via Decoding Framework. Preprints 2024, 2024102163. https://doi.org/10.20944/preprints202410.2163.v1
Jimale, E. L.; Wenyu, C.; Al-antari, M. A.; Gu, Y. H.; Agbesi, V. K.; Feroze, W. DecoStrat: Leveraging the Capabilities of Language Models in D2T Generation via Decoding Framework. Preprints2024, 2024102163. https://doi.org/10.20944/preprints202410.2163.v1
APA Style
Jimale, E. L., Wenyu, C., Al-antari, M. A., Gu, Y. H., Agbesi, V. K., & Feroze, W. (2024). DecoStrat: Leveraging the Capabilities of Language Models in D2T Generation via Decoding Framework. Preprints. https://doi.org/10.20944/preprints202410.2163.v1
Chicago/Turabian Style
Jimale, E. L., Victor Kwaku Agbesi and Wasif Feroze. 2024 "DecoStrat: Leveraging the Capabilities of Language Models in D2T Generation via Decoding Framework" Preprints. https://doi.org/10.20944/preprints202410.2163.v1
Abstract
Current language models have achieved remarkable success in NLP tasks. Nonetheless, individual decoding methods face difficulties in realizing the immense potential of these models. The challenge is primarily due to the lack of a decoding framework that can integrate language models and decoding methods. We introduce the DecoStrat which bridges the gap between language modeling and the decoding process in D2T generation. By leveraging language models, DecoStrat facilitates the exploration of alternative decoding methods tailored to specific tasks. We fine-tune the model on the MultiWOZ dataset to meet task-specific requirements and employ it to generate output(s) through multiple interactive modules of the framework. The Director module orchestrates the decoding processes, engaging the Generator to produce output(s) text based on the selected decoding method and input data. The Manager module enforces a selection strategy, integrating Ranker and Selector to identify the optimal result. Evaluations on the stated dataset show that DecoStrat effectively produces diverse and accurate output, with MBR variants consistently outperforming other methods. DecoStrat with the T5-small model surpasses some baseline. Generally, the findings highlight DecoStrat's potential for optimizing decoding methods in diverse real-world applications.
Keywords
decoding methods; data-to-text generation (D2T); language models (LM); natural language generation (NLG); natural language processing (NLP)
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
