Can Language Models Behave Like Wine Sommeliers? – Using Multiple Agents To Evaluate The Quality of Wine Descriptors Generated By Llama 2
- Wines are complex beverages whose taste can be described either numerically or textually, with the former involving the rating of the intensities of different aroma characteristics often with the help of a wine tasting wheel, and the latter with the help of crisp terms often in a poetic fashion. These are often done with the help of wine sommeliers who with one sniff can describe the wine. Usually, each sommelier has a unique style when it comes to textually describing a wine, research has shown that such differences have no negative impact in correctly classifying wines on the basis of their color, grape variety, region etc. Given the recent advancements in the field of Natural Language Processing, especially with the emergence of Large Language Models, we aim to check the capability of Llama 2 in its ability to generate texts pertaining to a specific color of a wine, given a list of aroma intensities as input prompts. In our experiments, we relied on data from Meininger and Falstaff, and on a combination of domain adaptation and pseudo-labeling techniques to create the corpus to train the Llama 2 model on. Also, we relied on a voting scheme of three differently trained classifiers to evaluate the wine-color specific text generation capabilities of Llama 2. Additionally, we employed the services of domain experts to evaluate the quality of a sample set of texts that was generated by Llama 2.
Author: | Siddarth Venkateswaran, Ronald Böck |
---|---|
URN: | urn:nbn:de:bvb:898-opus4-70911 |
DOI: | https://doi.org/10.35096/othr/pub-7091 |
ISBN: | 978-3-95908-325-6 |
Parent Title (German): | Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 |
Publisher: | TUDpress |
Place of publication: | Dresden |
Editor: | Timo Baumann |
Document Type: | conference proceeding (article) |
Language: | English |
Year of first Publication: | 2024 |
Publishing Institution: | Ostbayerische Technische Hochschule Regensburg |
Release Date: | 2024/03/08 |
First Page: | 141 |
Last Page: | 148 |
Andere Schriftenreihe: | Studientexte zur Sprachkommunikation ; 107 |
Institutes: | Fakultät Informatik und Mathematik |
research focus: | Information und Kommunikation |
Licence (German): | Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG |