- Titel
- Enhancing Information Extraction for Simulation Modelling Using Large Language Models
- Zitierfähige Url:
- https://nbn-resolving.org/urn:nbn:de:bsz:14-qucosa2-997869
- Übersetzter Titel (DE)
- Unterstützungen der Informationsextraktion für Simulationsmodellierung durch große Sprachmodelle
- Konferenz
- 21. ASIM-Fachtagung Simulation in Produktion und Logistik. Dresden, Germany, 24. bis 26. September 2025
- Quellenangabe
- Simulation in Produktion und Logistik 2025
Herausgeber: Sebastian Rank
Herausgeber: Mathias Kühn
Herausgeber: Thorsten Schmidt
Erscheinungsort: Dresden
Verlag: Technische Universität Dresden, Professur für Technische Logistik
Erscheinungsjahr: 2025
ISBN: 978-3-86780-806-4 - Erstveröffentlichung
- 2025
- DOI
- https://doi.org/10.25368/2025.282
- Abstract (EN)
- This study investigates the application of Large Language Models (LLMs) for structured information extraction in the context of simulation modelling. Specifically, the performance of Meta-Llama-3-70B-Instruct was evaluated in extracting simulation model components—such as sources, process steps, and parameters—from unstructured textual descriptions of logistics systems. The evaluation compared zero-shot and few-shot prompting strategies and analysed model consistency across 50 repeated runs per prompt configuration. Metrics such as precision, recall, and F1-score were computed using fuzzy string matching. Results show that few-shot prompting improves extraction performance overall, with statistically significant gains in categories such as model parameters and sources. However, inconsistent effects were observed for other categories, including a decrease in key metrics identification. Variability analysis using the coefficient of variation revealed differing levels of output stability across categories and prompts. The findings indicate that few-shot prompting with carefully chosen examples enhances model reliability but may introduce risks of hallucinated outputs and prompt-specific overfitting. Recommendations are provided for prompt engineering in simulation modelling applications. Future work will consider other LLM architectures, dynamic prompt selection, and integration of external validation mechanisms.
- Freie Schlagwörter (DE)
- Simulation, Künstliche Intelligenz, Modellierung, Optimierung, Large Language Models (LLMs), Simulationsmodellierung
- Freie Schlagwörter (EN)
- Simulation, Artificial Intelligence, Modelling, Optimisation, Large Language Models (LLMs), simulation modelling
- Klassifikation (DDC)
- 620
- Klassifikation (RVK)
- ZM 9500
- Publizierende Institution
- Technische Universität Dresden, Dresden
- Förder- / Projektangaben
- Zürcher Hochschule für Angewandte Wissenschaften (ZHAW)
Digital Futures Fund for R&D - Version / Begutachtungsstatus
- publizierte Version / Verlagsversion
- URN Qucosa
- urn:nbn:de:bsz:14-qucosa2-997869
- Veröffentlichungsdatum Qucosa
- 21.10.2025
- Dokumenttyp
- Konferenzbeitrag
- Sprache des Dokumentes
- Englisch
- Lizenz / Rechtehinweis
CC BY-NC-SA 4.0