[1Yin-A] Poster 1

[1Yin-A-20] NLData2Opt: Benchmarking Parameter Derivation and Prediction for Modeling Optimization Problems from Natural Language and Data

〇Lijia Liu¹, Kyohei Atarashi¹, Jiyi Li², Koh Takeuchi¹, Shunji Umetani³, Hisashi Kashima¹ (1. Kyoto University, 2. Hokkaido University, 3. Recruit Co., Ltd.)

Keywords:

large language model,optimization modeling

Modeling real-world problems as optimization tasks is challenging, requiring deep domain expertise. Recent works employ Large Language Models (LLMs) to support optimization modeling from natural language, but existing benchmarks assume well-structured inputs with complete problem descriptions and data. In practice, correcting deficiencies in the problem description and data is often necessary. We introduce NLData2Opt, a benchmark that requires LLMs to perform multiple reasoning stages: (1) extract parameters from data separated from the problem description, (2) compute missing parameters derived by transforming data attributes, and (3) predict unavailable future values from historical observations. We evaluate state-of-the-art methods and eight open-weight and proprietary LLMs using direct prompting. Results show substantial performance decline relative to existing benchmarks, indicating that benchmark progress does not directly translate to robustness under realistic data conditions.