Citation :

Rudi Simanjuntak, Hendra Gunawan, 2026. "Cross-Modal Economic Reasoning for Integrating Financial Text, Numerical Data, and Market Signals" ESP International Journal of Science, Humanities & Management Studies(ESP-IJSHMS)  Volume 4, Issue 3: 42-54.

Abstract :

Financial decision-making relies more and more on heterogeneous information, dispersed in financial reports, news articles, numerical indicators or time series such as price movements of stocks (stock prices), trading volumes and even planetary signals. Traditional analytic systems typically analyze these modalities separately, thus are incapable of discovering the connection between qualitative stories in economics and quantitative market activity. We propose a cross-modal economic reasoning network integrating financial text, numbers and market signals within the same analytical framework. We construct a model combining domain-aware language representations, temporal numerical encoders as well as market-signal extraction and cross-modal attention mechanisms which aims to capture relations among textual events, financial metrics with the dynamics of markets on them. Instead of considering different modalities as separate predictive variables, the framework focuses on temporal alignment, semantic correspondence between representations and causal modeling with evidence-weighted fusion Recent work has shown that including both news and time-series information can enhance multimodal financial models, yet ongoing issues remain such as (1) temporal misalignment between modalities from the same data record; (2) modality imbalance in historical data collection for non-textual fields (e.g. sentiment); noisy/null modal inputs across records due to original channel sources/text parser errors; limited generalizability of model learned just using one market type genre to another unrelated domain/market type source only on flavor impetus eXchange application — we build bridges but need better ones! As a result, we propose that the framework incorporates modality-specific processing first and then uses adaptive cross-modal reasoning to enable the model to identify supporting, disagreeing and contextual signals. It provides a conceptual basis for more interpretable financial forecasting, economic event interpretation, risk mitigation and multi-reasoning demand decision-support systems over heterogeneous types of finance information.

References :

[1] Abaci, D. (2019) ‘Fin BERT: Financial sentiment analysis with pre-trained language models’, arrive preprint

[2] Oval, R., Andrews, N. and Yan, X. (2024) ‘Financial forecasting from textual and tabular time series’, Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 8289–8300.

[3] Oval, R., Andrews, N. and Yan, X. (2025) ‘Multimodal language models for financial forecasting from interleaved sequences of text and time series’, Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, pp. 987–1001.

[4] D’Amico, S., Mercuric, F., Nubian, N., Peril, G. and Venter, C. (2026) ‘Learning across modalities: A systematic survey of multimodal models for financial analysis’, Information Fusion, 133, 104249.

[5] Yang, H., Liu, X.-Y. and Wang, C.D. (2023) ‘Fight: Open-source financial large language models’, arrive preprint arXiv:2306.06031. (arrive)

[6] Zhang, B., Yang, H. and Liu, X.-Y. (2023) ‘Instruct-Fight: Financial sentiment analysis by instruction tuning of general-purpose large language models’, Film Symposium at IJCAI 2023.

[7] Zhang, B., Yang, H., Zhou, T., Babar, M.A. and Liu, X.-Y. (2023) ‘Enhancing financial sentiment analysis via retrieval augmented large language models’, Proceedings of the ACM International Conference on AI in Finance.

[8] Zhang, Y., Dong, Z. and Cu, W. (2026) ‘Integrative stock price trend prediction via hierarchical LLM text processing and patch-based transformer with co-attention’, Expert Systems with Applications, 302, 130441.

[9] [Authors] (2025) ‘MM-transformer: A multimodal approach to economic time series forecasting with textual data’, Applied Sciences, 15(3), 1241.

[10] [Authors] (2026) ‘Multimodal information fusion for financial forecasting via cross-attention and calibrated uncertainty’, Machine Learning with Applications, 23, 100840. (Science Direct)

[11] [Authors] (2026) ‘Decoding market dynamics with financial knowledge graphs and cross-modal transformers’, Information Sciences, article 124031. (Science Direct)

[12] Name, R., Badness, A. and Talia, M. (2026) ‘A hybrid Prophet-based framework for multimodal forecasting with market sentiment signals’, Discover Artificial Intelligence, 6, 383.

[13] Aswan, A., Shaker, N., Parma, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I. (2017) ‘Attention is all you need’, Advances in Neural Information Processing Systems, 30.

[14] Devlin, J., Chang, M.-W., Lee, K. and Tout nova, K. (2019) ‘BERT: Pre-training of deep bidirectional transformers for language understanding’, Proceedings of NAACL-HLT, pp. 4171–4186.

[15] Brown, T.B., Mann, B., Ryder, N., Cubbish, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A. et al. (2020) ‘Language models are few-shot learners’, Advances in Neural Information Processing Systems, 33, pp. 1877–1901.

[16] Dosovitskiy, A., Beyer, L., Kalashnikov, A., Weissenborn, D., Zhao, X., Entertainer, T., Dehghani, M. et al. (2021) ‘An image is worth 16×16 words: Transformers for image recognition at scale’, International Conference on Learning Representations.

[17] Tsai, Y.-H.H., Bay, S., Yamada, M., Morenci, L.-P. and Salakhutdinov, R. (2019) ‘Multimodal transformer for unaligned multimodal language sequences’, Proceedings of the Annual Meeting of the Association for Computational Linguistics, pp. 6558–6569.

[18] Baltrušaitis, T., Abuja, C. and Morenci, L.-P. (2019) ‘Multimodal machine learning: A survey and taxonomy’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2), pp. 423–443.

[19] Goo, M., Cu, H., Cu, Y., Wang, Y., Wei, R., Ma, J., Zhang, Y. et al. (2023) ‘Large language models are zero-shot reasons’, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.

[20] Lim, B., Ark, S.Ö., Loaf, N. and Pfizer, T. (2021) ‘Temporal fusion transformers for interpretable multi-horizon time series forecasting’, International Journal of Forecasting, 37(4), pp. 1748–1764.

Keywords :

Cross-Modal Reasoning, Financial Text and Numerical Data, Market Signal Extraction, Multimodal Learning Methods, Economic Forecasting, Financial Analytics, Temporal Alignment of Market Signals, Attention Score Computation, Risk Assessment.