Representation Graphs and Mathematical Reasoning: A Comparative Case Study of Human and Large Language Model Solution Trajectories in Algebra and Geometry
DOI:
https://doi.org/10.68024/v5v4xr64Keywords:
Algebra, Geometry, Large Language Models, Mathematical Reasoning, Representation Graphs, Solution TrajectoriesAbstract
In this work, representation graphs are explored and mathematical reasoning is analyzed by comparing the solution trajectories of human learners and large language models (LLM) in both algebra and geometry. The comparative case study design was suggested to examine the interpretation and representation of mathematical problems, as well as the ways in which the problems were solved with equations, calculation, diagrams, and verbal explanations. The proposed sample was composed of 60 students from University and three LLMs. There were 20 mathematical problems (10 algebraic and 10 geometric tasks), resulting in up to 1260 attempts in total to solve the problems. Mathematical steps and transitions were represented graphically, and the accuracy of answers, representational diversity, validity of transitions, and errors were quantitatively analyzed. The preliminary results for the human participants were 75.0%, with LLM 1 (85.0%), LLM 2 (80.0%) and LLM 3 (75.0%). Human accuracy made 80.0% and 70.0% in algebra and geometry tasks, respectively, while the LLM 1 achieved accuracy of 90.0% and 80.0% in algebra and geometry tasks, respectively. It was also illustrated that there were differences in the representation frequency, the complexity of the solution and the distribution of errors. The above numerical results presented in this report were not the actual experimental results but were examples of the analytical framework proposed. The study provided the process approach to compare mathematical reasoning other than final answer. It can be used to provide a framework for mathematics instruction, for identifying errors, and for assessing answers generated by Artificial Intelligence. To be a reliable and widely applicable representation graph further empirical research with valid data is required.
Downloads
References
Ahn, J., Verma, R., Lou, R., Liu, D., Zhang, R., & Yin, W. (2024). Large language models for mathematical reasoning: Progresses and challenges. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, 225–237. https://doi.org/10.18653/v1/2024.eacl-srw.17
Ayyıldız Altınbaş, A., Solak, S., & Ertekin, E. (2025). The effect of multiple representations-based and problem-based instruction in linear algebra on pre-service teachers' dimensions of understanding and self-efficacy perceptions. Journal of Mathematics Teacher Education. https://doi.org/10.1007/s10857-025-09696-0
Calais, P., Franco, G., Tang, Z., Nikas, T., Meira, W., Jr., Terzi, E., & Crovella, M. (2025). Disentangling text and math in word problems: Evidence for the bidimensional structure of large language models’ reasoning. Findings of the Association for Computational Linguistics: ACL 2025. https://aclanthology.org/2025.findings-acl.656/
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., & Schulman, J. (2021). Training verifiers to solve math word problems. arXiv. https://arxiv.org/abs/2110.14168
Davoodi, A. G., Davoudi, S. P. M., & Pezeshkpour, P. (2025). LLMs are not intelligent thinkers: Introducing mathematical topic tree benchmark for comprehensive evaluation of LLMs. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. https://doi.org/10.18653/v1/2025.naacl-long.161
Desai, S., Bush, S. B., & Safi, F. (2021). Mathematical representations in the teaching and learning of geometry: A review of the literature from the United States. Electronic Journal for Research in Science & Mathematics Education, 25(4), 6–22.
Gholami Davoodi, A., Mousavi Davoudi, S. P., & Pezeshkpour, P. (2025). LLMs are not intelligent thinkers: Introducing mathematical topic tree benchmark for comprehensive evaluation of LLMs. Association for Computational Linguistics.
Jitendra, A. K., Dougherty, B., Sanchez, V., & Suchilt, L. (2024). Using multiple representations to foster multiplicative reasoning in students with mathematics learning disabilities. Teaching Exceptional Children, 57(2). https://doi.org/10.1177/00400599221115627
Li, C., Wang, W., Hu, J., Wei, Y., Han, H., Zhang, Z., & Peng, H. (2024). Common 7B language models already possess strong math capabilities. arXiv. https://arxiv.org/abs/2403.04706
Li, X., Wang, W., Li, M., Guo, J., Zhang, Y., & Feng, F. (2024). Evaluating mathematical reasoning of large language models: A focus on error identification and correction. Findings of the Association for Computational Linguistics: ACL 2024, 11316–11360. https://doi.org/10.18653/v1/2024.findings-acl.673
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K. (2023). Let's verify step by step. arXiv. https://arxiv.org/abs/2305.20050
Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K.-W., Galley, M., & Gao, J. (2024). MathVista: Evaluating mathematical reasoning of foundation models in visual contexts. International Conference on Learning Representations. https://arxiv.org/abs/2310.02255
Mirzadeh, I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., & Farajtabar, M. (2024). GSM-Symbolic: Understanding the limitations of mathematical reasoning in large language models. arXiv. https://arxiv.org/abs/2410.05229
Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Zhang, M., Li, Y. K., Wu, Y., & Gao, M. (2024). DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv. https://arxiv.org/abs/2402.03300
Singh, J., Nambi, A., & Vineet, V. (2024). Exposing the Achilles’ heel: Evaluating LLMs ability to handle mistakes in mathematical reasoning. Microsoft Research.
Wang, P.-Y., Liu, T.-S., Wang, C., Wang, Y.-D., Yan, S., Jia, C.-X., Liu, X.-H., Chen, X.-W., Xu, J.-C., Li, Z., & Yu, Y. (2025). A survey on large language models for mathematical reasoning. arXiv. https://arxiv.org/abs/2506.08446
Yan, Y., Su, J., He, J., Fu, F., Zheng, X., Lyu, Y., Wang, K., Wang, S., Wen, Q., & Hu, X. (2025). A survey of mathematical reasoning in the era of multimodal large language model: Benchmark, method & challenges. Findings of the Association for Computational Linguistics: ACL 2025, 11798–11827. https://doi.org/10.18653/v1/2025.findings-acl.614
Zhang, R., Jiang, D., Zhang, Y., Lin, H., Guo, Z., Qiu, P., Zhou, A., Lu, P., Chang, K.-W., Gao, P., & Li, H. (2024). MathVerse: Does your multi-modal LLM truly see the diagrams in visual math problems? arXiv. https://arxiv.org/abs/2403.14624
Zhang, Y., Xue, M., Liu, D., & He, Z. (2024). Rationales for answers to simple math word problems confuse large language models. Findings of the Association for Computational Linguistics: ACL 2024, 8853–8869. https://doi.org/10.18653/v1/2024.findings-acl.524
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Ayesha Masood, Samia Ashraf (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain the copyright and full publishing rights of their work. All articles published in Scholar Insight Journal are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
Under this license, readers are permitted to read, download, copy, distribute, print, search, link to the full texts, and reuse the published articles for any lawful purpose, including adaptation and commercial use, provided that appropriate credit is given to the original author(s), the source of publication, and the license terms are properly acknowledged.
The journal retains the right of first publication while authors maintain ownership of their intellectual property rights without restrictions.
