Evaluating the Effectiveness of ChatGPT in Solving Math Problems Across Varying Levels of Difficulty and Its Implications on Education
Location
Room 232, Schewel Hall
Access Type
Campus Access Only
Presentation Type
Oral presentation
Entry Number
2321
Start Date
4-16-2025 10:15 AM
End Date
4-16-2025 10:30 AM
School
School of Liberal Arts and Sciences
Department
Mathematics
Keywords
ChatGPT, mathematics, education, problem-solving
Abstract
This project aims to make the literature on LLMs, like ChatGPT, more accessible, while also evaluating its effectiveness in solving math problems of varying difficulty. In order to achieve this, I generated and ranked math problems based on their difficulty and the analysis/steps required to complete them correctly. From there, these problems were inputted into ChatGPT to be solved, and patterns of error were documented in order to see what steps or sections of these problems were conducted incorrectly. Additionally, analysis of this information allowed me to determine the benefits of ChatGPT and what areas it still needs improvement in, in terms of mathematical problem solving. Then, based on my findings, I discussed ChatGPT’s implications on math education, and presented ways to turn it into a “teaching tool” rather than a “cheating tool.” Ultimately, this evaluation is essential because of the popularity the service has gained, especially with students, who should be more knowledgeable about the tool that they rely on to complete tasks.
Primary Faculty Mentor(s)
Dr. Thomas Ales
Primary Faculty Mentor(s) Department
Mathematics Department
Additional Faculty Mentor(s)
Dr. Price Blair (Westover Honors College) Dr. Holly Gould (Education Department)
Rights Statement
The right to download or print any portion of this material is granted by the copyright owner only for personal or educational use. The author/creator retains all proprietary rights, including copyright ownership. Any editing, other reproduction or other use of this material by any means requires the express written permission of the copyright owner. Except as provided above, or for any other use that is allowed by fair use (Title 17, §107 U.S.C.), you may not reproduce, republish, post, transmit or distribute any material from this web site in any physical or digital form without the permission of the copyright owner of the material.
Evaluating the Effectiveness of ChatGPT in Solving Math Problems Across Varying Levels of Difficulty and Its Implications on Education
Room 232, Schewel Hall
This project aims to make the literature on LLMs, like ChatGPT, more accessible, while also evaluating its effectiveness in solving math problems of varying difficulty. In order to achieve this, I generated and ranked math problems based on their difficulty and the analysis/steps required to complete them correctly. From there, these problems were inputted into ChatGPT to be solved, and patterns of error were documented in order to see what steps or sections of these problems were conducted incorrectly. Additionally, analysis of this information allowed me to determine the benefits of ChatGPT and what areas it still needs improvement in, in terms of mathematical problem solving. Then, based on my findings, I discussed ChatGPT’s implications on math education, and presented ways to turn it into a “teaching tool” rather than a “cheating tool.” Ultimately, this evaluation is essential because of the popularity the service has gained, especially with students, who should be more knowledgeable about the tool that they rely on to complete tasks.