AI Chatbot ChatGPT Fails UPSC Prelims Exam with 30% Despite Prior Success in Other Challenging Tests
In recent times, ChatGPT has gained popularity as an AI chatbot with vast knowledge. Launched in November 2022, ChatGPT has successfully passed some of the world's most challenging exams, such as the MBA programme's final exam at the Wharton School of the University of Pennsylvania and the US medical exam.
The latest exam that ChatGPT was asked to attempt was the Indian Civil Services Examination, considered one of the most difficult exams globally, arranged by the Union Public Service Commission (UPSC). Analytics India Magazine (AIM) played a role in ChatGPT attempting the UPSC exam. When the chatbot was asked if it believed it could pass the UPSC prelim exam, it acknowledged the difficulty level and could not provide a definite answer.
In an attempt to gauge its ability to pass the UPSC Prelims 2022, Analytics India Magazine (AIM) posed all 100 questions from Question Paper 1 (Set A) to ChatGPT.
However, the AI chatbot could only answer 54 questions correctly, surprising the magazine company. As the cutoff for general category candidates was 87.54, it was clear that ChatGPT failed the UPSC examination. The questions covered various topics including Geography, Economy, History, Ecology, General Science, and current affairs related to India. Due to its knowledge cutoff in September 2021, ChatGPT was unable to answer current affairs questions, and it also provided incorrect responses to some questions on Geography and Economy, despite not having a specific timeline.
Despite claiming to have all knowledge up to September 2021, ChatGPT even provided an incorrect response to a History question, which should have been answered correctly. Additionally, in some multiple-choice questions, the AI chatbot created an extra option labeled "None of the above," which was not included in the original question. The UPSC examination is regarded as one of the world's most challenging tests, with 11-12 lakh candidates taking the exam annually, but only 5% of them qualify for the mains stage.