Toggle Categories

Why You Should Always Verify Code Written by AI: A Case Study with ChatGPT

August 01, 2024

In software development, precision and accuracy are essential. Although AI tools such as ChatGPT can help generate code snippets, they are not perfect. This article examines an example to underscore the importance of always verifying AI-generated code, focusing on a case involving an infinite loop.

The Scenario: Infinite Loop Example

Consider the following PHP code snippet intended to generate a unique verification code:

$flag = 0;
do {
	$number = rand(1000, 9999);
	// Check if this number exists in the database
	$sql = "SELECT number FROM table WHERE number = '". $number ."'";
	$sql_result = mysqli_query($con, $sql);
	if (mysqli_num_rows($sql_result) == 0) {
		$flag = 1;
	}
} while ($flag = 0);

Analysis of the Infinite Loop

In the above code, the `while` condition uses the assignment operator (`=`) instead of the comparison operator (`==`). This results in an infinite loop because the assignment `$flag = 0` always returns `0`, which is `false`, making the loop condition perpetually true. Thus, the loop will never terminate.

Why Did ChatGPT Make This Mistake?

ChatGPT, like other AI models, generates responses based on patterns it has learned from vast amounts of text data. However, it lacks the understanding and reasoning capabilities of a human programmer. Here are a few reasons why such mistakes can occur:

  1. Lack of Contextual Understanding: AI does not truly understand code. It predicts the next token based on learned patterns without comprehending the logic.
  2. Ambiguity in Language: Programming languages often have subtle nuances. A small mistake, such as using `=` instead of `==`, can lead to significant issues, which an AI might not always catch.
  3. No Execution Capability: AI models do not execute the code they generate. They cannot test or debug their output, meaning they might produce syntactically correct but logically flawed code.
  4. Training Data Limitations: AI is trained on a mixture of correct and incorrect code examples. Without the ability to discern the quality of the source material, it can inadvertently replicate mistakes.

The Importance of Verifying AI-Generated Code

Given the potential for errors, it is crucial to always verify and test code generated by AI tools. Here are a few steps to ensure correctness:

  1. Manual Review: Carefully read through the generated code to check for logical errors and ensure it aligns with your intended functionality.
  2. Testing: Run the code in a controlled environment to identify and fix any issues. Automated tests can be particularly useful.
  3. Code Reviews: If possible, have another developer review the AI-generated code. A second pair of eyes can often spot mistakes that you might miss.
  4. Static Analysis Tools: Utilize tools that can analyze your code for common mistakes and potential bugs.

While AI tools like ChatGPT can be incredibly useful for generating code snippets and providing programming assistance, they are not a substitute for human expertise and scrutiny. Always verify and test AI-generated code to ensure its correctness and functionality. By doing so, you can leverage the strengths of AI while mitigating the risks associated with its limitations.

Remember, the responsibility for the final code always lies with you, the developer. Use AI as a tool, not a crutch, and always prioritize thorough verification and testing in your development process.