AI-Generated Code Creates 70% More Problems: What the Study Reveals

Hello HaWkers, a recent study brought thought-provoking data: AI-generated code presents 1.7 times more problems than human-written code. This doesn't mean we should abandon AI tools, but rather understand how to use them effectively.

Have you ever had problems with code suggested by Copilot or ChatGPT? This study helps understand why this happens and what we can do about it.

The Study Numbers

The research conducted by CodeRabbit analyzed thousands of pull requests to compare human code versus AI-assisted code.

Main Results

Comparison of problems found:

Metric	Human Code	AI Code	Difference
Problems per PR	6.45	10.83	+68%
Correctness errors	Base	+45%	More bugs
Maintenance issues	Base	+52%	More tech debt
Vulnerabilities	Base	+31%	More risks
Performance	Base	+38%	Less efficient

💡 Context: The study analyzed PRs in real repositories, comparing commits with and without identified AI assistance.

Why AI Code Has More Problems

There are technical and behavioral reasons for these results:

1. Lack of Complete Context

AI doesn't understand your project as a whole:

Contextual limitations:

• Doesn't know the general architecture
• Ignores team-specific conventions
• Doesn't know about business requirements
• Unaware of decision history

2. Optimization for Appearance

Language models are trained to generate code that looks correct:

Common problems:

• Code works in simple cases
• Fails in edge cases
• Superficial error handling
• Incomplete validations

3. Developer Overconfidence

When we use AI, we tend to review less:

Observed behaviors:

• Accepting suggestions without fully reading
• Assuming generated code is correct
• Skipping tests for AI code
• Less critical questioning

Most Common Problem Types

The study categorized the problems found:

1. Correctness Errors

Bugs that make code not work as expected:

// Example of common AI-generated error
// Problem: doesn't handle empty array case
function findMax(numbers) {
  let max = numbers[0]; // Undefined if array is empty!
  for (let i = 1; i < numbers.length; i++) {
    if (numbers[i] > max) {
      max = numbers[i];
    }
  }
  return max;
}

// Corrected version
function findMaxSafe(numbers) {
  if (!numbers || numbers.length === 0) {
    return undefined; // or throw new Error
  }
  return Math.max(...numbers);
}

2. Security Issues

Vulnerabilities introduced by generated code:

// Vulnerable AI-generated code
// Problem: SQL Injection
function getUserByEmail(email) {
  const query = `SELECT * FROM users WHERE email = '${email}'`;
  return db.query(query);
}

// Secure version
function getUserByEmailSafe(email) {
  const query = 'SELECT * FROM users WHERE email = ?';
  return db.query(query, [email]);
}

3. Inefficient Performance

Solutions that work but are slow:

// Inefficient AI-generated code
// Problem: unnecessary O(n^2)
function removeDuplicates(arr) {
  const result = [];
  for (const item of arr) {
    if (!result.includes(item)) { // includes is O(n)
      result.push(item);
    }
  }
  return result;
}

// Efficient O(n) version
function removeDuplicatesEfficient(arr) {
  return [...new Set(arr)];
}

Categories With Most Problems

The study identified specific areas where AI fails more:

1. Error Handling

AI frequently generates superficial handling:

// Inadequate handling
async function fetchData(url) {
  try {
    const response = await fetch(url);
    return response.json();
  } catch (error) {
    console.log(error); // Just logs and continues
  }
}

// Adequate handling
async function fetchDataProper(url) {
  try {
    const response = await fetch(url);

    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${response.statusText}`);
    }

    return await response.json();
  } catch (error) {
    if (error instanceof TypeError) {
      throw new Error('Network failure. Check your connection.');
    }
    throw error; // Re-throw for caller to handle
  }
}

2. Input Validation

Incomplete validations are common:

// Incomplete validation
function processUser(user) {
  return {
    name: user.name.trim(),
    email: user.email.toLowerCase(),
  };
}

// Complete validation
function processUserSafe(user) {
  if (!user || typeof user !== 'object') {
    throw new TypeError('User must be an object');
  }

  if (!user.name || typeof user.name !== 'string') {
    throw new Error('Name is required and must be a string');
  }

  if (!user.email || !isValidEmail(user.email)) {
    throw new Error('Invalid email');
  }

  return {
    name: user.name.trim(),
    email: user.email.toLowerCase(),
  };
}

function isValidEmail(email) {
  const regex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
  return regex.test(email);
}

3. State Management

Problems with mutation and side effects:

// Problem: mutates original object
function addItem(cart, item) {
  cart.items.push(item); // Mutates the object!
  cart.total += item.price;
  return cart;
}

// Immutable version
function addItemImmutable(cart, item) {
  return {
    ...cart,
    items: [...cart.items, item],
    total: cart.total + item.price,
  };
}

How to Use AI Effectively

The data doesn't mean we should stop using AI, but use it better:

1. Review All Generated Code

Treat AI suggestions as drafts:

Review checklist:

• Does the code handle all edge cases?
• Are there adequate validations?
• Is error handling robust?
• Is performance acceptable?
• Does it follow project conventions?

2. Use AI for Specific Tasks

Some uses are safer:

Good AI uses:

• Repetitive boilerplate
• Format conversion
• Documentation and comments
• Simple unit tests
• Mechanical refactoring

Uses requiring more care:

• Complex business logic
• Security code
• Critical algorithms
• System integrations

3. Provide Adequate Context

The more context, the better the result:

// Vague prompt
// "create a validation function"

// Prompt with context
// "Create a function that validates Brazilian CPF.
// Should return { valid: boolean, error?: string }.
// Consider: formats with and without punctuation,
// check digits, and known invalid CPFs.
// The project uses TypeScript strict mode."

The Role of Code Review

Code review becomes even more important with AI:

1. Focus on Problem Areas

Pay extra attention to:

Risk areas:

• Error handling
• Input validations
• Complex conditional logic
• Resource management
• Database queries

2. Question the Logic

Don't assume it's correct:

Useful questions:

• Why was this approach chosen?
• Are there better alternatives?
• Were all cases considered?
• Is the code testable?

3. Analysis Tools

Complement human review with automation:

Recommended tools:

• ESLint with strict rules
• TypeScript with strict mode
• Static analysis (SonarQube)
• Automated tests
• Security scanning

Productivity Data

Despite the problems, AI still increases productivity:

The Trade-off

Productivity numbers:

• PRs 33% larger with AI
• Lines of code per dev: 4,450 -> 7,839
• 65% of developers use AI weekly
• 25% of big tech code is AI-generated

The Balance

The key is balancing speed with quality:

Recommended strategy:

• Use AI to accelerate initial writing
• Invest time in review
• Maintain high test coverage
• Monitor quality metrics

The Future of AI-Assisted Code

What we can expect:

Expected Improvements

Tool evolution:

• Models with more project context
• Integration with automated tests
• Proactive problem detection
• Integrated security suggestions

What Won't Change

Still necessary skills:

• Critical thinking
• Architecture knowledge
• Security understanding
• Debugging capability
• Code review

New Skills

What to develop:

• Effective prompt engineering
• Critical evaluation of generated code
• AI tool integration
• Verification automation

Practical Recommendations

To use AI effectively day-to-day:

For Individual Developers

Recommended actions:

• Never accept suggestions blindly
• Write tests for generated code
• Use AI as a pair, not a substitute
• Always maintain critical sense

For Teams

Team practices:

• Define AI usage guidelines
• Include AI verification in code review
• Monitor quality metrics
• Share learnings

For Organizations

Organizational strategy:

• Evaluate impact on quality
• Train developers on effective use
• Implement automatic guardrails
• Balance productivity and quality

Conclusion

The study showing AI code has 70% more problems isn't an argument against using AI, but a reminder that we, developers, need to ensure quality. AI tools are powerful for accelerating work, but don't replace human judgment.

The best approach is treating AI as a very productive assistant that makes mistakes. Use it to accelerate, but always review, question, and test. With this mindset, you get productivity benefits without sacrificing quality.

If you want to deepen your knowledge in code best practices, I recommend checking out another article: Discovering the Power of Async/Await in JavaScript where you'll discover how to write quality asynchronous code.

Let's go! 🦅

🎯 Join Developers Who Are Evolving

Thousands of developers already use our material to accelerate their studies and achieve better positions in the market.

Why invest in structured knowledge?

Learning in an organized way with practical examples makes all the difference in your journey as a developer.

Start now:

• 1x of $4.90 on card
• or $4.90 at sight

🚀 Access Complete Guide

"Excellent material for those who want to go deeper!" - John, Developer