Where Free AI Wins on Nuance, But Fails on the Final Number

Have you spent hours using expensive retirement projection software, only to wonder if a free tool like ChatGPT could do the job just as well? I put AI head-to-head with leading planning software, Boldin, to find out.

I'm always looking for ways to streamline the planning process. I wanted to find the answer to three core retirement questions:

1. How much will health coverage cost between 62 and 65?

2. When should we file for Social Security benefits?

3. How much should we convert in Roth conversions?

For this test, I used a case study of a couple with $2 million in a 401(k)/IRA, retiring at 62, and then compared the speed and quality of answers from ChatGPT against the detailed modules found in Boldin’s planning software. You can see this test in action in my video here.

Round 1: Health Coverage & Complexity

For most pre-retirees, health insurance is the biggest hurdle between early retirement (before 65) and Medicare.

• ChatGPT's Performance: ChatGPT was excellent. It provided a detailed, tailored response, complete with tables and steps, giving the client a clear understanding of how much health coverage would cost in those first few years of retirement.

• Boldin’s Performance: The planning software was disappointing. While it had a dedicated section for expenses and healthcare, it was less dynamic than the AI. It relies on linking out to external articles and resources rather than generating a tailored answer based on the user's situation.

Winner: AI (ChatGPT). ChatGPT provided a more helpful, customized starting framework for understanding early retirement healthcare costs.

Round 2: Optimal Social Security Filing Age

I asked the tools when the clients should file for Social Security benefits.

• AI Consensus: The AI tools were generally good. They recommended delaying benefits past the ACA subsidy eligibility (age 65) and suggested the higher earner delay until 70 if possible.

• ChatGPT Nuance: ChatGPT provided a key insight: you should include Social Security as an asset (like a bond) in your IPS to justify a potentially more aggressive allocation in your liquid portfolio. This aligns with modern, sophisticated planning.

• Software's Limitations: The software provided general guidance but, again, pointed to external resources instead of directly answering the client’s specific "When should I file?" question based on their inputs.

Winner: AI (ChatGPT). The ability of the AI to take the Social Security as a bond equivalent approach provides a valuable strategic insight that the traditional software did not offer directly.

Round 3: Roth Conversion Strategies

The final, high-stakes question: How much should be converted into Roth accounts to minimize lifetime taxes?

• ChatGPT's Framework: ChatGPT gave a very strong response, outlining principles like minimizing lifetime taxes, shrinking RMDs, controlling Medicare surcharges (IRMAA), and coordinating QCDs (Qualified Charitable Distributions) after age 70.

• Boldin's Performance: The professional software has a dedicated, pre-built Roth Conversion module within its "Explorer" tab. It allows users to run four pre-built strategies (hitting a certain tax bracket, maximizing estate value, etc.) against real-world tax tables. This level of sophisticated, multi-scenario analysis is currently beyond the capability of generic AI.

Winner: Professional Software (Boldin). The dedicated module for scenario testing, while complex, is essential for generating precise, multi-year, tax-optimal strategies.

The Ultimate Flaw: Clarity and Visualization

Despite the sophistication shown by the AI, the biggest weakness emerged when attempting to visualize and quantify the plan:

• Misleading Defaults: Many planning tools, by default, select Future Dollars instead of Present Dollars. This makes the ending portfolio balance look inflated by millions, leading to false confidence.

• Failed Visualization: The AI struggled to create helpful output, providing vague graphs with confusing scales and unformatted bullet points for key projections. When asked to run a stress test (low-growth scenario), the AI simply replied: "It seems like I can't do more advanced data analysis right now. Please try again later."

The Bottom Line: Free AI offers a great starting point for DIY investors, providing tailored frameworks and complex rationale that beat the generic help files of professional tools. However, when it comes to accurate, verifiable, stress-tested, and quantified analysis—the things that actually protect your retirement—professional software and, more importantly, a human advisor remain essential.

At Barrett FP LLC, we offer expert financial planning on an hourly basis, focused entirely on helping you achieve your goals.

Learn more about how we can help you integrate AI tools while ensuring your plan is fully stress-tested and accurate.