I Thought I Knew My Business. AI Proved Otherwise

The ability to analyze nearly a million rows of raw data has exploded some of my core assumptions.

If you arrange financing for small business owners, you should care what happens after the loan closes. That sounds obvious. But in the brokerage business, it’s surprisingly difficult to know. Many brokers focus on getting a deal approved. They collect a commission and move on. I’ve always wanted to know more. Which borrowers succeed? Which struggle? Are we helping clients obtain financing that positions them for long-term success?

The challenge has always been access to information. Lenders usually don’t share loan-performance data with brokers. Over the years, I’ve pieced together whatever information I could find. But it never gave me a complete picture. Recently, that changed. Every quarter, the Small Business Administration publishes data on its loan portfolio. The agency recently expanded those reports, creating a much richer dataset. For the first time, I finally had enough information to explore questions I’d been asking for years.

That said, the dataset contained nearly a million rows of information. Not long ago, analyzing something that large would have required consultants, databases, and a major investment of time and money. Instead, I uploaded the data into Claude. I isolated the loans my company originated and matched them to referral sources and channels. Then I started looking for patterns.

What I found surprised me. One referral source I always considered strong was producing loans that performed worse than the rest of our portfolio. At the same time, one of our direct-marketing channels performed better than I expected. Many in my industry assume these channels produce weaker borrowers. That’s what I’d always believed.

Some of the data confirmed my prior beliefs. Other findings challenged my assumptions, forcing me to reconsider how we evaluate various sources of business.

That’s what made the exercise so valuable. The technology helped me process the data, but it didn’t tell me what to think. In fact, the most important part of the project started after the results appeared on the screen. Were the results accurate? Was I missing something? Was the sample size large enough? Were there other explanations for what I was seeing? Most importantly, what should we do differently? Those questions required judgment, not technology.

The analysis has already influenced how we think about our business. We’ll spend more time evaluating referral relationships. We’ll pay closer attention to performance trends. Some decisions may reduce our loan volume—and thus our revenue—in the short term. But if you build a business that you want to last forever, you are willing to sacrifice some short-term profits for long-term health. In my business, this means building the healthiest portfolio of borrowers possible—while understanding there will always be some loans that go bad. 

But those aren’t decisions a machine made. The decisions were based on stronger evidence. After 16 years in business, I thought I understood what worked and what didn’t. Experience still matters. But this exercise reminded me that experience and evidence are not the same thing.  Artificial intelligence is helping me get access to new data that I could never have seen before. But what to do with the information and how to make changes based on it still involves human judgment. And I think that’s a good thing.

For years, many of my questions would have required resources available only to large institutions. Today, tools like Claude let small businesses analyze large amounts of data and challenge their own assumptions.

That’s powerful. Not because the technology replaces judgment, but because it gives business owners a better foundation on which to exercise it.