First Thoughts on Forecasting AI Agents

Democratize -> Delegate

I’ve worked on machine learning (ML) forecasting for most of my career. The biggest opportunity I’ve found to scale the impact of machine learning was to democratize the technology to an average person with no technical or coding background. Each user could become their own citizen data scientist. But now I’m starting to rethink that. Instead of trying to teach humans how to use AI/ML tools, how can we teach AI to use these tools? Shouldn’t AI be the best at using AI? I think so. This shifts my initial idea of democratizing to now delegating the task of creating ML forecasts to AI agents who can work on them 24/7. Having this full time virtual data scientist working for humans will have profound effects on the future of forecasting.

Why Forecasting with Machine Learning is Hard in Finance

Before we discuss the promise of new AI agents, let’s review why the current state of democratizing is hard to do. Most of the roadblocks to ML adoption end up in two buckets.

Not Enough Time: Finance people are too busy doing their current jobs to learn how to do it any other way. They don’t have time to learn to use new tools. No time to analyze the ML output alongside their traditional forecast methods. Even if they wanted to use the democratized ML tools (or even have a data science do it for them) the process of iterating to find the best combination of data inputs and ML techniques takes up too much time.
Black Box: There’s a certain leap of faith that needs to happen when using ML. Before with traditional excel models, a finance person could trace cell by cell how a specific forecast number was created. This goes out the window with ML. Making it harder to explain the forecasts to business partners and be accountable if the ML forecast is inaccurate. Either the finance person needs to be technical enough to explain the ML process and output or they need to come to someone like me to “read the ML tea leaves” for them to help understand the forecast output.

Can’t LLMs Create Forecasts?

At this point, you might be thinking to yourself “I thought everyone is saying these new large language models (LLM) from places like OpenAI are close to superintelligence, can’t they just create the forecast for us?”. LLMs are good at text, not numbers. You cannot simply give them historical revenue data and expect them to output a sophisticated forecast. It will not work with today’s models. There are some companies like Nixtla who have created generative AI models that are specific to time series that are worth looking into, but we’re still in the early days of these models.

Instead of training humans to use democratized ML tools, can we train AI to use the tools? This is where AI agents come into play. Think of them as LLMs who can call various tools and APIs on their own autonomously to accomplish a goal set by a human. In a sense they are like hiring a contractor or intern to go work on something for you, maybe check in from time to time, and continue to work for days or weeks until they finish the task.

Are AI Agents The Answer?

AI agents that can create forecasts will be like having a virtual data scientist right out of the box ready to go 24/7. Every finance person can have an agent working for them around the clock. Constantly iterating and updating their forecasts to make them more accurate and easier to understand. Agents will solve both of the biggest challenges in adopting ML forecasts. They fix the lack of time problem by being able to run 24/7 and at scale across the cloud. They also can help fix the black box problem by being able to easily explain any ML output in plain english to a user. This “chat with your forecast” capability will help teach finance users how ML works and help build their “ML intuition”. Let’s see what an AI forecast agent can actually do.

What Can an AI Forecast Agent Do?

Here are all of the “skills” or “workflows” I imagine an AI agent being able to do for finance users. Some require connecting to other agents, while others are self-contained within the realm of the forecast agent.

Research
- Being able to find and pull data to use in the ML forecast process. This is most likely done through a separate agent that specializes in this. For example, having an internal data agent in your company that allows other agents to connect to it and pull financial data from your core reporting systems.
- Run analysis on potential things that are impacting your business, and therefore your forecast. For example, understanding the current state of tariffs, and how their impact should be reflected in the ML forecast. Either by adding input data around tariffs or applying some sort of adjustment after the ML forecast process runs.
Data Formatting
- Once you have some data you want to use to produce a ML forecast, transform it in the right ways to make it easy for the ML forecast process to learn from the data and produce an accurate forecast.
Exploratory Data Analysis
- Teach the LLM about your data set. Run all of the classic EDA techniques and provide that information to the AI agent so it knows about the trends and patterns in your historical data.
Forecast Iteration
- Allow the AI agent to start running ML forecasts on your behalf. This is where it connects to existing AutoML like tools where it can kick off new ML runs, analyze the forecast accuracy, and make tweaks to the data or ML inputs to get an even better forecast. The user can give the agent constraints around the maximum amount of runs it can do and the end goal of a forecast error it wants the agent to obtain.
- Combining EDA with iteration is where the magic happens. This is where the agent can work 24/7 on constantly trying to improve the forecast and find new ways to make it even more accurate. This is how you get scalable impact by having a full time virtual data scientist working for you nonstop.
- Most AutoML systems will optimize all time series at once. An AI agent can go one time series at a time, knowing exactly what inputs and ML parameters to use to get the maximum accuracy. Even in complex hiearchical forecast processes.
Forecast Update
- Once you have that optimized forecast from skill #4, you might need to create updated forecasts every month going forward. The agent can now take new data and combine with what created the best forecasts previously to update new forecasts on the fly. This kind of memory helps save time and resources by not needing to run the whole AutoML forecast process from scratch each time you need a new forecast.
- Over time, inputs that initially created the most accurate forecast initially may now longer be relevant in creating the most accurate forecast going forward. The agent will be able to recognize when these forecasts deviate from historical accuracy and know when to properly retrain or even go back to the drawing board and try new forecast iterations to get back to the proper level of accuracy.
Forecast Explanation
- Users can ask the agent questions about the forecast and get answers back in plain english. The agent can rely on learnings from the EDA process, previous forecast iterations, and its research capabilities to answer any question.
- For tougher analysis questions asked by users, the agent will be able to write and execute code on the fly to return tables and charts that give greater insight into the forecast.
- This kind of QnA capability allows finance users to increase their understanding of the ML forecast output, which increases their trust in ML. Over time as this trust builds they will want to use the ML output even more, since now they know it’s accurate and are able to explain the outputs to their business partners.
Forecast Transformation
- After the final ML forecast is created, users can then instruct the agent to allocate it down to even more granular levels. For example a ML forecast might be created at a worldwide level, then be allocated down to a country level. All done intelligently and on the fly by the agent.
- Users can also make high level manual adjustments to the forecast and the agent will correct apply them. For example a user may want to adjust a revenue forecast by 10 million, and the agent can take that amount and spread it correctly to each country based on predetermined seasonality or any other technique asked by the user.
Scenario Analysis
- Being able to ask “what if” questions about a ML forecast has been tough to do historically. It requires changes to the input data and feature engineering process to get the final outputs. But with an agent this all becomes a simple ask away. The agent will be able to adjust the data and re-run the ML forecast process based on the scenarios asked by the user and easily explain the changes in the forecast that arise.
Variance Analysis
- If LLMs and agents can help us look into the future with forecasting, they can definitely help us look into the past and figure out the reasons why our forecast was different than what actually happened in the business for a specific quarter. Using ML to create future forecasts makes it easier to run variance analysis because everything is documented for the AI agent to sift through and understand. It knows exactly what historical data was used, what other ML params were chosen, and what the final ML models were that produced the forecast. It can fill in it’s gaps with its research skills to understand other internal and external factors that might have impacted the business as well. Making it easy to spot and explain forecast variances at scale.

Final Thoughts

Eventually all of us will be AI agent managers. Being able to to build and wield these agents will become the ultimate source of leverage in business. Agents applied in the forecasting space have so much potential to help finance teams better plan and make decisions about the future. They will be able to work faster and do kinds of work never thought possible before.

I’m currently working on building AI agent capabilities directly into my automated ML forecast framework called Finn. It’s completely open-source and free to use. Stay tuned for more updates if you’d like to use it!