Enterprises are no longer relying on a single generative AI model. Last year, the typical enterprise was already using around 11 generative models. And by 2027 this number is expected to grow by around 50%.
That's according to a survey of executives in US businesses commissioned by the IBM Institute of Business Value (IBM IBV) with Oxford Economics.
But why do companies need a mix of models and how do you optimize that mix for your organization? What do business leaders need to be aware of to make these decisions? This is the subject of an IBM IBV guide: The CEO's Guide to AI Model Optimization.
What model types are being used?
The guide explains the various types of models in the average organization's AI portfolio. This includes open source models like IBM's Granite series and the open offerings from the likes of Mistral AI; proprietary custom-developed enterprise models (models trained by the enterprise themselves); and an increasing number of embedded models that enterprise software vendors such as SAP, Salesforce and Adobe are integrating into their products.
On top of this, there are publicly available commercial models based on large specialized data sets for specific industries (such as the Google Med PaLM for the medical sector) and the popular publicly available large commercial models based on massive data sets (like GPT-o4 from OpenAI).
Why optimize the mix of AI models for your organization?
When selecting models, you don't necessarily need to go for the latest model, the one with the biggest data set, or the one with the newest functionality. Different model types have their specific advantages and disadvantages. There is no such thing as an all-purpose gen AI model suitable for every task.
Optimizing your organization's portfolio to suit your AI use cases is key. While business leaders do not need to know the nitty-gritty of how the models work, their tech teams need to help them understand the importance of investing in the right model for the right business application.
This can impact the overall value the enterprise derives from AI, including the productivity and efficiency improvements it delivers, whether it delivers a true competitive advantage, and the accuracy of its outputs. It will also affect the business's costs and the risks or limitations of using AI.
Why model size and ownership are essential differentiators
Model size versus the type of workflow you are considering is one of the first factors enterprises need to assess when optimizing their AI model portfolio.
Large models trained on hundreds of billions of parameters have greater breadth of knowledge and can handle a broad range of complex tasks out of the box. But they'll come at a high price.
Smaller models, trained on tens of billions of parameters, will cost less and be focused on specific tasks or use cases. They usually work faster and more efficiently. For example, they could be a good fit for applications such as code or language translation.
Ownership is another important factor.
Public commercial models make up around half of an average organization's AI portfolio, according to the IBM IBV research. They're efficient and quick to implement. They run on the public cloud, so companies can start using them sooner.
But if you use one of these, you need to be aware that others will be using the same models trained on the same data, so they're unlikely to deliver a competitive advantage. They're also not going to provide the control and privacy levels required for mission-critical applications or for organizations that need to prioritize protecting confidential data.
Proprietary gen AI models are internally developed and owned by the enterprise. So the organization has control over the data used in model training, meaning there's less risk of 'data pollution', such as AI outputs raising IP or copyright issues. And of course there is more control over where to run the model and where the data is located. Because they can be run on-premises or in private cloud setups, sensitive and confidential data can be protected and managed in compliance with regulations.
Open-source generative AI models, whether large or small, are more transparent about their data training. Because they are tested and scrutinized by the open-source community, they are considered less likely to infringe on IP or copyright. Plus having more eyes reviewing the code means security vulnerabilities, biases and errors can be spotted and fixed faster - and the fact that anyone can inspect the underlying datasets creates more trust. On top of this companies are free to customize the base model to their own needs, so there is potential for competitive differentiation.
Enterprise software that uses integrated AI models provides added value for those specific use cases - whether that's ERP, CRM or design - but obviously, you can't use the embedded models outside of those areas.
Balancing cost and performance
Business leaders recognize they need gen AI, but 63% cite cost and 58% cite model complexity as top concerns.
Costs can vary depending on the model used. Larger models will have higher compute and data storage costs and higher cloud bills. Plus, they may need more frequent updates, fine-tuning and maintenance - as well as the skills and expertise to do this work.
Smaller niche models will incur lower compute and storage costs and they will usually be faster to deploy with lower maintenance costs. They will have lower energy consumption with a smaller environmental footprint.
Picking the right model for the task helps optimize cost. Use larger, more expensive models when you have complex use cases and a need for multiple skill sets and higher accuracy to support critical decision making or to produce long-form content.
Use more efficient, lower-cost niche AI models for targeted tasks that need a quick response, such as a real-time chat assistant or spam detection.
As the tech matures, smaller models will be able to do more, making it easier to manage costs. IT industry analyst Gartner has predicted that by 2027, organizations will use smaller task-specific models 3x more than general-purpose large language models.
In his keynote at IBM Think 2025, Arvind Krishna hailed the benefits of smaller models, saying "To win, you are going to need to build special-purpose models, much smaller, that are tailored for a particular use-case that can ingest the enterprise data and then work."
Why AI optimization is never finished
Optimizing the model portfolio is not a 'set once and forget' task. Your strategy needs to embrace continuous improvement and optimization to help get the most out of AI as it evolves.
For example, IBM's IBV research suggests that organizations that use fine-tuning or prompt engineering to optimize their models see a 25% improvement in output accuracy. This can lead to improved forecasting and AI outputs that are personalized rather than generic.
Similarly, by adding your own enterprise data to existing gen AI models hosted in the private cloud or on premises, you're more likely to produce AI outputs that provide unique value to your organization.
So you need to continually look for ways to get more out of your AI model portfolio.
At the same time, you'll need to invest in improving the AI infrastructure to embrace powerful, more sophisticated models as they are introduced. As data volumes and model complexity increase, your infrastructure needs to keep pace to handle heavier workloads. It will also need to keep up with the fact that more parts of the business will use AI for different purposes. Cloud and network infrastructure will need to be upgraded and optimized, and you may need specialized hardware.
A prerequisite is having processes in place for tracking model performance. How else can you assess whether continuous AI model optimization strategies actually translate into performance improvements? You also need tracking systems to address model drift or decay and to correct for bias in AI outputs over time.
Continually monitoring and updating model governance is another important priority. This includes ensuring your governance processes still work as AI evolves, and staying in line with fast changing AI regulations.
The enterprise gen AI control center
IBM's report raises the concept of an "Enterprise Gen AI Control Center" which it suggests is likely to emerge in the near future. This would provide a 'user-friendly experience layer' to 'connect models, assistants and prompts across the portfolio'. Together with guard rails for security, privacy and compliance, it could ensure the appropriate model was used each time.
As the number and variety of gen AI models continue to expand, business leaders need to treat model optimization as a strategic priority.