Artificial intelligence is increasingly becoming a part of the product development process across a variety of industries, from customer relations and business processes to healthcare, finance and business intelligence. For a founder, the central question is what to build, how to build it and whether the investment will be rewarding.
Many believe that the creation of an AI product requires machine learning engineers, data scientists and an entire AI department. In most situations, this is not the case. With the right choice of the model, cloud platform and development partner, you can test and even launch a product without building an in-house AI team. At the same time, a detailed roadmap is needed to align the technical possibilities with the specific requirements of the product.
Start With a Business Problem, Not an AI Feature
The first task is to define the problem that needs solving. Simply adding a chatbot, virtual assistant or recommendation engine will rarely bring any substantial improvements, unless there is a clear understanding of what the current solution is lacking. Look at the processes involved in using the existing product and define pain points in terms of actions that require too much effort or time, involve repeated decisions or other issues.
For a SaaS product, this may include searching for the required information among large bodies of text, retrieving and structuring data from various unstructured sources or finding the right answer in a large pool of information. All of these can be addressed by AI, but each requires a specific approach.
At the same time, it is important to define whether this particular feature is worth the effort and whether AI is the right tool for the job. If the answer to both of these questions is positive, the next step is to define the technical requirements.
Assess the Feasibility of an AI Solution
With the choice of the application scenario, you will need to define the requirements for data, information and technology. Some solutions can be based on existing language models, while others may require elements of traditional machine learning or the use of document search and retrieval technologies. Each technical choice will have its own specifications.
For example, a solution that uses information from company documents to answer frequently asked questions can be based on a retrieval augmented generation (RAG) architecture. This refers to a specific technique that combines elements of document search and language model prompting to find relevant information in a database and present it in the required format. In this case, there is no need for large amounts of data to fine-tune a language model. The technology itself is quite complex and has a number of security and quality-related considerations.
At this point, it is important to define whether the proposed solution fits within the technical capabilities of the company. This includes data availability and quality, compatibility with the existing application, privacy and security considerations, quality control and economic feasibility.
In most cases, there is no need to pursue the proof of concept (PoC) to understand the general idea. At this stage, a discovery sprint can help identify potential issues and clarify the general vision.
Build the Roadmap in Stages
The technical and commercial uncertainty needs to be addressed in stages, with a product roadmap defining the key objectives at each step in sufficient detail without overstating the capabilities at any given moment.
Stage 1: Discovery and prioritization
The problem, target audience, goal and the current process will need to be described in detail. Based on this information, available features will be prioritised according to their value, feasibility, implementation complexity and other criteria. One specific application scenario will be chosen as a starting point based on these requirements.
For example, the support team may want to reduce the time spent on finding information without making all of the process automatic.
Stage 2: Proof of concept
At this point, an experiment is needed to define whether the chosen approach will actually work. The focus will be on the technical aspects of model training, prompting and responding, with reduced emphasis on the final application. The bar for success should be set realistically based on the expected value and limitations of the current approach.
The metrics for success should be defined before beginning this stage, with particular attention paid to accuracy, recall, latency, percentage of successfully handled prompts or other relevant criteria. The results of a PoC will not be an actual product, but it should realistically represent the expected outcome.
Stage 3: MVP Development
If the results of the proof of concept are encouraging, the next step is to develop a minimum viable product. This will involve the actual implementation of the feature within the application, along with the definition of the user interface, permissions, authentication and other aspects. The quality of model output will need to be evaluated in a consistent manner, but it should not be based on randomly selected prompts.
A set of test prompts will need to be defined and reused for quality assessment purposes. These prompts should reflect the variety of expected results, but they do not have to cover every single possibility. In some cases, human evaluation of model output may be necessary, depending on the type of application and the level of quality control.
Stage 4: Controlled launch
At this point, the controlled launch of the feature can begin. In most cases, it does not have to be available to all users, but the target audience can be invited to participate in the early stages of roll-out. The results will need to be analysed to define whether the original assumptions were correct and if the feature achieved the intended effect.
In addition to quantitative data, it is important to pay attention to qualitative improvements, such as the effect on the user. Did they accomplish the task faster and more efficiently, or did they encounter any difficulties or show reduced engagement with the product?
Stage 5: Optimization and expansion
Based on the results of the previous stage, further improvements can be made, additional prompting techniques explored and new features added. The depth of processing, coverage, speed and other aspects can be analysed to find opportunities for improvement. At this stage, the feature should be able to deliver value on an ongoing basis while being expanded in capabilities.
More complex applications can be developed once the basic version is ready, with multi-system agents being an option to consider. The process will have to be iterative, but it will provide opportunities for continuous improvements and new capabilities.
Choose the Right Technical Approach
Founders who want to avoid the expense and complexity of an in-house team do not have to develop their own foundation model. The choice of an appropriate approach is not without its challenges.
Language models that can be used through an API will be suitable for a wide range of applications, from summarization and information extraction to simple question-answering and similar tasks. In some cases, retrieval augmented generation (RAG) will be an option, depending on the availability of documents and other information.
Traditional machine learning techniques will continue to be relevant in cases where forecasting, classification, detection and other methods are needed, without the need for language processing. Finetuning or the use of specialised models may be required to achieve consistent results. They will involve additional costs and complexity if there is no sufficient amount of training data.
Self-hosted models, including those with open weights, will provide greater flexibility, but they will only be an option if the company has the required resources. Infrastructure, costs, expertise, ongoing maintenance and other considerations will need to be factored in.
Each of these options has its advantages and disadvantages, but the technical discovery should examine these capabilities in detail without focusing on popularity or other criteria. An AI roadmap consultant will be able to provide an independent assessment and help select the most promising technical directions.
Red Flags Founders Should Not Ignore
There are several red flags that can indicate that a feature will not be worth the effort or that the roadmap is unrealistic.
The feature does not have a clearly defined business purpose. If it is not possible to describe the problem that needs solving and the relevant metrics, the project is unlikely to be successful.
The proof of concept (PoC) works, but there is no understanding of how it will perform in practice. This includes not only best-case scenarios, but also edge cases, attempts to inject prompts and situations when the model refuses to respond.
There is an assumption of data availability and quality without a realistic assessment of the current capabilities. Information retrieval, permissions and other aspects can become a showstopper at any point if there is no reliable way to obtain the required data.
The economic viability of the solution is not analysed, with the focus on technical possibilities rather than the bottom line. In addition to the cost of an API, organisations will have to consider additional expenses, such as retrieval costs and additional model calls.
The model is assumed to make all of the decisions, with no quality control mechanisms in place. This can be an issue in certain applications, including finance, healthcare and law.
The feature is launched without a designated owner or maintainer, with no consideration of ongoing costs, tuning, documentation and changes to requirements.
Addressing these issues at the discovery stage will be more cost-effective than making changes later.
Manage Cost, Privacy and Reliability From Day One
An AI product roadmap must factor in economic, security and quality considerations from the start.
Aspects such as launch costs, ongoing expenses and potential savings will need to be analysed with realistic assumptions in mind. Estimating the number of users, frequency of use, body of information and model interactions will allow for a more accurate assessment of potential outcomes. At the same time, a discovery sprint will be needed to analyse these possibilities with sufficient detail.
Personal information and similar data will have to be analysed in terms of privacy laws and regulations. This includes technical aspects, information sent to models and responsibilities of third parties, with requirements varying depending on the industry, location and type of information.
The security and reliability of the solution will need to be analysed in sufficient detail, with variations in model quality, additional costs due to poor performance, latency and user expectations factored in. In the case of external models, rate limiting, failures and other considerations will have to be analysed.
Information provided by an AI model should not be assumed to be correct in all situations, with this being particularly important in healthcare, finance, law and similar fields. Specific use cases, quality control mechanisms and relevant regulations will need to be analysed on a case-by-case basis.
When Should You Bring in External AI Expertise?
External expertise can be beneficial in a variety of situations, but it is important to understand the limitations of such a partnership.
In most cases, an external consultant cannot replace technical and product leadership, with the choice of a partner being made based on the specific needs of the product and its own capabilities. A consultant can provide an independent assessment, but the product team should not outsource its responsibilities. Other considerations include technical analysis and feasibility, feature prioritisation, prototyping and implementation support, with the engagement being limited to specific outcomes rather than general advice.
An AI roadmap consultant can provide third-party validation, accelerate the launch of the product and ensure that the solution meets technical, economic and quality requirements. Founders do not need to have every available skill set before launching a product, but it is important to recognise the limitations of their own expertise.
In most situations, it will be more efficient to build an in-house team around a product-ready AI feature, rather than the other way around. Ongoing technical and commercial considerations will define whether additional specialists are needed.
A Practical Roadmap Is More Valuable Than a Long Feature List
A product roadmap is an essential tool for any organisation that wants to analyse the strengths and weaknesses of a particular idea. In the case of an AI product, this involves additional considerations, ranging from technical aspects to economic and quality-related factors.
A founder without an in-house AI team will benefit from a pragmatic approach that defines the most valuable use case, the technical requirements and the launch criteria in sufficient detail. The most important aspects include an understanding of the most valuable application scenario, the specific requirements for each technical solution and the success criteria for each stage of development.
At BrainWeld, this is the essence of venture engineering, AI consulting and product strategy – enabling organisations to turn ideas into products with measurable benefits.
Question and Answers
Can I build an AI product without hiring AI engineers?
Many products at the early stages can be built with the use of existing AI solutions and the appropriate engineers. At the same time, this will still require technical expertise, ranging from software development to QA and production engineering. AI engineers will still be needed, depending on the technical complexity of the chosen solution.
How long does it take to prepare an AI product roadmap?
The time required will vary depending on the complexity of the product, the availability of stakeholders and the number of options under consideration. In most cases, it will be more important to follow a prioritised discovery rather than adhere to a strict deadline. A realistic product roadmap will be built around the findings of a discovery sprint, rather than a set schedule.
Should a startup build its own AI model or use an existing one?
In most cases, the preferred option for a startup will be the use of an existing model. This will minimise the risks and costs of development, as well as the time and resources required. Custom model development, fine-tuning and similar approaches should only be considered if the business requirements explicitly require this.
How much should a startup budget for AI product development?
There is no reliable rule of thumb in this situation, as the costs will depend on the technical complexity of the chosen solution, infrastructure, volume of data and a variety of other factors. It may be beneficial to begin the process with the proof of concept to obtain more realistic estimates.
How do I know whether an AI feature is ready for launch?
Before launching any feature, it will be important to define the success criteria and test the solution against them. In most cases, it will be sufficient to cover a subset of possible prompts, but they should reflect the intended application of the feature. The quality of output, the ability to recognise edge cases and technical requirements will need to be analysed as well. A controlled launch will then define whether the feature is ready for general release.
Do I need an AI consultant if I already have a software development team?
This will depend on the technical expertise of the existing team and the requirements of the intended product. Software engineers will often be able to analyse the options without additional input, but there are situations when an external expert can provide fresh insight. An AI roadmap consultant can help make decisions about the use of AI in product development. This includes the identification of the most promising technical solutions and recommendations for implementation.