Problem Statement
The client has extensive experience running an IT MSP (Managed Service Provider) firm. Being a CEO of SME himself, he realized the value in being able to query in natural language over all of his connected software and be able to empower himself without depending on his employees. He came to us with his vision to connect common tools like Sharepoint, Outlook, Quickbooks, AutoTask, and 3CX and be able to extract insights quickly. The client had a mock prototype built in-house and our team re-engineered it from the ground up, architecting production-grade data pipelines and implementing LLM agents for question-answering.
Challenges
AI Trust
Would people trust the answers of the AI? Would it be correct? How do we ensure that only the latest data is being used to answer the question?
Remove AI hallucination
While LLMs are great at writing, they are not powerful enough for analysis over such a wide variety of data sources.
Data pipelining
Most of the tools to be integrated do not have API access or webhooks that can be used for getting the latest data or only the changes since the last time it was pulled.
Sensible UI/UX
The target users of Bloom are SME executives. We wanted to make sure that they intuitively understand how to use it and that the answers being created by the AI are actionable without needing additional processing.
Solutions
Verify, don't trust
We know that LLMs can hallucinate and it might not always be possible to have the correct data to answer the user's question. But, what is possible is to give users transparency on what data is being used to answer the question. We build this natively inside the chat interface so each part of the answer can be directly linked to the piece of the source.
Multi-step pipelines
We knew that this would be one of our biggest challenges, so we used our collective experiences to architect a hybrid data engine that always fetches the most important pieces of information from all sources. By using this data over multiple steps with LLMs, we removed inaccuracies and inconsistencies.
Continuous sync system
While challenging, we ended up building a mechanism on our end to detect what data had changed since the last time we pulled data from the data source and we only add the new data to our database.
User-centered AI design
By talking to users of the beta version, we were able to identify possible points of friction and then we implemented functionalities to directly target them. Based on the feedback collected on the answers, we fine-tuned the style, language, and tone of the AI to perfectly cater to the user needs.
Results
Built a complex business intelligence tool in less than 3 months
Despite the project's complexity, heavy workload, and all the challenges connected to the product's nature, we've successfully fulfilled all the tasks and got the platform ready from concept to launch in as little as 3 months.
In the client's words:
"The platform stands out for its ability to offer actionable intelligence to customers in real-time, marking a significant milestone in our progress."
Growing user base with incredibly positive responses from early users
"Most of the users are amazed by how quickly we are able to ingest their data and start providing insights to them within minutes."
Blazing fast AI system that just works
From the initial prototype to now, we have sped up the entire inference pipeline by 4x, which means faster answers for the users. By implementing an LLM evaluation framework in place, we were also able to reduce the hallucinations resulting in a system that business owners can trust out of the box.