Bringing ChatBI Online Requires Attention to Multiple Aspects
ChatBI is a hot topic, and many enterprises are either exploring it or taking a wait-and-see approach. Most ChatBI projects may fail, so relevant supporting work must be done well.
**Technology Route Selection**: Writing SQL with LLM is unreliable, which is reflected in accuracy, performance, and credibility. In terms of accuracy, business users have high requirements for the accuracy of answers to structured data. When LLM writes SQL, it is prone to errors such as time understanding errors and sorting logic errors. FineChatBI rewrites questions with LLM and calls a mature base to generate charts, improving accuracy. In terms of performance, the longest question return time that users can accept should be within 3 seconds, but the performance of LLM writing SQL is not good. In terms of credibility, the Text2SQL route gives users SQL statements that business users cannot understand and are difficult to debug. FineChatBI gives users clear and readable chart generation rules, making it convenient for adjustment and secondary chart generation.
**ChatBI Implementation**: It cannot be used out of the box and requires the right time, place, and people. The right time is to find a real scenario and have business needs. The right place is to have mature data and knowledge underlying preparations. The right people is to have supporting organizational driving forces. To find a real scenario, we need to have more conversations with business teams to understand data retrieval scenarios and pain points. We can also develop demos to find scenarios, but the success rate is low. ChatBI users need to be determined from the number of users, usage frequency, and usage scenarios. Underlying preparations include the data side and the knowledge side. On the data side, we need to avoid field name ambiguities, etc. We can prepare wide tables or combine them with an indicator management platform. Configuration on the knowledge side is inevitable and is divided into synonym and enterprise-specific knowledge configurations. In terms of organizational driving forces, ChatBI projects require roles such as leaders, product managers, and IT. The product manager is the core and leads the project process. It is not suitable to be used by leaders at the beginning and should be promoted linearly. Leaders make decisions and invest. IT is responsible for data preparation and underlying design. Business representatives and ITBPs also play important roles.
**Attention for Formal Launch**: In terms of security, it should have enterprise-level permission control capabilities and LLM should support local deployment. In terms of computing power cost, FineChatBI uses a small-sized open source LLM, with low cost. In terms of continuous operation investment, we need to consider team resources, methodologies, and product function support. ChatBI is first and foremost an enterprise-level application, and we should be cautiously optimistic about LLM. Commercialized ChatBI tests manufacturers. This article takes rapid data query as the first-stage implementation experience. ChatBI can help business users complete personalized analysis work.
Posted on December 25, 2024 at 17:43:56 on Wednesday.
