Increasing Productivity Using Generative AI: Our Maria Story

Summary of Africa Deep Tech Community meetup / conversation with Arinze Izukanne

Dialog

In this month’s session titled “ Increasing Productivity Using Generative AI: Our Maria Story”, Arinze Izukanne, Cofounder of Cerenyi.ai discussed his team’s experience building Cerenyi.ai, an LLM-based solution built for the enterprise. 

He shared the context that led to the creation of Cerenyi and then described the highs and lows that his team grappled with in building the solution.

 Arinze discussed the importance of prompt engineering in enhancing the performance of large language models and introduced a system called Maria designed to integrate with various ERP platforms. He also highlighted the potential of AI technology in Africa, emphasizing the need for predictive modeling and data privacy, and proposed a model that combines centralized and decentralized data management. Lastly, the team discussed the challenges and potential solutions for effectively using Large Language Models for enterprise productivity, the potential for AI systems to be perceived as human, and the implications this could have for society in general and Africa in particular.

Improving Large Language Models Performance

Arinze discussed the importance of prompt engineering in enhancing the performance of large language models. He highlighted that a well-crafted prompt can make a model perform better than a more advanced one with a simpler prompt. Arinze also emphasized the need to expand the capabilities of these models, such as adding vision and audio capabilities, to improve their effectiveness. He shared an example of how they integrated an AI system into their chat channels, which helped improve the quality and speed of their discussions. Arinze also discussed the challenges of large language models' limited context window length and the associated costs. He introduced a solution that his team leveraged to reduce costs and improve performance. Lastly, he mentioned adding long-term memory to their system, which allowed for more effective and human-like interactions within the network.

Maria System Integration and Privacy

Arinze presented a system called Maria, designed to integrate with various ERP platforms and synthesize information based on a single request. The system was showcased at a conference in France. Arinze emphasized the importance of privacy and security, suggesting that running models on-premises could be a more acceptable option for potential clients. Mercy and Zak raised concerns about privacy and the potential for AI to replace jobs, while Chukwuemeka asked about the impact of using Maria on Arinze's team.

LLMs in Software Development Challenges

Arinze discussed the advantages and challenges of using large language models (LLMs) for software development. He highlighted that LLMs allow for rapid prototyping and development, reducing costs and skill requirements. However, he cautioned against over-reliance on LLMs without proper testing and domain knowledge, as their outputs can sometimes be inconsistent or incorrect. 

Adopting AI in Africa: Challenges & Opportunities

Arinze discussed the potential for adopting AI technology like Maria in Africa. He believes the key factor is the social mindset and willingness to accept AI, which could provide benefits like overcoming language barriers and reducing training costs. While cost is currently a challenge due to reliance on cloud solutions, Arinze sees promise if the technology can be run locally. Arinze welcomes partners despite previous challenges, noting that reduced costs would enable more adoption across the continent.

Improving LLMs for Productivity Challenges

Chude and Arinze discussed the challenges and potential solutions for effectively using Large Language Models (LLMs) for productivity. Chude highlighted the need for training in Africa to improve prompting skills, as small differences in prompts can lead to significant response variations. They also noted the issue of model versioning changes affecting performance, suggesting the need for awareness and control over these changes. They also acknowledged the challenge of gaining trust in AI, particularly among developers who may prefer traditional methods.

AI Perception, Performance, and Adoption

Christine raised concerns about the potential for AI systems to be perceived as human and the implications this could have for society. Arinze responded by explaining how they defined the personality of their AI system, Maria, and how this personality can be adjusted. Alexander asked about the adoption of AI technology in Africa, and Arinze explained that the main cost centers are compute and power, but with renewable energy, it should be possible to deploy models on-prem and reduce costs.

Large Language Model System Development

Arinze discussed their system, Maria, as one that can connect to various enterprise apps for information retrieval. The system, which uses a large language model, excels in language capabilities and can understand and respond to queries in multiple languages. However, it struggles with tasks involving statistical or mathematical analysis. Arinze also mentioned the system's ability to learn and adapt, such as writing functions for API integration. The system's ability to delve deeper into research and understand language and culture was highlighted as one of its more surprising or unexpected features.

AI in Africa and Data Management

Lastly, Arinze discussed the potential of AI in Africa, emphasizing the need for predictive modeling and the importance of data privacy. Pius proposed a model that combines centralized and decentralized data management, with incentives for data contribution.