Building organisational knowledge: Perhaps most crucially, developing a system like this is hugely impactful to organisational knowledge and memory. Building out the data pipelines, codebases and organisational processes to maintain such a system will be crucial to massively accelerating the timelines of future AI projects within the council. Fundamentally, this is a resident engagement project, it has constructed an automatic means of collecting and sorting communications from residents. As such, it can be very easily adapted to other applications grounded in resident communications and engagement. Furthermore, building out these digital systems offers automated and robust collection of clean data (complete, correct and error-free), which will be crucial moving into the future.
Next Steps
Address reliability of AI outputs: One of the primary barriers to making the system production-ready is ensuring the reliability of the AI-generated summaries and urgency levels. As these outputs directly impact how complaints are prioritised and addressed, they must be accurate and consistent. More extensive testing is required to validate the AI’s performance under real-world conditions. If performance is deemed insufficient, we must look towards a more sophisticated model, leading me onto my next point…
Fine-tune with council data: The current model leverages general-purpose LLMs with Few-Shot Learning and Prompt Engineering to categorise, summarise and label (urgency level) complaints. This means that there is an inherent reliance on the general purpose data that is not visible to the council. Due to a lack of a clean available dataset, students had to resort to AI generated complaints to test their system, this needs to be addressed for obvious reasons. The council should look to build their own database of complaints categorisation, labelling and summarisation, such that the LLMs can be fine-tuned to match desired outcomes, and ultimately lead to more reliable and explainable behaviour from the AI model.
Scale and test: The system needs to be tested with the actual volume of complaints that the council receives to ensure it can handle the load effectively. This step is crucial for identifying any potential bottlenecks or performance issues. Can the system be effectively scaled to meet every council’s needs and prevent individual developmet?
Enhance features & integrate with other data sources: The students have identified some additional features that can be developed in the future to enhance the product, such as an expanded dashboard with more customisable tiles, a task-tracking login system for better complaint management, and automated updates for residents. Furthermore, there is an opportunity to integrate other databases such as ward boundaries, relevant stakeholders to location and type of issue. This will allow for even more sophisticated features, for example, if there are numerous complaints from a particular area and issue type relevant to a council member then they can be automatically and be able to respond immediately. This can be especially powerful in the case of disaster.
Implement data security: Implement advanced data encryption methods to enhance the security of resident data, ensuring compliance with data protection regulations.
Look at integration with council workflows: Ensure that the system integrates seamlessly with the existing workflows of the council. This involves training council staff to use the new system effectively and making any necessary adjustments based on their feedback.
Increase cost effectiveness: Work with the council to assess the affordability of implementing the system on a larger, and production scale. This includes exploring funding opportunities, potential collaborations with the private sector.