Abstract
In this white paper, we explore the implementation of a Large Language Model (LLM)-powered chatbot within m•compass, the competency management system that Mitrais develops and uses. The chatbot utilises Retriever-Augmented Generation (RAG) embeddings to answer employee enquiries related to company policies, competencies, and other system-related queries. Additionally, function calling is integrated to dynamically fetch personalised employee details. This document highlights the technical aspects of the system, its operational benefits, challenges encountered, and the future potential of AI-driven HR support systems.
Introduction
With the rapid advancement of artificial intelligence, organisations are increasingly leveraging AI to streamline internal processes and improve employee experiences. A notable innovation in this space is the deployment of LLM-powered chatbots, which provide on-demand support and information to employees. In our organisation, we have integrated an LLM chatbot into m•compass, allowing employees to access relevant information about their competencies, policies, and more, through natural language conversations.
m•compass is a competency management system developed by our organisation to ensure an integrated management approach on talent’s career progression, allowing us to have improved quality outcomes for clients as well as improvements in operational efficiency. Its features can accommodate common business processes in the talent assessment process, such as:
- Do a full cycle of talent competency review: talent to perform their own Self Review, managers to do Supervisor Review in assessing their staff’s competency level, and technical champions to do Champion Review. And finally, a panel review process to finalise the review results, and result memos can be generated once the review is finalised.
- Competency claim by staff and review by champions/supervisors can be done throughout the year, allowing real-time visibility on staff competency charts.
- Profile documents (CV) can also be generated, which include all approved competency claims of the talent.
This white paper outlines the technical implementation of the chatbot integration with m•compass, including the use of RAG embeddings for answering policy-related queries and function calling to retrieve employee-specific data, as well as the resulting impact on operational efficiency and employee satisfaction.
m•compass and Its Challenges
m•compass serves as a central hub for employee development, containing critical information regarding job roles, required competencies, performance evaluations, and more. Traditionally, accessing this information required navigating through various manuals, policy documents, or interacting with HR representatives, competency champions, etc.
However, as the workforce grew and the volume of questions regarding the system increased, there arose a need for an efficient, scalable way to provide employees with answers to their enquiries. The integration of an LLM chatbot was seen as a strategic move to address this challenge by offering real-time, accurate responses based on our competency policies.
Solution Overview: Implementing the LLM Chatbot
LLM Chatbot Architecture
The chatbot is built upon a state-of-the-art LLM, capable of processing natural language queries and delivering coherent, contextually accurate responses. The system leverages the following components:
- Retriever-Augmented Generation (RAG) Embeddings: The LLM is enhanced with RAG embeddings, which help the model retrieve relevant information from a large corpus of policy documents and knowledge bases. This allows the system to answer employee questions about competencies, policies, and organisational procedures with high accuracy.
- Function Calling: To personalise responses and provide specific employee information (e.g., performance evaluations, training completion), the system calls on internal databases to fetch relevant data dynamically. This integration ensures that responses are not only contextually relevant but also tailored to the individual employee’s profile.
Workflow and Integration
The chatbot operates in real-time, processing employee queries and retrieving information from both static (policy documents) and dynamic (employee data) sources.
The process flow is detailed below:

Figure 1 m•compass chatbot workflow
- The employee submits a question via the chatbot interface.
- The LLM, enhanced with RAG embeddings, parses the query and retrieves relevant information from internal policies and knowledge bases.
- If the query requires personalised data, function calls are made to fetch specific employee details from the internal system.
- The chatbot generates a coherent and personalised response, which is displayed to the employee.
Technical Details and Considerations
Challenges in Policy Retrieval
One of the primary challenges we faced was ensuring that the LLM could accurately retrieve and present information from a vast array of policy documents. This was addressed by using RAG embeddings, which combine traditional retrieval-based methods with generative language models to enhance the quality and relevance of the responses. The process begin with preprocessing the policy documents: they were segmented into manageable chunks (e.g., paragraphs or sections), tokenised, and converted into dense vector representations using a transformer-based embedding model. These embeddings were then indexed in a vector database optimised for fast similarity searches.
During query processing, the system first encodes the user’s question into a vector and retrieves the top-k most relevant document chunks based on cosine similarity or another distance metric. The LLM then uses this retrieved context to generate a coherent and contextually appropriate response, rather than relying solely on its pre-trained knowledge.
Integration with Employee Data Systems
Another significant challenge was ensuring seamless integration with internal systems to pull personalised employee data. Function calling was implemented to ensure secure and efficient retrieval of sensitive information while maintaining privacy and compliance with data protection regulations. The process likely involved defining a set of API endpoints each tied to a specific function, such as getEmployeeDetails(employeeID) or fetchCompetency(employeeID) that the LLM could invoke dynamically based on the query’s intent.
To implement this, the team uses LLM tool features to recognise when a query requires external data (e.g., “What’s my detail information?”) and map it to the appropriate function call. Data privacy was maintained by limiting data retrieval to the minimum necessary fields.
Ensuring Accuracy and Relevance
Given the complexity and variety of queries employees could ask, we focused on fine-tuning the LLM to handle diverse topics and provide precise answers. Continuous monitoring and adjustments were made to improve the accuracy of the responses, especially for more complex or ambiguous questions. We ensure relevance by incorporating user context wherever possible leveraging metadata like the employee’s role, or department to tailor responses. We also converted the policy document into Q&A format which could increase the accuracy of the responses.
Benefits and Impact
Enhanced Efficiency
The implementation of the LLM chatbot has drastically reduced the time HR staff spends answering repetitive or routine questions. Employees now have 24/7 access to the information they need, improving overall efficiency.
Improved Employee Satisfaction
The ability to quickly retrieve answers to competency-related questions has contributed to improved employee satisfaction and engagement. With shorter wait times and easier access to important information, employees are better supported in managing their professional development.
Scalability
The LLM chatbot has proven to be highly scalable, handling hundreds of employee enquiries without additional strain on HR resources. As the company grows, the system can easily accommodate more users and incorporate additional functionality.
Future Directions
As AI technology continues to evolve, there are several potential avenues for further improving the system:
- Enhanced Personalisation: Leveraging deeper integrations with employee performance data and development goals could allow the system to offer more tailored career advice and insights.
- Continuous Learning: The system could be further enhanced with continuous learning capabilities, allowing it to adapt to changes in policies and employee needs over time.
Conclusion
The implementation of the LLM chatbot in m•compass represents a significant step forward in leveraging Artificial Intelligence to improve employee engagement, streamline HR processes, and increase operational efficiency. By combining RAG embeddings and function calling, we have created a system that not only provides accurate policy-based answers but also personalises responses based on individual employee data. As the system evolves, we are confident it will continue to transform the way our employees interact with the Competency System, driving greater satisfaction and fostering a more agile workforce.