#AI > [!NOTE] Work In Progress > This is a work in progress With the rise of generative AI, and the massive impact this will have in to the education sector, it is unimaginable that Moodle would be left behind, either from the pedagogic impacts or the technical innovation. This paper sets out an idea for Moodle architecture to support the integration of processes to integrate Artificial Intelligences, in to the Moodle platform to enable the provision of each individual Moodle servers' own data with an LLM. # Simplification of AI Process We can take a simplification of the current process of generative AI building and create 5 high level steps. It is observed that each step here is, of itself, complex and complicated, so this in no way is attempting to minimise the efforts still required. Broadly speaking when interacting with an LLM, the process would be: 1. User sends a "prompt" to the LLM. 2. LLM will consider it's known knowledge. 3. LLM will consider any context is has got from prior interaction. LLM's themselves are typically "stateless", so the system the user interacts with would normally maintain this and pass it in as context for each request. 4. LLM generates a summarisation of the retrieved knowledge. 5. Summary is provided to the User Step 2 can further be broken down to incorporate steps that govern what the LLM "knows": 1. The LLM's inherent knowledge that it was trained with, using a large corpus of text. 2. Any fine-tuning that may have been performed to tailor it's responses to a particular subject area. 3. Any Prompt Engineering that may have taken place. This process provides "extra" context to the LLM, that is, in essence, a preamble provided to the LLM before the "main" user prompt (Step 3 above) 4. Retrieval Augmented Generation (RAG). This is a process that extends an LLM's knowledge by linking it to a database of further content, that is searched for similarity using a "vector". # Why Integrate Moodle? From a Moodle perspective, steps 2.3 Prompt Engineering and 2.4 RAG, would appear to be the most appropriate points for Moodle to be interacting with an LLM . Any single Moodle server already holds a wide range of diverse content, all of which is structure towards particular goals (i.e. belong to a course). A mechanism for this content to be easily used for RAG is then self-evident. Step 2.3 Prompt Engineering from a Moodle server, or more specifically a Moodle course, also becomes obvious. A given Moodle Course already defines a context, but affording a mechanism that additional hints about the course, or any given Moodle asset held within, refines the context for an LLM to observe in it summarisation. For instance a teacher could be offered a section on their course settings page that allows them to add both system prompts (such as "you should not offer to write any code for students"). # Integrating It is vital, as an open source product, but also to allow for the widest possible adoption, that Moodle implements it's integration with AI in a way that allow system administrators to select the most appropriate LLM provider(s). There is already a significant level of abstraction afforded for this via tools such as Lang Chain. It is also obvious that PHP itself will not be a suitable basis for a lot of the AI work, given the complexity and maturity of many of the AI pipelines available, particularly in Python. It would be suggested that the power and existing capabilities of a Python based layer for interacting with actual LLM providers is implemented, and then the most common "entrypoints" to these be accessed via a PHP "AI" API. ## Linking to an LLM To support a wide range of LLMs, we shall introduce an AI management layer into system administration. This layer will introduce new plugin type, similar to the OAuth2 providers, to allow systems administrators to create LLM provider instances. Each LLM Provider Instance, will link a Moodle server to a given LLM, using the relevant configuration process (API key, OAuth etc), and make available within Moodle an LLM Interface Object (LLMIO) to allow Moodle plugins to interact with LLMs via a standardised Moodle API. The main objective of the LLMIO is to provide: 1. Access to the "query" interface offered by the LLM, for fetching responses from an LLM, and to pass the relevant context. 2. Access Embedding model related to the configured LLM. By affording multiple LLMIOs and a LLMIO selector control for plugin developers, any AI functions can be attached to a single, site-wide LLM, or individual plugins may be configured to use a specific LLM for all instances, or developers may afford admins (or users) to select a specific, available LLM for a single instance of a plugin's activity. Administrators could also have the option to define an LLM Provider instance at different contexts: site-wide, or against a Moodle context, making the LLM Provider only available from the configured context & below. ## Populating As demonstrated in Querying, a store of documents in a Vector Store is a prerequisite for a RAG step, and Moodle is ideally suited as a source of these documents. We propose extending the Activity plugin API in the first instance, firstly we propose adding a new FEATURE_ constant, FEATURE_VECTOR_STORE, to allow them to indicate that they will add content to a Vector Store. Secondly a number of interface methods would be defined to allow plugins to implement specific chunking, and embed generation processes. Any plugin that declares support for writing to the vector store, but does not implement plugin-specific methods would fall back to a default implementation (in this case a call to `*_supports(FEATURE_VECTOR_STORE)` would return "null"). In the case of embeddings generation, these need to be generated in a way that is compliant with the LLM in use, and a configured LLMIO would be used by the plugin to generate access the underlying LLM's API to generate these. > [!question] Not Supporting AI > For the moment, we've defined the result of `*_supports(FEATURE_VECTOR_STORE)` to return: > * `true` - The Plugin will prepare content for the VectorStore itself > * `null` - The Plugin will not prepare content for the VectorStore itself, but will try to use a default implementation (e.g. simple textual content). > * `false` - The Plugin is excluding itself (and it's content) from attempting to add any of it's content to the VectorStore. ### Example 1: File Resource The simplest bundle of content in Moodle is the File Resource. This represents *any* form of file content, from text files, PowerPoint presentations, to complicated CAD formats. Using the new AIAPI, the mod_resource plugin would, by default, use the defined "site-wide" LLM Provider Instance. As a resource activity is created, and the content payload added to it, the ``resource_set_mainfile($data)`` function is amended, so that as the payload is saved from the activity's draftarea, the details of the activity is added to list of new or updated module content to be processed by the AIAPI's `upsert_content` task. The `upsert_content` tasks runs at regular intervals, and pops each activity's details from the list. Using the activity's details the task calls back to the `mod_resource` plugin, to see if `resource_supports(FEATURE_VECTOR_STORE)` returns `true`. The plugin returns a `true` result to indicate that it will implement all of the necessary code to add it's content. A `null` result would result in a "simple" process for adding exactly 1 "text document" representing the activity's content would be followed. In this case `resource_supports()` does return `true`, so the task can then execute the `mod_resource` plugin's `resource_vectorstore($resourceid)` function. The `resource_vectorstore()` function checks to see which LLM Provider is configured for the resource instance (in this case the site-wide one), and instantiates an LLMIO object. Within this method: 1. the activity's details are used to retrieve the list of files / content that has been added, 2. each of the files is converted in to text, and then split into appropriate sized "chunks" (as defined in the Embeddings configuration of the LLMIO). 3. the chunks are used to generate the Embeddings 4. the Embeddings are stored into the VectorStore defined in the LLMIO, along with metadata about the activity itself (course id & course contextid, mod_resource primary key, course module id & context, parent context) > [!info] > Internally the LLMIO may identify / generate appropriate "content" from the actual activity's data and then pass this onto a Python process to do the *actual* interaction with LLM. # Using Added Content Once activity plugins have added content to a VectorStore, the AIAPI can now be used by other aspects of Moodle. For the sake of this article, we're going to assume that a Moodle Virtual Chat Bot activity plugin has been created, equally this could be a core site-wide "Moodle Chat bot", but allowing teachers to create instances of bots allows for *different* constraints to be set within a course context. This VCB would handle the user's interaction, and configuration of activity-level context information, such as should the interaction cover the whole courses' content or specific course-modules (CMs) ![[MVCB.png]] When the user types a prompt and sends it to the Moodle Server, a number of processes are kicked off: 1. From the user prompt an embedding is generated using the same embedding model as the server's LLM (e.g Open AI text-embedding-ada-002 ) 2. A list of Moodle content is identified using the context configuration, and any items that are not visible to the querying user are noted. 3. The embedding is used to perform a vector search against a Moodle-wide vector store database (that has been filled with "documents" from the Moodle server). 4. Once the "similar" documents has been retrieved, the list of documents' meta data is compared against the "exclusion list" generated for the querying user in Step 2. Any documents that match exclusion are removed from the result set. 5. The result set is then passed to the LLM as system messages to influence the generated response. In the above pseudo-sequence, chat activity has been configured to use the site-wide LLM and we have already created and populated a VectorStore (configured as part of the LLM Provider Instance), by extending the mod_resource plugin. Instead of the chat plugin directly performing a Vector Search based on the user's input, the user's input is passed to an LLMIO instance associated with the site-wide LLM, to generate a query vector. ``` $coursecontext = context_course::instance($courseid); $llm = ai::get_instance_for_context(context_system::instance()); $queryvector = $llm->get_embedding($userprompt); $aicontext = $llm->search_vector($queryvector, $coursecontext); $response = $llm->chat($userprompt, $aicontext); ``` The LLMIO's (`$llm`) `search_vector()` method takes 2 parameters: 1. The query vector based on the user's input 2. A Moodle Context object, in this case the virtual chat activity operates over the whole course, so a a `context_course` instance is used. The context sets the limit of the RAG results returned. Internally the `search_vector()` method will perform a vector search of the VectorStore, and then filter the results, using the results' metadata, and performing checks such as: 1. Is the result visible to the querying user, either because it is hidden, has a hidden until, or is subject to a Moodle Restrict Access Condition. 2. Is the result sub-ordinate to the specified context? As the RAG search is looking at *similarity* between the user's prompt and the content held in the VectorStore, it may identify content that is similar, but not contained within configured scope of the search (i.e. it's in a different course). # Adding Context Context in this case refers to the AI context, this is the extra system information that is provided to an LLM to influence how it behaves. We propose extending the standard Moodle form for categories, courses and activities, with an additional field for appropriate users to provide extra prompts. These would "stack" hierarchically, with subsequent Moodle Contexts' prompts, being appended to the stack of system prompts. This stack of prompt messages, is then suffixed with the RAG results and then the user's prompt before being handed to the LLM to respond to (and handled automatically via an LLMIO's `chat()` method).