Open edX AI Extensions with self-hosted models

The AI Extensions plugin makes one outbound call per AI feature. When a learner or course author triggers an AI feature, the LMS or Studio sends a request to the model endpoint configured for that instance. The endpoint returns a completion, and the plugin displays that text to the learner or author. Every other design decision follows from this path: which application servers send the request, which address they reach, what credentials they carry, and what the interface shows while the model is working. For a self-hosted model, the endpoint is the Ollama or vLLM server you run, so the first decision is which network zone it sits in and which LMS and Studio hosts are allowed to reach it.

The endpoint address, model name and credentials are set in the plugin's configuration, which Tutor renders into the LMS and Studio settings during deployment. The exact setting names, and whether the plugin reads separate values for learner features and author features, must be confirmed in the plugin's documentation for your Verawood installation, along with the plugin version that supports Verawood. Store credentials in your deployment's secret mechanism, not in course content or in shared configuration files that many people can read. Protect the endpoint itself as well. The local Ollama API on port 11434 needs no authentication and binds to 127.0.0.1 by default, so reaching it from application containers means changing its bind address and adding a firewall rule. The vLLM --api-key option covers only some paths, so place a reverse proxy in front of vLLM that exposes only the routes the plugin calls.

Tutor is the official Docker-based distribution for Open edX, and its plugin and hooks system is licensed under Apache 2.0. A Tutor plugin is installed into the Tutor environment, enabled, and then applied by regenerating the configuration and images and redeploying the platform. The exact install and enable commands depend on your Tutor release, so confirm them in the Tutor documentation before you schedule a change. Because a plugin that changes settings also changes the rendered configuration, treat every plugin change as a platform change. Pin the plugin version to one tested against your Verawood images, run the change in a non-production environment first, and keep the previous configuration so you can roll back the plugin without rebuilding the rest of the platform.

Verawood added new frontend plugin slots, and that is the reason the integration can place AI elements in the interface without changing core platform code. Slots are defined points where micro-frontends accept additional components, so a plugin can add a learner assistant or an author tool in the right place and keep its code separate from the platform's. Confirm which slots the plugin uses in Studio and in the LMS, and confirm that those slots exist in the Verawood release you run. Then plan for the failure case. If the model endpoint is slow, overloaded or unreachable, the component must show a clear message and let the page keep working. Test that behavior with the endpoint stopped and with a slow endpoint before any learner sees the feature, since a page that waits on a model call can make the whole course experience feel broken.

Course teams can use the self-hosted model to draft question text, distractors and feedback for problems they then edit in Studio. The model saves drafting time, but it does not know which learning outcome a question must test, whether a distractor is accidentally correct, or whether the feedback matches the course's own explanation. For that reason, every draft should pass a named reviewer before it reaches a learner. Name that person in writing, give them authority to reject a draft, and make sure they are not the same account that prompted the model. Treat the output as a first draft, and decide before rollout which course roles may generate drafts at all.

Each course should have its own prompt template. A template captures the course subject, the learner level, the question types the course uses, the target reading level and the tone of feedback. These fields keep drafts consistent across a course team and make each output easier to check against the course objectives. Store templates in version control or in the configuration repository your operators already manage, with a version number for every change. Record the template version on each draft so a reviewer can see which instructions produced it. To reuse templates, start from an approved base and fill in only the course-specific fields, so a change to the base is made once, on purpose, and reviewed before it spreads.

Keep one course's material out of another course's template. Do not paste source text, answer keys or learner data from a finished course into a template for a new one, and do not share a template between courses with different audiences or confidentiality needs. Confirm with your implementer how the AI Extensions Framework assembles context for each request, including whether it sends course content beyond the problem being drafted, and which logs capture prompts and responses. Self-hosting keeps inference on your infrastructure, but application logs, caches and shared dashboards can still mix material across courses. Test this before launch by drafting a question in one course and checking that no phrase from a second course appears in the output or the logs.

Define the review workflow in writing before enabling drafting. Decide who approves: normally the course author who owns the learning outcome, plus a subject reviewer who did not write the draft. Decide where drafts live: in a state that learners cannot see, such as a separate draft store or an unpublished component, and confirm in your setup that a draft cannot reach a live course by accident. Decide how rejected drafts are recorded: keep the draft, the reviewer, the reason code, the template version and the date. Those records show patterns, such as distractors that are repeatedly too close to the correct answer, and they let auditors see the control operating. Ask your implementer to include these records in the written scope and to confirm which of them the framework supports natively.

Cost for self-hosted AI in Open edX comes from five places, and only one of them is a licence. The first is accelerator or server capacity. A vLLM deployment needs GPU, ROCm or other supported accelerator capacity sized to the models and to the number of learners using the assistant at once, while an Ollama deployment can run smaller models on ordinary server hardware. The second is storage for model files, which grows with each model you keep for testing or rollback. The third is the surrounding infrastructure: the Tutor-based Open edX hosting, a reverse proxy that exposes only the endpoints learners need, and logging. On licences, Ollama's pricing page states that running models on your own hardware is always unlimited. Its cloud plans (Pro at $20 per month or $200 per year, Max at $100 per month, Team at $500 per month, and Enterprise at custom pricing) apply only if you call its hosted models. vLLM has no paid edition, so its cost is the hardware and the people who run it.

Implementation is the one-time part, and its size depends on decisions made before any configuration begins. Start with the learner assistant scope: which pages it answers on, what it declines to answer, and whether it can see learner data. Then design the review workflow for AI-drafted questions, including who approves a draft, where drafts wait until approved, and how rejections are recorded. Each course with its own prompt template adds authoring and review time, so count the courses in the first release. The build also covers plugin configuration in the Tutor setup, network controls (Ollama's local API has no authentication, so it must stay on 127.0.0.1 or a private address set through OLLAMA_HOST), and concurrency settings such as OLLAMA_NUM_PARALLEL and OLLAMA_MAX_QUEUE. Testing covers draft quality, review gates, behavior when the model is slow or unavailable, and access controls.

Recurring work starts with runtime upgrades. vLLM ships new minor releases about every two weeks, including v0.28.0 on August 26, v0.29.0 on September 9, v0.30.0 on September 22 and v0.31.0 on October 5, 2026. Ollama also ships frequent releases, including v0.40.5 on 14 October 2026. Each upgrade needs a regression run against the review workflow and the question test set. Model updates are the second recurring item, because a new model version can change draft quality, so repeat the quality tests before switching. Monitoring is the third. vLLM exposes Prometheus-format metrics at /metrics covering request counts, token throughput, latency and KV cache use, and someone must watch those metrics along with Ollama's queue behavior. Prompt maintenance follows course changes. Assign a named owner to each item, because unowned monitoring and stale prompts are where running costs tend to grow without anyone noticing.

A quote should follow from a written scope, and the scope should state the assumptions behind the price. Ask for the model names and parameter sizes, the accelerator type and count or the CPU and memory profile, the expected number of concurrent learners, the Ollama or vLLM settings assumed, and whether any hosted provider is included. Ask for the test plan: functional tests of the learner assistant, acceptance tests for AI-drafted questions through the review gate, a load test at the concurrency you specified, and failure tests for model timeouts. Ask the vendor to separate one-time costs from recurring ones, and to state who operates the system after handover. DeployOpen works to fixed scope agreed in writing before work starts, and retainers are available for ongoing upgrades, monitoring and testing. A quote without these assumptions cannot be compared with another.