GitLab Runner autoscaling on VMware vSphere

GitLab Runner autoscaling on VMware vSphere for on-premises CI

CI jobs that must run inside a data center on VMware vSphere often end up on a fixed pool of runner VMs. Sized for peak demand, that pool sits idle most of the day. Sized for typical demand, it queues jobs during release windows. Autoscaling is the usual answer, and GitLab Runner's autoscaling executors support hypervisors as well as public clouds, so the runners can stay in your data center. The vSphere path depends on a community fleeting plugin that clones VM templates to create and remove runner instances. GitLab does not test that plugin, so the reliability of each clone, power-on and registration step depends on the plugin version you choose and on the testing you do yourself.

Choose this path when jobs must run on vSphere for policy, data residency or network reasons, and when demand is bursty enough that a fixed pool wastes capacity. A fixed pool is simpler to operate and has no clone cycle to debug, but it forces a sizing decision you will revisit often. The Kubernetes executor suits teams that already run a cluster and accept container-based job isolation, and it avoids template management altogether. If your jobs need full-host access, a VM-based design may still fit, and the Instance executor covers that trade-off. Compare the options on isolation, operating effort and who on your team will own each part.

Before you commit, confirm the following against the plugin version you select: which GitLab Runner versions it supports, how it handles template cloning, power-on and instance removal, which vCenter permissions it requires, and how it reports failures during scale-out. The GitLab Runner documentation advises keeping Runner major and minor versions aligned with GitLab, so pin the plugin and Runner to versions you have tested together. Then decide who owns the plugin after go-live. Every upgrade needs a staging run that creates and removes instances under realistic job load, and the support plan should name who fixes a broken plugin and how quickly that person can be reached.

Which vSphere runner pattern fits your team

This table compares four ways to run GitLab Runner on VMware vSphere: fixed runner VMs, autoscaling with a fleeting plugin on cloned templates, the Instance executor on dedicated VMs, and the Kubernetes executor on a cluster hosted on vSphere. Read each row against your own isolation requirements and team capacity. The main takeaway is that more automation shifts work from job-by-job isolation toward plugin support and platform operations that your team must own.

Pattern	Isolation for jobs	Operational load on your team	Support to arrange and fit
Fixed runner VMs	Jobs share long-lived hosts, so isolation depends on the executor and cleanup you configure	Low automation but steady patching, capacity planning and manual scaling	Confirm OS and runner patch ownership; fits small, stable workloads with predictable demand
Autoscaling with fleeting plugin on cloned templates	Fresh clones per job or instance reduce leftover state, but the plugin is community-maintained	Higher: template builds, vSphere permissions, plugin upgrades and capacity limits	Ask who tests the plugin against your vCenter version and who responds when clones fail
Instance executor on dedicated VMs	Each job runs on a whole VM, giving the strongest separation from other jobs	Moderate to high: VM lifecycle, image maintenance and cost of idle capacity	Confirm autoscaling support for your vSphere setup and image rebuild ownership; fits full-host jobs
Kubernetes executor on a vSphere-hosted cluster	Pods share nodes, so isolation depends on namespaces, RBAC and node policy	High: the cluster itself, storage, networking and runner upgrades in addition to CI	Confirm cluster support contract and platform team coverage before choosing; fits teams already running Kubernetes

What it means that GitLab does not test community plugins

GitLab's test coverage and release process cover GitLab Runner and the plugins GitLab documents, but not community fleeting plugins. In a vSphere setup, the fleeting plugin is the component that talks to vCenter to clone a VM from a template, wait for it to accept connections, hand it to the runner manager, and delete it when the autoscaler no longer needs it. The runner decides when to scale, but the plugin performs the actual infrastructure calls. That makes the plugin a dependency that GitLab does not validate against your vCenter version, template or network layout. Treat the plugin as your own code to qualify, not as part of the GitLab release.

Practical consequences follow from that. When you move GitLab Runner to a new release, the runner may change how it calls the plugin interface, and a community plugin built against an older interface may fail only on your fleet. Plugin defects often surface only in your environment: a template with a different guest tools version, a datastore that is slow to clone, a folder permission that blocks deletion, or a naming pattern that collides with another automation. None of these show up in GitLab's own tests. Confirm with the plugin maintainers which Runner versions they have tested and how they report breaking changes, and record that answer before you commit to the plugin.

Pin the exact plugin version and the GitLab Runner version in configuration management, and promote changes through a non-production runner manager that runs a representative pipeline load. Keep runner and GitLab versions aligned, as the GitLab Runner documentation advises. Your test cycle must also follow GitLab's release cadence. The GitLab maintenance policy describes a minor release on the third Thursday of each month and patch releases twice a month, and it backports bug fixes only to the current stable release while security fixes also reach the previous two monthly releases. Schedule a recurring review for each cadence, and plan to test any plugin update before it reaches production, even when the change looks like a security patch.

Security risks of cloning vSphere templates for every job

Cloning a vSphere template for each CI job gives the runner manager broad power over your virtualization layer and turns every clone into a potential path to internal systems. The cards below pair each risk with a control you can check before the autoscaler goes near production. Treat them as a checklist for the design review, not a one-time setup step.

Scope of vCenter credentials

The runner manager authenticates to vCenter to clone, power on and delete VMs, so whatever account it uses defines the blast radius if the manager is compromised. Create a dedicated vSphere role limited to the folder, resource pool, datastore and network the runner fleet needs, and deny access to other clusters and templates. Confirm the account cannot modify the template itself, since that would let a compromised job alter the image every future job boots from. Rotate the password on a schedule and store it in a secrets manager rather than in plain text in config.toml.

Template hygiene and inherited secrets

Every clone inherits what was in the template at the time it was captured, including SSH host keys, cached cloud credentials, shell history, package tokens and runner registration data. Before converting a VM to a template, remove credentials, regenerate host keys on first boot and confirm no runner token is baked into the image. Rebuild templates from a documented pipeline so a leaked key can be traced to a specific template version. Ask who signs off on template changes before they reach the autoscaler.

Network segmentation from internal systems

Job VMs run code from merge requests, and some of that code is untrusted until reviewed. Place the runner network in a segment that can reach the GitLab instance and required package mirrors but cannot reach databases, domain controllers, management interfaces or the vCenter API. Enforce this with firewall rules at the segment boundary rather than relying on the VM's own settings. Test it by running a job that attempts to reach each restricted service and confirming it fails.

Cleanup of clones that fail to delete

When a job is cancelled, the manager crashes or vCenter times out, clones can be left powered on with valid runner tokens and job artifacts still on disk. These orphaned VMs consume capacity and hold credentials long after the job ended. Set a maximum lifetime for every clone, alert on VMs in the runner folder older than that limit, and run a periodic sweep that deletes them. Confirm with your vendor how the fleeting plugin records clones so the sweep can match them reliably.

Isolation between trust levels

A runner that executes jobs from a sandbox project should not share a VM pool, or a registration token, with a runner that deploys to production. Give each trust level its own runner manager, its own vSphere folder and its own template, and tie deployment runners to protected branches. Using separate tags makes it harder for a job to land on the wrong pool by accident. GitLab's hardening guidance covers the runner and CI/CD settings that complement this separation, and you should apply them gradually and test each change.

Audit of template and role changes

A template change or a new vSphere permission can alter what every future job runs with, so changes need the same review as code. Enable vCenter event logging for template edits, clone operations and role assignments, and forward those events to your central log store. Restrict who can edit the template folder and require a named approver for each change. Review the audit trail on a schedule, and confirm retention meets your internal policy.

How to set up vSphere autoscaling and verify each stage

This sequence takes a team from version alignment to a working autoscaled runner fleet on vSphere, with a check and a rollback point at each stage. Each check confirms that the next stage is safe to start.

Confirm GitLab and Runner versions match
Record the version of your self-managed GitLab instance and install a GitLab Runner manager and the fleeting plugin at the same major and minor version. GitLab's Runner documentation advises keeping Runner major and minor versions aligned with GitLab, so check the compatibility notes for the plugin version you intend to use. Done means the version record is saved in your change ticket and every component matches the agreed pair. Rollback point: if versions cannot be aligned, stop here and fix the GitLab or Runner version before touching vSphere.
Build and harden the vSphere template
Create a base virtual machine with the operating system your image standard requires, install only the Docker or other job dependencies the jobs need, apply your hardening baseline, and convert it to a template. Remove SSH host keys, machine IDs and any registration secrets so each clone generates its own identity. Done means a clone of the template boots, joins the network, and starts with no runner registration and no stored credentials. Rollback point: keep the previous template version so you can revert the template reference.
Create a least-privilege vCenter role
Ask your vSphere administrator to create a dedicated role for the plugin account that grants only the permissions needed to clone, power on, power off, read and delete the runner virtual machines in the designated folder and resource pool. Avoid granting the Administrator role or global permissions. Done means a test login can perform a clone and delete in the target folder and is denied actions outside it. Rollback point: remove the role assignment and recreate it from the reviewed permission list if the test fails.
Place the runner manager outside the job pool
Deploy the runner manager on its own virtual machine that is not a clone target and does not sit in the folder where autoscaled jobs run. The manager holds the credentials that create and delete job machines, so it should be managed separately and have its own network path to GitLab. Done means the manager reaches your GitLab instance, can reach vCenter, and does not appear in the autoscaling folder. Rollback point: if the manager is found in the job folder, move it out before enabling autoscaling.
Configure the fleeting plugin and capacity limits
Set the plugin connection to vCenter, the template reference, folder, resource pool and network in the runner configuration file. Set maximum concurrent instances and idle counts so the fleet cannot grow beyond what your vSphere cluster can hold. Confirm each plugin-specific configuration key against the plugin's own documentation, because key names and options are set by the plugin and not by GitLab. Done means the manager starts without configuration errors and reports zero or the idle count of machines in vCenter. Rollback point: disable the autoscaling section and restart the manager to return to a fixed runner.
Register runners with authentication tokens
Create the runner in the GitLab interface or API so it receives a glrt- authentication token, then store that token in your configuration management system rather than in a registration command you keep in a script. Assign tags that match the jobs you intend to route to this fleet and leave the runner restricted to the projects that need it. Done means the runner shows as online in GitLab and the tags match the job definitions. Rollback point: revoke the new token and create a fresh runner if the token was exposed.
Run a test job and verify clone lifecycle
Run a small pipeline job tagged for the fleet and watch vCenter during the run. You should see a new clone created from the template, the job execute on it, and the clone removed or returned to the idle pool after the job completes. Check that the clone names, folder placement and network assignment match your design, and that no leftover machines remain after the idle timeout. Done means the test job passes, the clone is created and deleted as configured, and the vCenter event log shows only the expected plugin actions. Rollback point: set the maximum instances to zero to stop new clones while you investigate.

Who supports the plugin when it breaks

Your team owns the plugin from end to end. Engineers choose the fleeting plugin version, pin it in the runner configuration, build and maintain the vSphere templates and decide when each change reaches production. This model suits organizations that already operate GitLab Runner and have staff who can read Runner release notes and trace provisioning failures in vCenter.

Version pinning: Record the exact plugin and GitLab Runner versions in configuration management, and keep the Runner version aligned with your GitLab version, as the GitLab Runner documentation advises.
Upgrade testing: Run a staging runner manager against a staging vSphere resource pool, execute representative pipelines and confirm that instances clone, register and deregister cleanly before production.
Incident response: On-call engineers triage stuck jobs, failed clones and orphaned virtual machines, and they open issues with the plugin maintainers when the defect sits in the plugin.
What you provide: A dedicated vCenter account scoped to one resource pool, an automated template build pipeline, and staff with Linux and VMware skills who are available for on-call work.
Not covered: Nothing is handled externally, so GitLab upgrade planning, vSphere capacity decisions and any plugin defect fall to your team with no contracted response.

Confirm that each upgrade has a named owner and that the staging environment matches production closely enough to catch template drift.

Limits to confirm in your vSphere and GitLab environment

A vSphere autoscaling runner fleet depends on several moving parts that GitLab does not test on your behalf: the community plugin, the hypervisor API, your capacity and the GitLab release you run. Resolve each item below with the plugin maintainers, VMware documentation or your own test cluster before you commit to a design.

Fleeting plugin version and vSphere support

Confirm which version of the community fleeting plugin for vSphere you intend to use, and whether its release notes explicitly list the vSphere versions it supports. Ask the maintainers how the plugin handles template cloning, power state and instance cleanup, and whether known issues are tracked in a public issue list. Record the exact plugin version in your runbook, since the plugin is not covered by GitLab testing and its behavior can change between releases.

vSphere version and cloning API behavior

Check how your vCenter and ESXi versions handle full clones from a template, linked clones and customization specs under the load your autoscaler will create. Ask whether the plugin's API calls match the vSphere API version in your environment, and test clone creation and deletion at your expected concurrency. Failed clones often surface as stuck instances, so confirm how errors are reported back to the runner manager.

Storage and network capacity for clones

Each autoscaled runner is a full virtual machine, so confirm that your datastores can absorb the clone I/O and the disk space of idle and busy instances at once. Verify that the port group, IP address pool and DHCP or DNS behavior can support new hosts being added and removed continuously. Ask your storage and network teams to validate these limits against your own maximum job concurrency rather than an average.

Supported executor types in your runner version

Confirm which executors your chosen GitLab Runner version supports with the fleeting-based autoscaling path, and whether the Docker Autoscaler executor is the intended choice for your jobs. Check whether any of your pipelines depend on features tied to a specific executor, such as privileged containers or services. Test one representative pipeline on the exact runner version before rolling the design out to all teams.

GitLab release and upgrade stops

Keep the runner manager and the GitLab instance on aligned major and minor versions, and plan each change against the required upgrade stops that GitLab publishes. Check which upgrade stop falls within your upgrade window, and confirm that the runner version you deploy is supported with the GitLab version you will be running after it. Record both versions together so a GitLab upgrade does not silently leave the autoscaler behind.

Ongoing maintenance of the plugin

Find out whether the plugin is actively maintained, how often it releases, and who responds to security issues and vSphere API changes. Ask whether the maintainers publish a deprecation policy and a compatibility matrix. If the project is slow or unclear, decide now whether your team or a support partner will own patching, forks or a move to another autoscaling approach.

Questions teams ask before autoscaling runners on vSphere

What drives the cost and effort of autoscaling GitLab Runner on vSphere?

Cost comes from three places: the vSphere capacity the fleet reserves, the labor to build and test the setup, and the work to keep it current. Capacity depends on peak concurrent jobs, job size and how many idle VMs you keep warm. Build effort rises with the number of template images (operating systems and toolchains), the network and DNS design, the permission model for the vCenter service account and the number of runner tags. Recurring costs include Runner and GitLab upgrades, template rebuilds for patches and plugin maintenance. Ask any quote to itemize fixed build scope, the number of templates, the environments covered and whether ongoing support is included.

How long does a vSphere runner autoscaling build take?

The duration is set by the written scope, and no fixed timeline applies to every environment. The main drivers are the number of template images, the network segmentation between runner managers and VMs, access approvals for vCenter and the test effort needed against your real pipeline workloads. Change windows and the time your vSphere administrators need for clone and approval cycles also add to the calendar. A useful quote breaks the work into milestones: discovery, template build, a pilot fleet, cutover and handover. Confirm the vCenter approval lead time with your infrastructure team before you agree on dates.

What must our team provide before a vSphere runner build starts?

Your team needs to provide the access, environment facts and decisions that the build depends on:

vCenter, cluster, resource pool and datastore details, plus a service account with permissions limited to the clone and power operations the plugin needs
Network and DNS details for the runner managers, the cloned VMs and the GitLab instance
The golden template source or an agreed operating system baseline, with image ownership defined
A GitLab instance you can reach from the runner network, and a decision on runner authentication tokens
Secrets handling, change approval rules and a named owner who can make decisions during the build

Missing items in this list are the most common cause of delayed starts, so resolve them before the scope is signed.

Can an internal platform team build vSphere runner autoscaling alone?

An internal team can build it alone if it has working skills in vSphere automation, Linux image building, GitLab Runner configuration and security review. The harder parts are the clone and template lifecycle, the credential model and failure handling when VMs fail to boot or register. The community fleeting plugin is maintained outside GitLab, so your team would own its troubleshooting and version choices. A practical test is to ask whether two or more people can run a pilot, diagnose a failed clone and perform a Runner upgrade without outside help. If one of those steps depends on a single person, plan for that gap in the scope.

What recurring work follows a vSphere runner autoscaling go-live?

Recurring work falls into four areas. First, version alignment: GitLab's docs advise keeping Runner major and minor versions aligned with GitLab, and GitLab ships a minor release on the third Thursday of each month with patch releases twice a month, according to the GitLab maintenance policy. Second, template refreshes for operating system patches and toolchain changes. Third, plugin and vSphere API checks after each upgrade. Fourth, capacity reviews, log monitoring and rotation of the vCenter and runner credentials. Assign an owner for each area and add the cadence to your change calendar before go-live.

How does a vSphere runner support plan differ from a one-time build?

A one-time build delivers a configured fleet, its documentation and a handover, after which your team runs it. A support plan is an ongoing arrangement that covers the work that follows go-live, such as monitoring, Runner and plugin upgrades, template refreshes and help with incidents on the autoscaling path. DeployOpen offers ongoing support as a retainer, and the written scope defines what it covers. Ask each proposal which GitLab and Runner versions are covered, whether the community plugin and its upgrades are included, how incidents are reported and escalated, and what reporting you receive each month.

Does GitLab support the community fleeting plugin for vSphere?

GitLab does not test community plugins, so GitLab's own support covers GitLab and GitLab Runner, not how a community plugin behaves on your vSphere environment. Your plan must therefore cover the plugin itself: version pinning, testing after each Runner upgrade and a documented rollback path. Premium adds priority support for GitLab, which still does not extend to community components. Read the GitLab Runner documentation for the supported executor and plugin behavior, and confirm with the plugin maintainers which vSphere versions they test against before you commit to the design.

How DeployOpen scopes and supports vSphere runner autoscaling

DeployOpen plans, builds, documents and supports GitLab Runner autoscaling on your VMware vSphere environment inside your own data centers. We start from a written scope that names the community plugin, the template and the support boundary before any build begins.

How we scope

Confirm the vSphere version, cluster layout, template source and network segments the runner fleet will use.
Record the community fleeting plugin version and state in writing that GitLab does not test it.
Agree which runner, template and autoscaling failure cases the written scope covers and which it excludes.
Engagements are fixed-scope, so deliverables and timelines are agreed in writing before work starts.

How we build

Build the runner manager placement, vSphere service account permissions and template cloning settings in your environment.
Set maximum instance counts, idle capacity and cleanup behavior, then test scale-up and scale-down under realistic job queues.
Verify that runner and GitLab versions stay aligned, and that failed clones are removed rather than left running.
Test runner registration with authentication tokens and confirm jobs cannot reach networks outside their approved segment.

How we hand over

Document the template build process, plugin configuration, capacity settings and the steps to rebuild the fleet.
Provide runbooks for template updates, plugin version changes and the rollback path if a new plugin release fails.
Walk your platform and vSphere administrators through alerts, log locations and the checks that show a healthy fleet.
Deliver the configuration in your own repositories so your team owns the code, templates and change history.

How we support

Ongoing support, runner upgrades and continuous testing are available as retainers, scoped separately from the build.
Ask any support provider to confirm how they will test each new community plugin release before it reaches production.
Agree in advance who owns the response when a community plugin defect affects job scheduling, and who escalates it upstream.
Include runner version alignment with each GitLab upgrade stop in the retainer scope so the fleet keeps pace with GitLab.

GitLab Runner autoscaling on VMware vSphere