What Is GPUWorks? The Beginner's Guide to the 40MB Open-Source GPU Fleet Platform

CodexFuse Team · 2026-11-05T05:57:20.018Z · 20 min read min read

What Is GPUWorks? The Beginner's Guide to the 40MB Open-Source GPU Fleet Platform

What Is GPUWorks?

GPUWorks is an open-source platform that turns a mixed fleet of GPU servers into one cluster for serving AI models, with a control plane and web interface that download as roughly 40 megabytes. If you searched for what is gpuworks, here is the short version: it is a lightweight management layer that pools graphics processors from different vendors, deploys models from a tested catalog, and serves them behind a single OpenAI-compatible endpoint, without requiring a pre-existing Kubernetes cluster.

This is gpuworks explained in plain terms. ZStack, a cloud infrastructure company founded in 2015, open-sourced GPUWorks on November 4, 2026, under the Apache License 2.0, with the code published in the flitai-io/gpuworks repository on GitHub. The project calls itself a Model-as-a-Service platform, which means the model is the product: you bring the GPU hardware, and GPUWorks handles placement, serving, and monitoring.

The control plane runs on an ordinary Linux host or an Apple Silicon Mac, and GPU nodes join it over an outbound WebSocket connection, so nodes need no inbound ports. From a browser, an administrator onboards nodes, views available compute, and deploys models.

The installer verifies its own checksum during setup, and if you ever need to check a file hash by hand, a SHA-256 hash generator computes the same kind of digest in your browser. The machine that runs the control plane is the same kind of box described in our guide to VPS hosting, except here the workload is inference, not websites.

The full arc from empty host to serving model has five steps:

1. Install the GPUWorks server on one host with a single command, and open its web interface on port 8800.
2. Add GPU nodes from the web interface, by running the generated agent command on each GPU host.
3. Pick a model from the built-in catalog, and review the memory estimate and placement the scheduler proposes.
4. Deploy the model, which starts the matching inference engine on the chosen GPUs.
5. Call the model through the OpenAI-compatible gateway, using an API token created in the interface.

That arc is the whole pitch. Everything after this section is detail: what the small download buys you, what hardware it supports, what the catalog actually contains, and where the vendor's numbers need careful reading.

Why Does a 40MB Control Plane Matter?

A 40MB control plane matters because GPU management software usually grows heavy, and heavy management software is its own project to run. GPUWorks describes its download as 40-plus megabytes, including the web interface, while the repository title rounds the same platform to about 45 megabytes. The GPUWorks 40MB figure is a size class, not an exact measurement: this is a small, single-binary control plane, not a stack of databases, queues, and orchestrators.

The server and the agent are each one static binary, and the server keeps its state in a single SQLite file. That design has a practical consequence for small teams. A platform that starts in about a minute on a spare machine can be tried before it is trusted, and a control plane with few bundled components has fewer pieces to patch.

The 60-second claim needs the same careful reading. It describes getting the platform itself running, not onboarding a thousand servers, installing drivers, or pulling model weights.

Those steps each take their own time, and the release does not claim otherwise. The honest test is whether the small footprint survives contact with real hardware, which is why the catalog and the hardware matrix matter more than the download size.

How Do You Install GPUWorks?

You install GPUWorks by running one installer command on a Linux host or an Apple Silicon Mac, then adding GPU nodes from the web interface. This is the gpuworks install path as documented in the project repository, and the repository documentation is the authoritative place for the exact syntax, which may change between releases.

The server install uses a download script from dl.gpuworks.ai, which picks the fastest download source, verifies the checksum, and sets up a service on port 8800. The installer prints the web interface address and the initial admin password when it finishes. Adding the quiet flag accepts every default, and a manual install route exists for administrators who want to check each step themselves.

Each GPU node joins by running a generated command from the Compute nodes section of the web interface, which installs the agent and points it at the server with a node token. The agent dials out over WebSocket, so a node behind a firewall that blocks inbound connections can still join. Nodes that cannot reach out are handled the other way around, with the server dialing in.

The repository also documents installing with an AI coding agent, by pasting a prompt that runs the installer and verifies the result. If you use that route, our guide to how to use Claude Code covers the permission habits that keep an agent with shell access safe. After install, verify the server with its health endpoint, which should report a status of ok, and confirm the install summary shows the address, install directory, data directory, and password location.

Deployment then follows the catalog. Open Deploy, pick a model such as Qwen3-1.7B and a recommended inference template, review the configuration evaluation, and deploy. Create an inference token under API Tokens, and call the model by its public model name through the gateway on port 8800.

What Hardware Does GPUWorks Support?

GPUWorks supports six accelerator families in one cluster: NVIDIA, AMD, Huawei Ascend, T-Head PPU, Hygon DCU, and Apple Silicon. The point of the list is the mix.

Most schedulers assume one kind of GPU, so a team that buys whatever hardware is available ends up running several separate stacks. GPUWorks hides those differences behind one scheduler, one catalog, and one gateway.

The support matrix in the repository grades each family by feature, and the grades are worth reading before planning a rollout. Whole-GPU deployment is supported across all six families, but shared-GPU and sleep-and-wake support vary, and prefill-decode disaggregation is experimental on Hygon. Apple Silicon is a special case: it runs native processes such as MLX and llama.cpp, with no shared-GPU or multi-node support listed.

GPUs are detected with the vendor command-line tools, such as nvidia-smi and npu-smi, or, for AMD, through sysfs and the kernel driver interface. That matters because it means no single vendor library sits between GPUWorks and the hardware. The repository also notes that Kubernetes clusters and Slurm partitions are planned as future capacity sources, not current ones, so teams that already run those systems should read GPUWorks as a serving layer to evaluate, not a replacement they can drop in today.

What Is in the GPUWorks Model Catalog?

The GPUWorks catalog ships 392 models from 77 organizations, with 1,235 launch specifications that pin each model to an engine, an accelerator, a container image, and fixed weight revisions. A launch specification is the reusable profile that bundles the image, the inference engine, the weights, and the parameters for one model on one kind of chip. Picking a model means picking a specification that is already known to work, instead of hunting for flags.

The catalog covers chat, reasoning, and code models such as Qwen, DeepSeek, GLM, Kimi, MiniMax, Llama, Gemma, and gpt-oss, plus embedding, rerank, speech, image, and video models. Models outside the catalog can be registered from Hugging Face, ModelScope, a local directory, NFS, or an HTTP file server, and deployed with custom arguments. Before anything starts, GPUWorks shows how much GPU memory the weights, the key-value cache, and the engine overhead need, which GPUs the scheduler picked, and why the others were rejected.

The specification counts below are computed from the repository catalog totals, and the percentages show where the tested coverage actually sits. NVIDIA dominates the catalog, while AMD and Apple Silicon are thinly covered, which is the first thing a team on those chips should check.

Dividing the totals gives the average coverage per model: 1,235 specifications across 392 models is about 3.2 specifications per model. That average hides the skew in the table, so read the row for your hardware, not the average. A model with a dozen NVIDIA specifications and none for your chip is not a model you can deploy in one click.

How Does Time-Shared Scheduling Work?

Time-shared scheduling lets several large models take turns on the same GPU pool, instead of reserving a dedicated pool for each model. Enterprises do not need to keep a separate set of machines idle for a model that is only used at certain hours. GPUWorks coordinates the turn-taking automatically, with priority, yield-on-idle, and scheduled-rotation policies that administrators configure.

Two mechanisms make the sharing work. The first is dGPU, which puts a hard memory quota on each instance so several models fit on one card, using vendor-specific virtualization on each chip family.

The second is sleep and wake, which lets vLLM models take turns on the same GPUs without a full restart. Weights stay resident in host memory, so recovery skips engine restarts and repeated weight loading from storage.

The vendor reports that model service wake-up is about 200 times faster than a cold start, and gives a worked example. On 8 H20 GPUs, a cold start that reads weights from network storage takes 16 to 17.5 minutes, while L1 memory wake-up restores service in about 4 seconds.

Doing the arithmetic on those vendor figures gives 960 seconds divided by 4 seconds, or 240 times, at the short end, and 1,050 seconds divided by 4 seconds, or about 263 times, at the long end. The vendor's figure of about 200 times is therefore conservative against its own example, not exaggerated, and DeepSeek-V4.1-Flash returning in as little as 3.6 seconds would imply an even larger ratio on the same cold-start range.

Models that need multi-GPU coordination are switched as a whole GPU group, which reduces manual stop and start work and resource reconfiguration. Prefill-decode disaggregation adds a second kind of sharing: resources for input processing and for content generation are allocated separately, based on actual load, so scale-out targets the real bottleneck. The engine pairings for disaggregation differ by chip, with vLLM and NIXL on NVIDIA and vLLM-Ascend with Mooncake on Ascend, so check the row for your hardware before assuming a feature transfers.

How Does GPUWorks Scale to a Thousand Nodes?

GPUWorks scales by starting small and adding nodes to one management plane, with the vendor reporting unified management of more than 1,000 nodes. Bare metal, virtual machines, and cloud containers across on-premises data centers, private clouds, and public clouds sit on that single plane. That reduces the burden of maintaining each environment separately, which is the usual cost of a scattered fleet.

The headline test is vendor-published and must be read as such. In tests on physical hardware, one GPUWorks server managed 1,003 GPU servers and brought up more than 500 inference replicas in 4 minutes, delivering 69,000 tokens per second, while the server itself used only 9 CPU cores and 1.6 gigabytes of memory. The repository adds detail the release omits: the drill ran on rented GPU machines on Alibaba Cloud, across 7 regions, with 2,000 concurrent requests through the gateway.

Three caveats from independent coverage apply to every one of those numbers. First, the release does not publish the GPU model in those servers, the GPUs per server, or the network layout, so 1,003 is a count of servers, not of GPUs.

Second, the throughput figure names no model, prompt length, batch size, or precision, so it cannot rank GPUWorks against another serving stack. Third, the replica rate of more than 500 in 240 seconds is at least 2.1 replicas per second on average, which shows scheduler pace, not an end-to-end user experience, because weights may already have been staged on the nodes.

The management-server overhead implied by the test is strikingly small, and the arithmetic is worth doing once. Dividing 1.6 gigabytes by 1,003 servers gives about 1.6 megabytes of server memory per managed server, and 9 cores across 1,003 servers gives about 111 servers per core.

Those ratios describe the management server only: the GPU nodes still consume whatever the models demand. When a model outgrows one host, it can be served in parallel across multiple machines, and as request volume grows, administrators spin up additional replicas to share the load.

GPUWorks vs vLLM: What Is the Difference?

GPUWorks vs vLLM is a category question, not a contest between two serving engines, because GPUWorks sits above vLLM rather than replacing it. vLLM and SGLang are the inference engines that actually run a model on a GPU, handling batching, sampling, and token generation. GPUWorks decides where each engine runs, which hardware it runs on, and how requests reach it, then serves the result behind one OpenAI-compatible endpoint.

That layering is the cleanest way to understand the product. If you already run vLLM by hand, GPUWorks does not ask you to adopt a proprietary runtime.

It asks you to hand the placement, scaling, sharing, and monitoring of those engines to a small control plane, and it keeps the engine choice visible in every launch specification. A team that only needs one model on one machine gains little from that layer, while a team juggling several models across mixed hardware gains the most.

Two related products are worth naming once, to prevent confusion. Algeon is a GPU-in-SQL query engine, which is a different product class from a fleet serving platform, and it is not a rival to GPUWorks.

Training-heavy shops may still lean on batch schedulers, and teams already invested in Kubernetes may see little reason to switch. The sweet spot is an organization that wants to serve models on mixed hardware without building a platform team first.

What Does the OpenAI-Compatible Gateway Give You?

The OpenAI-compatible gateway gives you one endpoint for every model in the cluster, so existing application code can point at GPUWorks with little change. Chat, completions, embeddings, rerank, audio, images, video, and realtime are served under the version-one path, with message and response endpoints passing through. That compatibility is the practical reason a team can try self-hosting without rewriting its clients.

Behind the endpoint, GPUWorks routes by least-request, prefix-affinity, or failover rules, and it tracks API keys with per-key token usage. Administrators can view GPU health, request latency, and usage, and diagnose failures with a built-in AI assistant. Each scheduling decision can be inspected, so teams can see why a deployment landed where it did.

Operations tooling rounds out the gateway. GPU and engine metrics, searchable logs, and per-request traces with a timeline are available in the interface, with trace export in the OpenTelemetry format.

A web terminal reaches hosts and containers, and a built-in benchmark sweeps concurrency to find the service-level limit. A read-only MCP endpoint lets Claude Code or any MCP client ask questions about the cluster, and models and container images can be imported offline within a corporate network, which matters for air-gapped sites.

Who Should Use GPUWorks, and Who Should Wait?

GPUWorks fits teams that already own or rent GPU hardware and need to serve several models on it without hiring a platform team. The clearest fits are small labs with a handful of mixed cards, enterprises with scattered on-premises and cloud GPUs, and agencies that run client models on rented instances from providers such as RunPod, Vast.ai, or Alibaba Cloud. Those teams gain the most from one catalog, one scheduler, and one endpoint.

Teams that should wait include anyone whose hardware sits outside the tested matrix, anyone who needs Kubernetes or Slurm integration today, and anyone who wants training and fine-tuning on the same platform now. The repository is explicit that serving comes first, with fine-tuning and training jobs planned for the same accelerators later. A single-model, single-machine deployment also gains little, since the control plane solves a fleet problem.

Evaluation should start small, on one spare machine, with one catalog model on hardware the matrix covers. Confirm the memory estimate against the real run, confirm the wake-up behavior on your own workload, and confirm the gateway routing with your own client. A platform that starts in a minute earns trust the same way any open-source infrastructure does: by working on your hardware, not by its launch numbers.

What Are GPUWorks' Limits and Open Questions?

GPUWorks' limits start with the age of the public project and the thinness of independent evidence. The GitHub repository was created on October 22, 2026, and the open-source announcement followed on November 4, 2026, so the public track record is days old at the time of writing. At fetch time the repository showed 98 stars, 29 forks, and 1,067 commits, which describe a project developed before its public launch, not a community verdict.

The second limit is the missing detail behind the scale test, already flagged by independent coverage. No GPU model, GPU count per server, network layout, model name, or batch configuration is published for the 1,003-server run, so the numbers cannot be reproduced or compared. The third limit is catalog skew: AMD and Apple Silicon coverage is thin, and feature grades vary by chip, so the one-click promise depends heavily on which hardware you own.

Open questions for any evaluator include load balancing, queueing, and model versioning behavior behind the gateway, which the release does not detail. Those features separate a demonstration from a production service, and they belong on the checklist of any team that evaluates the project. The announcement also names no price, because the code is free, and no support or governance model, so support expectations should be set from the repository, not from the press release.

ZStack GPUWorks: What Did the November 4 Launch Actually Change?

The November 4 launch changed the availability of GPUWorks, not its existence, and that distinction matters for anyone reading the announcement. ZStack GPUWorks is the same platform under its company name: ZStack open-sourced its GPU management and inference platform on November 4, 2026, under the Apache License 2.0, which lets companies use, modify, and redistribute the code, including inside paid products. The repository predates the announcement, with creation dated October 22, 2026, so the launch is best understood as a public release of existing work, not a project written in a day.

What is genuinely new for outsiders is the combination: a small open-source control plane, a tested model catalog, mixed-vendor scheduling, and an OpenAI-compatible gateway, offered without a Kubernetes prerequisite. Before the launch, that combination was a vendor product story.

After it, any team can download it, read the code, and test it on a spare machine. That is the real change, and it is a useful one even if the scale numbers await independent confirmation.

Watch what happens next in the repository, not in the press cycle. Contributor activity, issue handling, catalog growth on AMD and Apple Silicon, and the promised Kubernetes and Slurm capacity sources will tell you whether GPUWorks becomes infrastructure or remains a launch story. For now, it is a credible, small, and testable option for GPU fleet serving, with vendor numbers that deserve the careful reading this guide has given them.

Sources

ZStack, "A 40MB Platform to Manage a Thousand GPU Servers: GPUWorks Is Now Open Source" — GlobeNewswire, November 4, 2026; fetched November 5, 2026. (Open-source launch, Apache License 2.0, 40-plus megabyte download, single-command 60-second install, no pre-existing Kubernetes required, 1,003 GPU servers, more than 500 replicas in 4 minutes, 69,000 tokens per second, 9 CPU cores and 1.6 gigabytes of server memory, nearly 400 models and more than 1,000 launch configurations, NVIDIA, AMD, Ascend, T-Head PPU, and Hygon DCU support, time-shared scheduling, 200-times wake-up example, repository flitai-io/gpuworks.)

flitai-io/gpuworks, GitHub repository and README — fetched November 5, 2026. (Repository created October 22, 2026; 98 stars, 29 forks, 1,067 commits at fetch time; 40 megabyte download in the README body with an about-45-megabyte repository title; 392 models from 77 organizations, 1,235 launch specifications by accelerator; server and agent as single static binaries, SQLite state, WebSocket agent connection, port 8800, install commands, supported-hardware matrix, dGPU, sleep and wake, prefill-decode disaggregation, OpenAI-compatible gateway, MCP endpoint, offline import, Kubernetes and Slurm as future capacity.)

Tech Insider, "GPUWorks Open Source: 1,003 GPU Servers, 40 MB" — November 4, 2026; fetched November 5, 2026. (Launch-day analysis: Apache 2.0 terms, control plane including web interface, missing GPU model and network detail in the scale test, 60-second claim as platform startup rather than fleet onboarding, derived per-server overhead ratios, vLLM and SGLang as the engines beneath the platform.)

Aura-V Intelligence, "GPUWorks launches 40 MB open-source AI inference platform for heterogeneous GPU clusters" — November 4, 2026; fetched November 5, 2026. (Launch brief: mixed Ascend, T-Head PPU, Hygon DCU, and NVIDIA cluster, OpenAI-compatible API, more than 1,000 launch configurations for nearly 400 models, single-command install in about 60 seconds without Kubernetes.)

Frequently Asked Questions

What is GPUWorks in one sentence?

GPUWorks is an open-source, roughly 40-megabyte platform that pools mixed GPU servers into one cluster and serves AI models behind a single OpenAI-compatible endpoint, without requiring Kubernetes.

Is GPUWorks free?

Yes, GPUWorks is open source under the Apache License 2.0, which lets you run, modify, and redistribute the code, including inside commercial products. You still pay for the GPU hardware, cloud instances, and electricity underneath it, and the announcement names no support pricing.

Do you need Kubernetes to run GPUWorks?

No, GPUWorks explicitly requires no pre-existing Kubernetes cluster, and it installs with a single command on a Linux host or an Apple Silicon Mac. Kubernetes clusters are listed in the repository as a future capacity source, not a current requirement.

How is GPUWorks different from vLLM?

GPUWorks is not a serving engine, so it does not replace vLLM. vLLM and SGLang are the engines that run models on GPUs, while GPUWorks sits above them, choosing placement, scaling replicas, sharing GPUs between models, and routing requests through one gateway.

What GPUs does GPUWorks support?

GPUWorks supports NVIDIA, AMD, Huawei Ascend, T-Head PPU, Hygon DCU, and Apple Silicon in one cluster, with feature grades that vary by family. Whole-GPU deployment is supported across all six, but shared-GPU, sleep-and-wake, and disaggregation support differ, so check the matrix row for your hardware before planning a rollout.