Nick Mermiges, an attorney from Columbia (South Carolina) and founder of PLI Labs and proprietarylegal.ai, upgraded his home server equipped with eight used Nvidia V100s. It now runs DeepSeek V4.1 Flash in "near-NVFP4" mode, where Nvidia's 4-bit NVFP4 format is applied to the "hot" experts of the model's Mixture-of-Experts (MoE) architecture, while the remaining experts operate in 2-bit. The memory upgrade, installation of an RTX 3090, and replacement of a faulty power supply increased token match with the reference NVFP4 version of DeepSeek to 95.1% and significantly expanded the context window—at the cost of some speed. This case is illustrative as an example of building private LLM infrastructure from outdated data center hardware for professions where data must not leave the office.

What Happened
The upgrade proceeded along several lines. The memory array was expanded to 1 TB using eight 128 GB modules running at 2933 MT/s; an RTX 3090 was added to the system; and the source of repeated crashes was identified in the power supply, which was ultimately replaced. The quantization scheme became granular: of the 15,360 routed experts in the MoE model, 5,949 (i.e., 38.7%) were converted to NVFP4, while the rest were set to 2-bit. Compared to the reference NVFP4 version of DeepSeek, next-token match increased from 92.2% to 95.1%, KL divergence decreased from 0.196 to 0.079 nats, and on the author's test of 513 questions, the number of changed answers dropped from 79 to 8. The context window reached 524,000 tokens, and the server handles two parallel requests; the price for accuracy is a portion of generation speed. Tom's Hardware describes this configuration in its headline as running 80% of DeepSeek v4.1 Flash at 4-bit NVFP4 precision.
Context
This case rests on three foundations. First is the mixture-of-experts architecture: DeepSeek V4.1 Flash is built from a large set of routed experts, among which a router distributes each token, meaning the load within the model is unevenly distributed. Second is quantization granularity: according to the method's logic, quality suffers more from quantizing frequently used "hot" experts than rarely used "cold" ones, so converting them to NVFP4 (Nvidia's 4-bit floating-point format) while keeping "cold" experts at 2-bit appears to be a justified compromise rather than uniform compression of the entire model. Third is the hardware: the V100 is a Volta-generation data center GPU with 32 GB of HBM2, long since removed from first-line sales and available on the secondary market; R580 is the last driver branch for this platform, setting a finite lifespan for such builds. The author's logic is tied to his profession: legal practice deals with confidential materials that must not leave the office perimeter, making local inference more attractive than cloud APIs.
Why This Matters for the Industry
For the industry, the main signal is the lowering of the hardware threshold: component-wise quantization of MoE models lowers the requirements for on-premise inference of large models to the level of the secondary GPU market. This blurs the lower price range of cloud APIs and opens a niche for "private AI" for legal, medical, and financial firms where data must not leave the perimeter. While this is currently a single reference rather than a standardized method, if independent replications with metrics against the full model appear, expert-granular mixed quantization should be expected as an option in open inference stacks; nearby, niche tools are likely to emerge—hot expert profilers, variable precision auto-tuners, ready-made deployment templates for Volta hardware—and integrators may begin selling assembled "private AI servers" to small firms. Within a two-year horizon, this pattern could shift the value of new GPU generations from "ability to run" to latency and energy efficiency, and blur the "cloud vs. on-premise" boundary: sensitive data is processed locally, while mass workloads move to the cloud. The limitation of this niche is that used Volta platforms are purchased at the end of their lifecycle, and the performance ceiling of outdated GPUs remains lower than that of modern accelerators.
Why This Matters for Users
For readers considering their own build, this case provides specific benchmarks. Speed: code generation runs at 150–180 tokens/s compared to 160–215 in the previous configuration, while legal texts slowed down by 9–23%—a transparent cost for the increase in accuracy that can be factored into capacity planning. There are also engineering lessons: expanding memory to terabyte scale requires NUMA tuning, and a build on Volta lasts only as long as the platform has driver support, which is important to consider when buying used GPUs. For owners of sensitive data, the application scenario is clear: prototypes of local AI within the office or client perimeter can be built without cloud tokens or API keys. However, the case does not provide a ready-made box product—a precise component list and a custom implementation of quantization will be required.
What Is Still Unknown / Limitations
The main caveat: the improvement was measured relative to an already quantized reference NVFP4 version of DeepSeek, not the full model—this is a reduction in error of one quantization relative to another, not an improvement of the model; relative to full precision, quality is likely lower. The test of 513 questions is the author's own: its composition and reference answers are not disclosed, and "changed answers" are not the same as "correct" ones, so the metric shows consistency between configurations, not correctness. It is not disclosed what routing statistics were used to select the 5,949 "hot" experts or which inference framework was applied, so independent replication is currently impossible. The role of the RTX 3090 in the final configuration and the relationship between the "80%" figure in the headline and the actual distribution of experts by precision remain unexplained. There is no independent replication: discussion on Hacker News counts 1 point and 0 comments, and materials about the case are limited to Tom's Hardware and this thread. For production with strict SLAs, the build is not yet ready—it is a single stand, not a replicable method.
Sources
- Tom's Hardware — article on server upgrade with eight V100s to run DeepSeek V4.1 Flash
- Hacker News — discussion of the eight V100 upgrade case for DeepSeek V4.1 Flash
Author
Look at AI, editorial team
