How to train an AI model for your business on a low-end PC

Fine-tuning an 8B model in 4-bit takes 6 GB of VRAM, which is what an RTX 2060 shipped with in 2019 (Unsloth, Unsloth Requirements, retrieved 29 August 2026). Ten hours of a rented RTX 4090, enough for several failed attempts, is $3.40 at Runpod's community rate. The hardware question that stops most people from starting is the smallest question in the project.

The question that stops them afterwards is the data. OpenAI's fine-tuning guide puts the floor at ten examples, recommends starting with fifty well-crafted ones, and adds the sentence worth printing out: "If 50 examples have no impact, rethink your task or prompt before adding training data." Fifty good examples of your business answering a question the way you want it answered is a weekend of writing. That is the budget that matters.

So this is the order I would work in, with the memory numbers and the rented GPU prices written next to each step. Everything sourced here is a vendor's own documentation, a published paper with a method, or a pricing page carrying a date, all checked on 29 August 2026. Where a number is my own arithmetic on someone else's rate, I say so.

Three different things get called training

"Train a model on our data" covers three jobs with three different costs. Working out which one you are asking for decides everything after it.

Pretraining from scratch is the one to rule out. Google trained Gemma 3 at 1B parameters on 2 trillion tokens and at 27B on 14 trillion (Google, Gemma 3 model card, retrieved 29 August 2026). No business case needs you to repeat that, and no low-end machine is going to.

Retrieval is the opposite end, and nothing is trained. Your documents get chunked, embedded and stored, and the model reads the relevant pieces at question time. The embedding model can be tiny: all-MiniLM-L6-v2 is 22.7 million parameters mapping text into 384 dimensions, Apache 2.0 licensed, and it runs on a CPU (Hugging Face, sentence-transformers/all-MiniLM-L6-v2, retrieved 29 August 2026).

Fine-tuning with adapters sits between the two, and it is what people usually mean. The base weights stay frozen and a small set of new matrices learns on top of them. LoRA's authors report cutting trainable parameters by 10,000 times and GPU memory by 3 times against fine-tuning GPT-3 175B with Adam, with no added inference latency (Hu et al., LoRA, arXiv 2106.09685). QLoRA quantises the frozen base to 4-bit and trains the adapters over it, which is how a 65B model fits on one 48GB GPU while keeping 16-bit fine-tuning performance (Dettmers et al., QLoRA, arXiv 2305.14314, May 2023).

The QLoRA paper's headline result is worth holding onto for scale. Guanaco reached 99.3 percent of ChatGPT's level on the Vicuna benchmark after 24 hours of fine-tuning on a single GPU. That was 2023 hardware against a 2023 benchmark, so read it as a claim about the method rather than a current score.

The memory table is most of the hardware question

Unsloth publishes minimum VRAM by model size and method, which answers the question people have about their own machine better than any benchmark does.

# Minimum VRAM to fine-tune, by method.
# Source: Unsloth Requirements, retrieved 29 Aug 2026.
#
#   params     QLoRA 4-bit     LoRA 16-bit
#      3B          3.5 GB            8 GB
#      7B            5 GB           19 GB
#      8B            6 GB           22 GB
#      9B          6.5 GB           24 GB
#     14B          8.5 GB           33 GB
#     32B           26 GB           76 GB
#     70B           41 GB          164 GB
#
# Unsloth calls these the absolute minimum.

Read the columns rather than the rows. The same 8B model wants 6 GB in 4-bit and 22 GB in 16-bit, so one precision setting moves you across three price brackets of hardware. Unsloth also names the usual cause of running out anyway, which is batch size, and the fix, which is to set it to 1, 2 or 3.

Two other lines on that page matter more than the table. Minimum CUDA capability is 7.0, covering V100, T4 and RTX 20 series upwards. A GTX 1070 or 1080 "works, but is slow." And with no NVIDIA card at all, Unsloth still runs for chat and data preparation, though not for the training pass.

The practical brackets fall out of that. 4 GB gets you a 3B fine-tune. 6 GB gets you 8B. 8 GB gets you 9B at 6.5 GB and not 14B at 8.5 GB. If your machine sits below all of it, skip to the rented GPU section, because the answer is 34 cents an hour rather than a new laptop.

One precision setting moves the same model across three brackets of hardware. Nothing else in this project has that much reach.

Most business questions are lookups, not training problems

The request that arrives most often is some version of this: the model should know our prices, our stock, our return policy, our client list. Every one of those is a lookup, and a fine-tune is a bad database.

Weights do not update when a price does. A model fine-tuned on last quarter's catalogue answers confidently about last quarter's catalogue, and the confidence is the failure, because a wrong price delivered fluently costs you more than no answer would have.

A fine-tune changes how a model answers. It does not change what is true this morning.

Retrieval covers that case, and it covers it on hardware you already own. The embedding model runs on a CPU. The store can be a file. Only generation needs a GPU, and you can rent that or call an API for it.

OpenAI's fine-tuning guide opens with a gate rather than a recipe: "Only invest in fine-tuning after setting up evals." The reason given is that you need a dependable way to tell whether the tuned model beat the base model. Without that file, a fine-tune becomes a feeling about output quality, and feelings do not survive a client asking what changed.

What a fine-tune is good at is narrower and worth naming. OpenAI's own list is classification, nuanced translation, specific output formats, and fixing cases where the model ignores instructions. All four are questions of behaviour rather than fact. If your problem is on that list, train. If your problem is that the model does not know something, look it up for the model instead.

The dataset is the project

The numbers on dataset size are smaller than most people expect, and they arrive with a stopping rule, which is the more useful half.

OpenAI puts the minimum at 10 examples, reports improvements from 50 to 100, and recommends starting with 50 well-crafted demonstrations. Then the rule: "If 50 examples have no impact, rethink your task or prompt before adding training data." That one line saves more money than any hardware decision here, because the failure it prevents is a month spent labelling data for a task that was never a training problem.

The other end of the scale is worth seeing for its ratio rather than its total. Stanford's Alpaca fine-tuned LLaMA 7B on 52,000 instruction-following demonstrations. Generating that data cost "less than $500" through the OpenAI API. The training run took 3 hours on eight 80GB A100s and cost "less than $100" (Stanford CRFM, Alpaca, 13 March 2023).

Five sixths of the budget on the famous cheap fine-tune went to data and one sixth to compute. Hardware anxiety is pointed at the smaller number.

In a business the fifty examples usually exist already, in unstructured form. Quotes you have sent. Support replies you were happy with. The WhatsApp thread where you explained the return policy correctly. The work is not inventing them. It is pulling each one apart into the input and the ideal output as a matched pair, which is tedious and cannot be delegated to the model you are training.

One thing to get right before that file exists. Strip names, phone numbers, addresses and account identifiers out of every example. A fine-tune bakes what it saw into the weights, and there is no delete statement for that. Retrieval keeps customer data in a store you can query, edit and drop rows from, which is a far better answer when someone asks you to remove their data.

You cannot delete a row from a weight matrix. Treat the training file as permanent, because it is.

Renting is cheaper than the argument about buying

If your machine cannot hold the run, the fix is an hourly rate rather than a purchase.

# Rented GPU, on-demand, community cloud rate.
# Source: Runpod pricing, page updated 27 July 2026.
#
#   RTX A5000   24 GB     $0.16 / hr
#   RTX 4090    24 GB     $0.34 / hr
#   A40         48 GB     $0.35 / hr
#   A100        80 GB     $1.19 / hr
#   H100 PCIe   80 GB     $1.99 / hr
#
# Ten hours on the 4090, which covers several failed
# attempts on a small model, is $3.40.

Secure cloud runs higher on the same page: $0.74 an hour for the 4090, $1.39 for the A100 80GB. Check the live listing before you plan around either figure, because this is the fastest-moving number in the post. The $3.40 is my arithmetic on their rate, not a quote from them.

That rate also settles the buy question without a spreadsheet. A thousand hours of a rented 4090 is $340. Count the hours you expect to use in a year before you price a card, and remember a card you own has to sit in a machine with the power supply and the cooling to match it.

Google Colab gives GPUs away, and its terms are worth reading before you depend on it. Colab's FAQ says "Colab resources are not guaranteed and not unlimited," that in the free tier "access to expensive resources like GPUs is heavily restricted," that notebooks run for at most 12 hours, and that the limits are not published because they change. That makes it a good place for a first attempt and a bad place for a run that has to finish before a client call.

What it costs to serve the result

A fine-tune produces an adapter, and serving it means holding a model in memory. This is where the machine you own comes back into the question.

Quantisation does the same job at inference that it did at training. Ollama documents q8_0 as using roughly half the memory of f16 "with a very small loss in precision," and q4_0 as roughly a quarter. That gap is the difference between a model that fits on your card and one that spills into system memory and crawls.

The line people miss is context. Ollama's FAQ states that required RAM scales by the number of parallel requests multiplied by the context length, against a default context of 4096 tokens, so four parallel requests at 2K context allocate as though you had asked for 8K. Flash attention slows that growth, and once it is on, the key-value cache can be quantised as well.

There is a separate post on this blog with the weights-only arithmetic for open models. The line worth repeating here is that the weights are the floor and never the total.

For a small business the base model to look at first is a small one. Gemma 3 ships at 1B, 4B, 12B and 27B, with 128K context from 4B upwards and 32K on the 1B, and Google's model card says the size makes deployment possible in "environments with limited resources," naming laptops and desktops. It carries the Gemma licence rather than Apache 2.0, which is a separate conversation with whoever reviews your contracts.

The order of work, with the GPU last

#  1  Write the eval set. 20 to 30 real inputs with the
#     output you want. Score the base model on it.
#  2  Try the prompt. Then try the prompt plus retrieval.
#     Score both on the same file.
#  3  If both fail the eval, write 50 examples.
#  4  QLoRA an 8B base. 6 GB minimum, batch size 1.
#  5  Score the adapter on the same eval set.
#  6  If it did not beat step 2, stop. The problem is the
#     data or the task, not the training.
#
# Steps 1 to 3 need no GPU.

Most projects stop inside the first three steps, and that is a result rather than a failure. A prompt that clears the bar is cheaper to run, cheaper to change and easier to explain to a client than an adapter nobody can inspect.

Step one is the part everyone skips. Twenty to thirty real inputs with the output you want, written down before you change anything, scored against the base model. Without that file you cannot tell a better model from a better mood.

Step six needs discipline. If the adapter did not beat the prompt on the same eval set, the honest read is that the data or the task was wrong, and another epoch fixes neither.

When training is the wrong answer

Five cases where the answer is not a fine-tune. Recognising them early is most of what expertise looks like on this problem.

The answer changes. Prices, stock, availability, anything with a date on it. Use retrieval, or a tool call into the database that already holds the truth.

The model has to be right about facts it has never seen. Retrieval again, and put the source in the output so a reader can check it.

You have fewer than about fifty examples of the behaviour you want and cannot write them. There is nothing to train on yet. Generating examples with a larger model gives you that model's habits rather than your own, which is the opposite of the reason you started.

The data cannot leave your machines and no card you have fits the model. Rent from a provider whose contract your client will sign, or drop this part of the scope.

You cannot measure the result. Fine-tuning with no eval set is paying to change something you have no way of seeing.

What I would tell someone starting

I have two products in production where a language model does the central work. Recuvix generates SEO-structured blog drafts. NovaEdge News ingests Hindi RSS feeds, rewrites each item and publishes on a schedule with nobody operating it. Neither one has a fine-tune in it.

That is not a position against fine-tuning. It is what happens when you write the eval set first. Both times a better prompt and a better pipeline cleared the bar, and the training run stayed on the list of things I had not needed yet.

So: write the eval set, try the prompt, add retrieval, and only then look at your VRAM. If you get that far, the number is 6 GB for an 8B model in 4-bit, or 34 cents an hour when your machine is short. Budget the compute under ten dollars and the data at a weekend, and you will be wrong in the direction that costs less.

Two other posts on this blog carry the rest of it. One covers ten open source models worth starting from and the licence attached to each. The other covers the model call being the cheap part of an AI feature, which is this same lesson seen from the production side.