# How to train your own writing adapter with LoRA

> Teach an existing open-weights model a recurring style or task on your own hardware, without training a whole language model from scratch.

By Nate Morton · 2026-10-08 · [Workshop](https://lfkai.town/workshop/)

Web version: https://lfkai.town/workshop/local-writing-model/

You might want an AI model that drafts short customer replies, writes clearer documentation, or produces essay openings you actually like. A useful starting point is a collection of examples showing the work you want it to do.

LoRA, short for low-rank adaptation, trains a relatively small set of additional weights alongside an existing model. The original model’s weights stay frozen. The result is an adapter you load with that model to change how it responds.

The base model supplies its existing capabilities. Your examples guide the adaptation.

## Why do this instead of using a prompt?

Try a good prompt first. For occasional work, instructions and a few examples may be enough.

LoRA becomes interesting when you want the same behavior repeatedly and have examples that demonstrate it: a writing register, a consistent response format, or a narrow task.

Three approaches serve different purposes:

| Approach | What it does |
|---|---|
| Prompting | Supplies instructions and examples for the current request. |
| Retrieval | Brings relevant documents into the request, useful for current information and citations. |
| LoRA | Trains additional weights to adapt recurring behavior, style, or task performance. |

They can work together. An adapter does not replace factual context or editorial judgment. It can learn a pleasing rhythm and still invent a personal memory.

## 1. Choose a model and setup

This example uses MLX-LM on an Apple Silicon Mac. Windows and NVIDIA systems use different training stacks.

The example model is:

```text
lmstudio-community/Qwen3.8-27B-MLX-4bit
```

It’s the model I trained my own writing adapter on. Use whatever open-weights model your machine can run: a 3B or 8B model trains faster and fits in less memory, and the steps are the same. Swap its name into the commands below.

Because the base weights are quantized to 4 bits, this uses the quantized LoRA training path, commonly called QLoRA.

Create a workspace and install the training tools:

```sh
mkdir writing-lora
cd writing-lora

python3 -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade pip
python -m pip install "mlx-lm[train]"

python -m mlx_lm lora --help
python -m pip freeze > requirements-lock.txt
```

The virtual environment separates this project’s packages from other projects. The last command records your installed versions.

You need memory beyond the downloaded model’s file size: training also uses working memory for activations and optimizer state. Training on the 27B model with the settings below peaked at about 28 GB on my Mac.

## 2. Turn good writing into training examples

Each example pairs a request with the response you would want.

For an existing essay, write a short instruction describing the passage, then use the passage as the answer. Keep the instruction faithful to what the passage actually does.

Here is a synthetic format example, not a real author’s writing:

```json
{"source_id":"essay-001","prompt":"Write a short first-person passage about repairing an old chair.","completion":"The chair wobbled every time I leaned back. I had learned to balance my weight instead of fixing the loose joint. On Saturday I finally turned it over. The repair took less time than finding the glue."}
```

Save examples in `examples.jsonl`, with one JSON object per line.

Use writing you own or have permission to use. Remove duplicates, formatting damage, sensitive information, and habits you normally edit out.

A few hundred carefully chosen examples is a reasonable pilot, not a guaranteed threshold. Include different topics so the model has a chance to learn the writing rather than one repeated subject.

The `source_id` is bookkeeping. Every passage or revision from the same original document should share the same ID.

## 3. Keep some examples out of training

Create three sets:

- **Training:** examples used to update the adapter.
- **Validation:** separate examples used to monitor progress.
- **Test:** examples reserved for evaluating the selected result.

Split by original document. Putting one paragraph in training and a nearly identical revision in validation gives you a misleadingly easy evaluation.

The accompanying `build_dataset.py` groups examples by source, removes exact normalized completion duplicates, and creates the files:

```sh
python build_dataset.py examples.jsonl data
```

The result is:

```text
data/
  train.jsonl
  valid.jsonl
  test.jsonl
  split_sources.json
```

Also check example lengths with the model’s tokenizer and chat template. The request, response, and formatting all count, so a word limit does not guarantee that an example fits. The accompanying `check_lengths.py` counts them and lists any that run over:

```sh
python check_lengths.py lmstudio-community/Qwen3.8-27B-MLX-4bit data/train.jsonl --max-seq-length 1024
```

## 4. Train a first adapter

```sh
MODEL="lmstudio-community/Qwen3.8-27B-MLX-4bit"

python -u -m mlx_lm lora \
  --model "$MODEL" \
  --train \
  --fine-tune-type lora \
  --data data \
  --adapter-path adapters/writing-v1 \
  --mask-prompt \
  --iters 200 \
  --batch-size 1 \
  --num-layers 8 \
  --learning-rate 1e-5 \
  --max-seq-length 1024 \
  --grad-checkpoint \
  --steps-per-report 10 \
  --steps-per-eval 20 \
  --val-batches -1 \
  --save-every 50 \
  --seed 0
```

These are starting settings, not a universal recipe.

- `--mask-prompt` trains against response tokens rather than the request.
- `--batch-size 1` keeps the working batch small.
- `--num-layers 8` limits which layers receive adapters.
- `--grad-checkpoint` trades extra computation for lower activation memory.
- `--iters 200` means training steps, not 200 complete passes through the dataset.

The adapter configuration and weights are saved under `adapters/writing-v1`.

Watch training and validation loss. If training loss keeps improving while validation loss persistently worsens, the adapter may be memorizing the examples rather than generalizing.

More training is not automatically better. And loss is not a verdict about writing quality.

## 5. Compare it with the unchanged model

Use an unfamiliar topic and the same generation settings. Qwen3.8 can think out loud before it answers; the chat-template setting below turns that off so you compare the writing itself.

Without the adapter:

```sh
python -m mlx_lm generate \
  --model "$MODEL" \
  --chat-template-config '{"enable_thinking": false}' \
  --prompt "Write a short first-person passage about missing a train." \
  --max-tokens 250 --temp 0.7 --seed 42
```

With the adapter:

```sh
python -m mlx_lm generate \
  --model "$MODEL" \
  --adapter-path adapters/writing-v1 \
  --chat-template-config '{"enable_thinking": false}' \
  --prompt "Write a short first-person passage about missing a train." \
  --max-tokens 250 --temp 0.7 --seed 42
```

Repeat this with several prompts and seeds. Start with neutral requests so elaborate style instructions do not disguise what the adapter learned.

Ask:

- Does it follow the request?
- Is the rhythm or wording better?
- Does it work on topics outside the training material?
- Does it invent memories or repeat training passages?
- Would you keep more of this draft?

After selecting a version, evaluate the held-out test set:

```sh
python -m mlx_lm lora \
  --model "$MODEL" \
  --adapter-path adapters/writing-v1 \
  --data data \
  --test --test-batches -1
```

## 6. Use the adapter

You can continue using the generation command, or serve it through a local API:

```sh
python -m mlx_lm server \
  --model "$MODEL" \
  --adapter-path adapters/writing-v1 \
  --chat-template-args '{"enable_thinking": false}' \
  --host 127.0.0.1 \
  --port 8090
```

The adapter is tied to its base model and training configuration. It is not a universal style file you can attach to any model or runtime.

## What counts as success?

A useful first result is a model that produces drafts you prefer to its default output.

It may learn a register without becoming unmistakably one author. It may improve at one task while getting worse at another. Keep before-and-after examples, change one thing at a time, and inspect the dataset before assuming you need a larger model.

Local training does not require sending your writing to a hosted AI service. That changes if you use one to generate the examples, enable remote logging, or upload the adapter. Treat adapters as potentially sensitive because models can memorize training passages.

The process is straightforward: choose the behavior, show good examples, train an adapter, and test whether the change is useful.

## Workflow

1. **Choose a model and set up.** Pick an open-weights model your machine can run and install MLX-LM in its own virtual environment. This guide uses Qwen3.8-27B in a 4-bit MLX build.
2. **Turn good writing into training examples.** Pair a short, faithful instruction with each passage you want the model to learn from, one JSON object per line.
3. **Keep some examples out of training.** Split by original document into training, validation and test sets, and check that every example fits the sequence length.
4. **Train a first adapter.** Run a short LoRA training pass and watch training and validation loss together.
5. **Compare it with the unchanged model.** Generate from the same prompts and seeds with and without the adapter, on topics outside the training material, then evaluate the held-out test set.
6. **Use the adapter.** Generate from the command line or serve the model and adapter through a local API.

## Takeaways

A useful first result is a model that produces drafts you prefer to its default output. Choose the behavior, show good examples, train an adapter, and test whether the change is useful.

## Files

### build_dataset.py

```
"""Split prompt/completion examples into train, valid and test sets by source document.

Usage: python build_dataset.py examples.jsonl data [--seed 0]

Each input line is a JSON object with "source_id", "prompt" and "completion".
Every example from the same source lands in the same split, so a paragraph and
its near-copy can't sit on both sides of the evaluation. Exact duplicate
completions (after normalizing case and whitespace) are dropped.
"""
import argparse
import json
import random
import re
from collections import defaultdict
from pathlib import Path


def normalize(text):
    return re.sub(r"\s+", " ", text).strip().lower()


def load(path):
    examples, seen, dropped = [], set(), 0
    for number, line in enumerate(Path(path).read_text(encoding="utf-8").splitlines(), 1):
        if not line.strip():
            continue
        record = json.loads(line)
        missing = {"source_id", "prompt", "completion"} - record.keys()
        if missing:
            raise SystemExit(f"line {number}: missing {', '.join(sorted(missing))}")
        key = normalize(record["completion"])
        if key in seen:
            dropped += 1
            continue
        seen.add(key)
        examples.append(record)
    return examples, dropped


def split_sources(sources, seed, valid_share=0.1, test_share=0.1):
    sources = sorted(sources)
    random.Random(seed).shuffle(sources)
    if len(sources) < 3:
        raise SystemExit("need at least three source documents to make three splits")
    n_valid = max(1, round(len(sources) * valid_share))
    n_test = max(1, round(len(sources) * test_share))
    return {
        "valid": sources[:n_valid],
        "test": sources[n_valid:n_valid + n_test],
        "train": sources[n_valid + n_test:],
    }


def main():
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("examples")
    parser.add_argument("out_dir")
    parser.add_argument("--seed", type=int, default=0)
    args = parser.parse_args()

    examples, dropped = load(args.examples)
    by_source = defaultdict(list)
    for record in examples:
        by_source[record["source_id"]].append(record)

    splits = split_sources(by_source, args.seed)
    out = Path(args.out_dir)
    out.mkdir(parents=True, exist_ok=True)
    for name, sources in splits.items():
        rows = [r for s in sources for r in by_source[s]]
        with open(out / f"{name}.jsonl", "w", encoding="utf-8") as handle:
            for r in rows:
                # MLX-LM reads prompt/completion; source_id stays in split_sources.json.
                handle.write(json.dumps({"prompt": r["prompt"], "completion": r["completion"]}, ensure_ascii=False) + "\n")
        print(f"{name}: {len(rows)} examples from {len(sources)} sources")
    (out / "split_sources.json").write_text(json.dumps(splits, indent=2) + "\n", encoding="utf-8")
    print(f"dropped {dropped} duplicate completions")


if __name__ == "__main__":
    main()
```

### check_lengths.py

```
"""Count each example's tokens the way training will see them.

Usage: python check_lengths.py MODEL data/train.jsonl [--max-seq-length 1024]

Applies the model's chat template to the prompt and completion together, so the
request, response and formatting all count. Lists examples over the limit.
"""
import argparse
import json

from transformers import AutoTokenizer


def main():
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
    parser.add_argument("model")
    parser.add_argument("examples")
    parser.add_argument("--max-seq-length", type=int, default=1024)
    args = parser.parse_args()

    tokenizer = AutoTokenizer.from_pretrained(args.model)
    lengths, over = [], []
    with open(args.examples, encoding="utf-8") as handle:
        for number, line in enumerate(handle, 1):
            if not line.strip():
                continue
            record = json.loads(line)
            messages = [
                {"role": "user", "content": record["prompt"]},
                {"role": "assistant", "content": record["completion"]},
            ]
            text = tokenizer.apply_chat_template(messages, tokenize=False)
            tokens = tokenizer.encode(text, add_special_tokens=False)
            lengths.append(len(tokens))
            if len(tokens) > args.max_seq_length:
                over.append((number, len(tokens)))

    if not lengths:
        raise SystemExit("no examples found")
    lengths.sort()
    print(f"{len(lengths)} examples; median {lengths[len(lengths) // 2]} tokens, longest {lengths[-1]}")
    for number, count in over:
        print(f"line {number}: {count} tokens, over the {args.max_seq_length} limit")
    if not over:
        print(f"all examples fit within {args.max_seq_length} tokens")


if __name__ == "__main__":
    main()
```