垂直领域大模型微调实践经验 (English)

Generated: 2026-06-20 14:03:01

---

After two years of working on vertical domain large model fine-tuning, my deepest realization is this—this thing really isn't just about piling on parameters.

Let's not beat around the bush. I'll just tell you about the pits I've fallen into. From GPU memory blowouts to models spouting nonsense, from feeding in ten thousand pieces of garbage data until the model had a total "memory wipe," to later seeing a single high-quality Q&A tangibly improve model performance… Today I'm laying out all these experiences one by one. After you read this, you'll at least save yourself the half a year I wasted on trial and error.

---

Let me start with a story—then you'll understand why I went all in on fine-tuning

A while back, a client insisted I use GPT-4 for medical Q&A. Every single call had to stuff in a long prompt like "You are a professional medical assistant, please analyze the following lab report…" Sure, but just change a few words in the question, and the answer would start drifting. Once the model actually said, "Based on your lab results, you might be pregnant"—and it was a hepatitis B report. My blood pressure shot through the roof.

After a month, the API bill made my heart ache. Spending more money doesn't guarantee getting things done right.

Later, I fine-tuned a 7B model on medical data and deployed it locally. On specific tests, it was more reliable than GPT-4, with nearly zero inference cost. Can you believe it? Fine-tuning makes the model truly understand your business logic, instead of having to cram every time.

---

Choosing a base model—don't worship parameters, match the problem

The first base models I tested: Qwen-7B, DeepSeek-R1-Distill-Qwen-7B, BLOOMZ-7B—all with the same medical dataset.

Guess how they ranked?

• BLOOMZ-7B was significantly better at medical knowledge Q&A than the other two. Why? Because its pre-training corpus already had a ton of PubMed literature. For knowledge-intensive domains like medicine, law, or scientific research, prioritize a base model whose pre-training data covers your field. Bigger parameters don't automatically work better—BLOOMZ-7B's medical performance crushed many 13B general-purpose models. Sounds counterintuitive, but that's the reality.
• DeepSeek-R1-Distill-Qwen-7B shined on reasoning tasks. It was distilled from a 671B model, so even a small amount of data could move the needle—great for diagnostic reasoning, contract clause analysis, and other logic-heavy scenarios.
• Qwen-7B has good Chinese support, a mature ecosystem, and works with LLaMA-Factory for one-click fine-tuning—most beginner-friendly.

My advice: First check the base model's pre-training corpus, and choose the one closest to your domain. Don't let people tell you bigger is always better. Pick the wrong base model, and no matter what you do later, you'll be fighting an uphill battle.

---

Model architecture—don't expect one model to rule everything

The biggest blunder I made here: trying to solve all problems with a single fine-tuned model.

Result? Medical Q&A got good, but everyday conversation turned dumb; professional terminology improved, but common sense answers fell apart. It was like I was messing with myself. Eventually I learned my lesson: let the fine-tuned model handle only core business reasoning, and delegate side tasks to RAG or a general model.

For example, my current medical product has this architecture:

• Fine-tuned model (7B quantized to 4.2G): handles lab report interpretation, symptom triage
• RAG: loads the latest drug instructions and clinical guidelines
• General model (original Qwen): deals with common questions like registration, department introductions

Three parts working together, each doing its own job. The fine-tuned model doesn't have to carry everything—7B parameters is plenty.

Also, something I've verified multiple times: A 10B-parameter model quantized to 4bit retains its capabilities significantly better than a 7B model quantized the same way. If your