Fine-tuning OSS-120B / Qwen3-30B on 90k surgical Q&A: SFT vs DPO, multi-turn, and RAG integration?

I’m planning to fine-tune OSS-120B (or Qwen3-30B-A3B-Thinking-2507) on a mixed corpus: ~10k human-written Q&A pairs plus ~80k carefully curated synthetic Q&A pairs that we spent a few months generating and validating. The goal is to publish an open-weight model on Hugging Face and submit the work to an upcoming surgical conference in my country. The model is intended to help junior surgeons with clinical reasoning/support and board-style exam prep.

I’m very comfortable with RAG + inference/deployment, but this is my first time running a fine-tuning effort at this scale. I’m also working with a tight compute budget, so I’m trying to be deliberate and avoid expensive trial-and-error. I’d really appreciate input from anyone who’s done this in practice:

1. Multi-turn behavior: If I fine-tune on this dataset, will it noticeably degrade multi-turn / follow-up handling? Should I explicitly add another 5–10k dialog-style, multi-turn examples (with coreference + follow-ups), or will the base model generally preserve conversational robustness without increased hallucination?
2. SFT vs RL: The dataset is ~25% MCQs and ~75% open-ended answers; MCQs include rationales/explanations. Would you recommend RL after SFT here? If yes, what approach makes the most sense (e.g., DPO/IPO/KTO/ORPO vs PPO-style RLHF), and what data format + rough scale would you target for the preference/reward step?
3. Two inference modes: I want two user-facing modes: clinical support and exam preparation. Would you bake the mode-specific system prompts into SFT/RL (i.e., train with explicit instruction headers), and if so, would you attach them to every example or only a subset to avoid over-conditioning?
4. RAG / tool use at inference: If I’m going to pair the model with RAG and/or a web-search tool at inference time, should that change how I structure fine-tuning or RL? For example: training with retrieved context, citations, tool-call patterns, refusal policies, or “answer only from context” constraints.
5. Model choice: Between OSS-20B and Qwen3-30B-A3B, which would you pick for this use case? I slightly prefer OSS-20B for general non-coding performance, but I’m unsure whether its chat/harmony formatting or any architecture/format constraints create extra friction or difficulties during SFT/RL.