Local Ai model training

I’m fine-tuning Qwen2.5-7B on my own dataset. It answers simple questions but hallucinates on complex questions. What is the best approach to improve accuracy and reasoning over my data? Should I use fine-tuning, RAG, agents, knowledge graphs, or another method? My hardware is RTX 5070 Ti (16GB), 64GB RAM, 20-core CPU.