How do you increase CPU and GPU utilization for RL projects?

We always learn how to do RL but never how to make optimal use of compute. Did anyone gain experience? Better parallelization, MPI, distributed RL etc.