It depend on your cpu.
If you get a error saying that the service is still running, then run the follwing command to stop the service before removing it.:
# rmmod -f kvm_intel
Reboot the computer after that :
# apt-get remove libvirt-bin
And finally, the modprobe command :
# modprobe -r kvm_intel
After that, you can run Virtualbox wihtout any error.