I’d like to self host a large language model, LLM.
I don’t mind if I need a GPU and all that, at least it will be running on my own hardware, and probably even cheaper than the $20 everyone is charging per month.
What LLMs are you self hosting? And what are you using to do it?
Ollama, llama3.2, deepcode and a bunch of others.
Using a GPU but man they’re picky, they mostly want Nvidia gpus.
Do NOT be afraid to run on the cpu. It’s slow, but for 1 user it’s actually mostly fine.