• DarkThoughts@fedia.io
    link
    fedilink
    arrow-up
    1
    arrow-down
    1
    ·
    10 months ago

    Have you missed the first part where I explained that I couldn’t get it to run through my GPU? I would only have a 6650 XT anyway but even that would be significantly faster than my CPU. How far I can’t say exactly without experiencing it though, but I suspect with longer chats and consequently larger context sizes it would still be too slow to be really usable. Unless you’re okay waiting for ages for a response.

    • Flumpkin@slrpnk.net
      link
      fedilink
      English
      arrow-up
      1
      ·
      10 months ago

      Sorry, I’m just curious in general how fast these local LLMs are. Maybe someone else can give some rough info.