• morrowind@lemmy.mlOP
    link
    fedilink
    English
    arrow-up
    10
    arrow-down
    11
    ·
    7 months ago

    I know most people are over AI, as am I, but if we’re gonna have it, I’m glad to see there’s a focus on it being local

      • morrowind@lemmy.mlOP
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        4
        ·
        7 months ago

        The goal is to make it work on device in the next 4 years. That’s the point of an “AI PC”

        • Possibly linux@lemmy.zip
          link
          fedilink
          English
          arrow-up
          10
          ·
          7 months ago

          Why would Microsoft want that? They make money from the cloud. There is a reason they want you to move to Azure.

          • shiftymccool@programming.dev
            link
            fedilink
            English
            arrow-up
            3
            ·
            7 months ago

            They want you to foot the electric bill for the LLM processing, they’re still going to collect your data. Double-win for MS!

            • gravitas_deficiency@sh.itjust.works
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              1
              ·
              7 months ago

              But they DO want that sweet sweet customer data. If you think there’s not going to be some sort of user data and behavior profiling bullshit going on at the very least, I’ve got a bridge to sell you.

      • morrowind@lemmy.mlOP
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        11
        ·
        7 months ago

        The MS implementations won’t, but once they build the capability, we can make our own

            • Possibly linux@lemmy.zip
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              7 months ago

              On which platform?

              Basically you need three things. You need the ollama software, a LLM model such as mistral and a front end like openwebui.

              Ollama is pretty much just a daemon that has a web api apps can use to query LLMs.

              • kaboom36@ani.social
                link
                fedilink
                English
                arrow-up
                1
                ·
                7 months ago

                Linux, specifically nobara (a gaming focused fedora distro) for me

                Do you have any guides you would recommend?

                • Possibly linux@lemmy.zip
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  ·
                  7 months ago

                  Actually it is pretty easy. You can either run it in a VM or you can run it in podman.

                  For a VM, you could install virtual manager and then Debian. From there you need to of course do the normal setup of SSH and disable the root login.

                  Once you have a Debian VM you can install ollama and pull down llava and mistral. Make sure you give the VM plenty of resources including almost all cores and 8gb of ram. To setup ollama you can follow the guides

                  Once you have ollama working you can then setup openwebui. I had to use network: host with the ollama environment variable pointed to 127.0.0.1 (loopback)

                  Once that’s done you should be able to access it at the IP of the VM port 8080. The first time it runs you need to click create account.

                  Keep in mind that a blank screen means that it can’t reach ollama.

                  The alternative setup to this would be podman. You theoretically could create a ollama container and a openwebui container. They would need to be attached to the same internal network. It probably would be simpler to run but I haven’t tried it.

    • webghost0101@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      9
      ·
      7 months ago

      Its only partly local and i bet only because the alternative is to expensive.

      Just look at the kinda crap windows pulls today.

      I thought i had disabled pretty much all the spyware, them on the microsoft online dashboard i found

      • full machine specs
      • A list off all applications installed
      • edge search history

      Btw, did you know that anything typed in start/search now counts as a search in edge? I wander whats behind that move.

      In other words i understand much of the ai may function on a local level but it be fully integrated with cloud systems and you can safely assume a full record of your prompts and uses is stored on microsoft servers and delivered to bill gates for reading before bed.

      • morrowind@lemmy.mlOP
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        1
        ·
        7 months ago

        full machine specs

        just fyi, every website you visit has access to this, it’s not private

        • webghost0101@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          3
          ·
          7 months ago

          Depends on your browser extensions, you can even pretend to be a phone if you want. ;)

          And also not the same level of detail as microsoft is just lifting from your hardware like motherboard model and manufacturer, how many drives, how they are formatted and how much space is used on each of em.

      • Railcar8095@lemm.ee
        link
        fedilink
        English
        arrow-up
        1
        ·
        7 months ago

        Btw, did you know that anything typed in start/search now counts as a search in edge? I wander whats behind that move.

        It searches for everything you search so that by the time you press enter, it already has the result of the last one, hence seeming faster.

        • webghost0101@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          7 months ago

          What i meant is, what good reason is there for local search to always count as a Bing internet search.

          If i wanted to look up something online id open up an internet browser with the search engine of my preference.

          Spamming my local search results with online rubbish is the opposite of value, even micrsoft devs acnowledge this which is why microsoft powertoys has macbook style local searchbar thats way faster and better at finding your files and offline.

          • Railcar8095@lemm.ee
            link
            fedilink
            English
            arrow-up
            1
            ·
            7 months ago

            Well, it does search online too. That the implementation is atrocious is a different thing.

    • BombOmOm@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      1
      ·
      7 months ago

      Yeah, that is a big deal for privacy reasons. There is no reason one needs to send such information to companies.