• PeterPoopshit@lemmy.world
    link
    fedilink
    English
    arrow-up
    5
    ·
    edit-2
    1 year ago

    Use llama cpp. It uses cpu so you don’t have to spend $10k just to get a graphics card that meets the minimum requirements. I run it on a shitty 3.0ghz Amd 8300 FX and it runs ok. Most people probably have better computers than that.

    Note that gpt4all runs on top of llama cpp and despite gpt4all having a gui, it isn’t any easier to use than llamacpp so you might as well use the one with less bloat. Just remember if something isn’t working on llamacpp, it’s also going to not work in exactly the same way on gpt4all.

      • NotMyOldRedditName@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        edit-2
        1 year ago

        Check this out

        https://github.com/oobabooga/text-generation-webui

        It has a one click installer and can use llama.cpp

        From there you can download models and try things out.

        If you don’t have a really good graphics card, maybe start with 7b models. Then you can try 13b and compare performance and results.

        Llama.cpp will spread the load over the cpu and as much gpu as you have available (indicated by layers that you can set on a slider)