So we’ll all have our own familiar soon?

  • MalReynolds@slrpnk.net
    link
    fedilink
    English
    arrow-up
    27
    ·
    3 months ago

    Ditto on the hate, technical, but important distinction here, they support open-weight ML. They do not release training source code or data sets to actually make your own (granted you’d need millions in video cards to do it, but still). Open-source gets thrown around a lot in AI, presumably virtue signalling, but precious few walk the walk.

    Never underestimate the value of getting hordes of unpaid workers to refine your product. (See also React, others)

    • istanbullu@lemmy.ml
      link
      fedilink
      arrow-up
      6
      ·
      3 months ago

      I understand the distinction, but it’s still waaay better than what OpenIAIClosedAI is doing.

      Also people are really good at reverse engineering. Open weights models can be fine tuned or adapted. I am trained a Llama 3 Lora not that long ago.

      • MalReynolds@slrpnk.net
        link
        fedilink
        English
        arrow-up
        1
        ·
        3 months ago

        Agreed, and the chance of it backfiring on them is indeed pleasingly high. If the compute moat for initial training gets lower (e.g. trinary/binary models) or distributed training (Hivemind etc) takes off, or both, or something new, all bets are off.

        • istanbullu@lemmy.ml
          link
          fedilink
          arrow-up
          1
          ·
          3 months ago

          The compute moat for the initial training will never get lower. But as the foundation models get better, the need for from-scratch training will be less frequent.