Bill Gates feels Generative AI has plateaued, says GPT-5 will not be any better::The billionaire philanthropist in an interview with German newspaper Handelsblatt, shared his thoughts on Artificial general intelligence, climate change, and the scope of AI in the future.

  • grabyourmotherskeys@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    ·
    11 months ago

    Another way to think of this is feedback from humans will refine results. If enough people tell it that Toronto is not the capital of Canada it will start biasing toward Ottawa, for example. I have a feeling this is behind the search engine roll out.

    • raptir@lemdro.id
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      1
      ·
      11 months ago

      ChatGPT doesn’t learn like that though, does it? I thought it was “static” with its training data.

      • HiggsBroson@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        11 months ago

        You can finetune LLMs using smaller datasets, or with RLHF (reinforcement learning from human feedback) wherein people can give ratings to responses and the model can be either “rewarded” or “penalized” based off of the ratings for a given output. This retrains the LLM to produce outputs that people prefer.

        • niisyth@lemmy.ca
          link
          fedilink
          English
          arrow-up
          2
          ·
          11 months ago

          Active Learning Models. Though public exposure can eaily fuck it up, without adult supervision. With proper supervision though, there’s promise.

            • niisyth@lemmy.ca
              link
              fedilink
              English
              arrow-up
              3
              ·
              11 months ago

              Bias is inevitable. Whether it is AI or any other knowledge based system. We just have to be cognizant of it and try to remedy it.

      • grabyourmotherskeys@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        11 months ago

        I was speculating about how you can overcome hallucinations, etc., by supplying additional training data. Not specific to ChatGPT or even LLMs…

    • Toes♀@ani.social
      link
      fedilink
      English
      arrow-up
      3
      ·
      11 months ago

      Toronto is Canadian New York. It wants to be the capital and probably should be but it doesn’t speak enough French.