• Eccitaze@yiffit.net
    link
    fedilink
    English
    arrow-up
    8
    arrow-down
    1
    ·
    edit-2
    3 months ago

    People always assume that generative AI (and technology in general) will continue improving at the same pace it always has been. They always assume that there are no limits in the number of parameters, that there’s always more useful data to train it on, and that things like physical limits in electricity infrastructure, compute resources, etc., don’t exist. In five years generative AI will have roughly the same capability it has today, barring massive breakthroughs that result in a wholesale pivot away from LLMs. (More likely, in five years it’ll be regarded similarly to cryptocurrency is today, because once the hype dies down and the VC money runs out the AI companies will have to jack prices to a level where it’s economically unviable to use in most commercial environments.)

    • Zron@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      3 months ago

      To add to this, we’re going to run into the problem of garbage in, garbage out.

      LLMs are trained on text from the internet.

      Currently, a massive amount of text on the internet is coming from LLMs.

      This creates a cycle of models getting trained on data sets that increasingly contain large sets of data generated by older models.

      The most likely outlook is that LLMs will get worse as the years go by, not better.