Skip to main content

LLM

https://blog.kagi.com/llms - LLMs are bullshitters. But that doesn't mean they're not useful.

Simplifying a little, LLMs have always been trained in the same two steps:

  1. The model is trained to predict what comes next on massive amounts of written content. This is called a "base" model.

Base models simply predict the text that is most statistically likely to be next.

  1. The base model is trained on curated sets or input:output pairs to finetune the behavior.

LLMs don't think; they act in probabilities

You should not go to an LLM for emotional conversations. An LLM is capable of emitting text that is a facsimile of what an emotional conversation sounds like. An LLM is not capable of emotions. Models outputting statistically probable text cannot and should not be a replacement for human connection.

https://philosophersmag.com/large-language-models-and-the-concept-of-bullshit/

https://link.springer.com/article/10.1007/s10676-024-09775-5

https://www.manning.com/books/machines-that-think

This engaging book begins by establishing a solid conceptual understanding of how large language models actually work, how they manage to sound so human, and what “thinking” really means for a machine.