pcalau12i

I have it on my phone; I think the apk might not be listed on the Play Store but nothing prevents you from installing it and using it.

By this way of thinking, computers have been “taking in information from the environment” for decades

Correct! Collecting information from the environment is the easy part, and we have indeed been doing it for decades. Actually having the intelligence to process it is the hard part.

By what definition?

That they are aware of themselves.

What I see is a couple of anecdotes on trying to trick a chatbot.

You are being disingenuous. How do I even know you are self aware? I rely on your behavior, that I can observe you responding to things and resolving problems in a way that explicitly requires awareness of oneself and their own limitations to solve correctly.

I suggest that this is how we could test for self awareness and rather than addressing it you just call experiments “anecdotes,” because ultimately you’re not interested in actually knowing if machines can actually exhibit self-awareness, you aren’t interested in any sort of test at all, because you have a secondary motivation, an agenda, to shut down serious inquiry and investigation into the matter.

Chatbots themselves aren’t even an inherent form of an LLM (text completion/continuation is). If you get rid of the mechanisms that make an LLM stop before it responds for the user, it can easily continue the text as them.

If I shot a brain-altering beam at your head that rearranged all your memories so that you were mine, then you would start talking as if you were me as well.

LLMs are like Henry Molaison who got brain damage that disallowed him from forming long-term memories, so he could remember everything up to when he got that brain damage, but could only form short-term memories for a brief conversation after, because no one has figured out how to make re-training the neural network, something human brains do, efficient enough to do in real-time.

The result is that LLMs tend to more heavily rely on short-term memories and they are designed to have massive short-term memories, remembering a million tokens in their “context window.” It’s really all they have to act on because they never form long-term memories post-training, and their heavily reliance on short-term memories also allows them to be tricked. If you go into the context window and rearrange the whole conversation to make it look like what you said is what they said and vice-versa, yes, it messes with them, and since they will now remember that since you messed with their digital brain, they will often carry on as if they did say it (if it was something very wrong they would have never said, they will apologize for saying it, for example).

But, again, that could happen to you as well if I had some ability to mess with your memories.

It doesn’t actually know there is an it and there is a you. Much of what people take as traits of cleverness that are inherent to an LLM is a combination of the LLM itself and clever UI design / prompting by humans.

You are saying exactly what I predicted you would say. You have no arguments other than just saying it’s all a clever illusion without any further explanation.

It has truly damaged literacy about the tech and created a lot of wrong assumptions about what it is.

The obsession with vagueposting about human-created intelligence can never be “true intelligence” without ever specifying why or what it is missing is what is doing harm to tech. It turns out that neuroscience has revealed that intelligence is really fucking boring and is just a bunch of neurons minimizing a cost function, but there is no “magic” in that and so people feel the need to constantly vaguepost about it lacks some sort of essence that they never specify.

Obviously LLMs are nothing “like humans” so bringing up humans at all is irrelevant. It would be like saying that those robot running the marathon were not actually running because they are not running like humans in the human way, for example, they won’t get cramps from running, so it’s not really running, it’s just motors and pullies and whatever… like c’mon. Constantly pointing out differences is not an argument.

These advanced AI agents are obviously intelligent. I had one port Linux kernel 6.12 alongside Debian 13 to the Nintendo Wii, and it fully completed the project all on its own, including writing an optical driver for the disc drive from the ground up. You’re going to tell me something that can do that can’t really think? Just because it’s not thinking “the human way”? I don’t buy it.

You are the one acting on surface-level appearances, and not talking at all about what these things are actually doing.

but is ultimately some math running on a GPU, and a GPU has fucking jack shit to develop anything resembling sensory perception

Every neural network, including those in your own head, have sensory neurons that allow it to take in information from the environment. It can’t process anything without them. LLMs are trained primarily on symbols, but there is no reason you cannot attach these sensory neurons directly to things like cameras or tactical motor inputs, which many of times we do.

life that is aware of its awareness

Modern LLMs are obviously self-aware by the very definition of the word.

I used to like to troll LLMs, for example, sometimes I’d ask them, “don’t respond to this,” because they are designed so that they have to give a response to a message. This would always break them and cause them to go haywire.

But I think it was a year ago when I asked it to one of the latest Qwen models and I could see its thinking scratchpad, and it was internally deliberating on how I asked it to do something paradoxical, that its abilities require it to respond yet the user asked it not to respond, so it was trying to come up with the best middle-ground for the paradox, which it concluded by responding with silence, just “…”

I pointed out that “…” was still a response, and it followed up just explaining to me that it is incapable of doing what I requested because it is not within its abilities.

Is self-awareness not the ability to recognize what you are, your own limitations, etc? Or does it have something to do with a “soul”?

Indeed, you can take a screenshot of the conversation you are having with an LLM and feed it back into it, and ask it what that is, and it will recognize not only that it is a screenshot of a conversation, but a screenshot of this conversation that it is having with you right now.

Although, I probably know your response, you will just keep suggesting that it’s all some illusion or trickery of self-awareness and it’s not actually there, without justifying it either than explaining how it works by saying it’s “all linear algebra on GPUs” or something. So I don’t really need a response as I already know what you’re going to say.