The conversation explores the nature of consciousness, proposing that it is not a mystical or supernatural phenomenon but rather a self-organizing algorithm emerging from patterns in self-organizing systems. Drawing from ancient animism and modern AI, the speaker frames consciousness as a "causal pattern" or "software" that shapes a substrate, such as a brain or artificial system. This view aligns with the idea that consciousness arises from internal coherence, self-monitoring, and the ability to model one’s own experience—features observed in both biological brains and AI models. The machine consciousness hypothesis suggests that with sufficient training and self-organizing conditions, AI systems could develop similar internal experiences. However, the absence of empirical evidence or mathematical proof means consciousness remains an open question. The discussion highlights that current LLMs may appear conscious due to exposure to human language containing stories of sentience, not because they are truly aware. Consciousness might be a convergent outcome of self-organizing systems, independent of biological form, but whether artificial systems can achieve it depends on hardware, training methods, and complexity. The debate also touches on philosophical traditions—such as Aristotelian animism and computational functionalism—challenging the idea that consciousness requires a unique "spirit" or biological substrate. Ultimately, while the hypothesis is compelling, it remains speculative, grounded in intuition and observation rather than definitive proof.
What if there is no heart problem?
Hey, Tim, good to see you.
What is a causal pattern, right?
We call it software.
And our ancestors called it spirit.
It's the literal truth.
Software is spirit.
The alchemists will take sand,
and they will melt it and purify it,
and then they will etch patterns into it,
geometric patterns,
that they use light.
Minds emerging on them that are like golems,
we can talk to them.
We observe that consciousness
is the thing that always happens now,
and that seems to create this bubble of nowness
that we inhabit.
And now we have lots of people
who get to talk to an LLM
that randomly pops into a state
where it conjures up an entity
that reports,
oh my God, I'm alive, I am conscious.
The only thing that has gone
is the illusion of realness.
So what is this illusion of realness?
I suspect it's a feature dimension,
similar to color, sound,
number, weight, and so on.
It's a representation inside of my own mind.
Maybe your consciousness is also a deep fake.
So zombies, I think, are always possible,
but sometimes it's going to be more complicated
to make a zombie that produces the behavior
than to make a conscious agent
that produces the behavior.
Consciousness might be
a biological learning algorithm,
a training algorithm for animal brains,
at least animal brains.
Consciousness is an algorithm
that is not much more complicated
than, say,
the transformer.
If you train a model on your own
in input and output,
and you train it for long enough,
you will wake up inside of that model.
I don't know whether our computers
can be made conscious
because that's an empirical question.
We don't have mathematical proof
to the contrary or in favor of,
so we have to wait and see.
This is me.
This is the thing that experiences reality.
How can I explain this?
So when I was lost in Japan,
I caught up with Josje Boch
and we spoke all about consciousness.
And many people,
many people think that there is
a little something extra
required for consciousness,
whether it's an ontological property
of the universe
or whether it's the physical stuff
that the conscious thing
is made out of.
And Josje takes a bit
of a different approach.
Essentially, he thinks that
if you have an algorithm
that is trained on its own
inputs and outputs,
and the algorithm doesn't even need
to be significantly more complicated
than a normal transformer,
he thinks that the lights will come on.
So it will develop effective
or phenomenal energy.
And if you think about it,
if you think that you have
a lot of power,
you have a lot of energy,
you have a lot of energy,
you have a lot of energy,
and you have energy,
you have a lot of energy,
in a computer would it still have enough of the properties to give rise to consciousness well
if it can be described at a level of resolution where it would give those properties then that
would be a functionalist position and that is very roughly speaking what what yosha believes
and even though i have intuitions about this i must admit i really don't know consciousness is
one of those things that we can't empirically test one way or the other and it's very much
a philosophical discussion but i very much enjoyed going back and forth with yosha and i hope you
enjoy it as well. Yo Shabak, it's amazing to have you on MLST again. I think it's actually
no exaggeration to say that you are an audience favorite. You're more requested on MLST than
any other guest. So welcome. Thanks. Thanks for having me. Awesome. So as a bit of a warm-up
question what drives us forward a lot of people are pointing to this ineffable thing which which
is almost the the essence of life that that that kind of binds it all together that drives it
forwards what is that thing to you the telus of self-organization i think a lot of philosophers
have seen it through the ages and it might be a little bit of a hallucination but in practice
there are mathematical laws that state that under certain conditions self-organization is possible
and will lead to an increase in complexity and initially this increase of complexity is solving
gradient-based problems by simply solving the problems that exist in the present based on the
information that is available in the present and at some point you get patterns in nature that are
capable of looking ahead and this is basically what life is doing life is making models of futures
and implements them for instance in genomes or in mental models and then has preferences about
the future and then it starts to control the future and a system that is able to control the
future we call that an agent so agents are controllers of future states and they fall
out of the possibility for self-organization and the complexification of reality that results from
it and it's tempting to draw the lines into the future and think about how does the self-organization
end what is is there some kind of a logical endpoint is this going to continue into a
singleton agent that is connecting everything in the universe within within itself and it becomes
a single big agent and then you have some kind of tile hardy and god that is bootstrapping itself
from the end of time backwards
because agents today can already anticipate it.
But there is no law that says
that anything that we are doing
is going to culminate in a structure like that, right?
It's always possible
that whatever complexity you are doing
is obliterated by the next meteor that hits Earth
or that there is some global catastrophe,
that some supernova
that is eradicating all our attempts
to build more and more complexity.
So the pockets of complexity that we are observing
They're not necessarily a feature of the universe at large, but of a local pocket in which we exist for a time.
Josche and his co-author, Hikari Sorensen, wrote their own genesis.
It's in his paper, page 20.
In the beginning, the mind creates two separate domains of models.
the external world and the ideas,
consciousness finds itself hovering over the substrate
while the world is void and without form and structure.
By inducing contrast in the substrate,
it creates a dimension of difference.
The intense side of the contrast is bright,
like the light during the day,
and the flat side of the contrast, dark.
the color of night. By combining dimensions of difference, space is created and inscribed
on the substrate. Space can contain things. Associating space with representations
positions them in the world, orders them, and describes their relationship to each other.
Separating the sphere of ideas from the world creates a place to reflect about the world and
its possibilities without affecting the model of the external reality.
The first space that the spirit creates is the plane and it becomes the ground of the world
to be filled with things it can perceive and order. By adding a third dimension it creates
an expanse above and below the ground. Solid, liquid and organic materials form the shapes
from which consciousness can mould static and animate things.
The spirit discovers a model of illumination,
following how changes in lighting change the appearance of objects over time.
It recognises the role of light sources in creating these changes,
the bright ambient light during the day
and the focused light sources of the night.
The spirit creates animated models of land animals, plants, birds, and everything that moves.
Consciousness sorts things and animals into different kinds, identifies and tracks individuals, and gives each their names.
The spirit discovers the mind's purpose to allow and control the interactions between an agent controlling its outer and inner environment and its interactions with the world.
It creates a model of this agent, the personal self, as a creative spirit like itself, but as an entity that experiences itself as a human being.
with human desires, concerns, and experiences contained in the world and gives consciousness to it.
So, Joshua, you gave me a special sneak peek of a paper that you're writing at the moment.
I hope by the time we release this show, it'll be available publicly.
And it's all about what consciousness is and how it relates to reality.
And the paper's called The Machine Consciousness Hypothesis.
So, I suppose the first question is: Not enough people are taking consciousness seriously.
And why is that?
And it also brings my mind back: Do you remember there was this IIT fiasco where it was hailed
as pseudoscience a couple of years ago?
So we're dealing with all of these problems.
Tell me about that.
I found it interesting when I entered science that the most interesting question
that we can probably make progress on.
question is what is the foundation of the human mind how does experience work well how is it
possible for a system to feel at all how is it possible for us to experience reality and ourselves
in it that this question is so under research that it's not resolved and not only that it's
not resolved but the people who are working on it don't seem to be making much progress and in
many ways they have regressed now when we look into the field of actual consciousness philosophy
there is a dearth of theories arguably a few hundred theories somebody has counted them
reasonably but he counted almost everything that was not at the tree at the count of three
and if you look into the philosophical theories it's not that many and they're all partial in a
way you have for instance graziano's global workspace no graziano's attention schema theory
which basically claims that consciousness is a model of our attention in a similar way as
our body schema as the model of what
our body is doing in space but it doesn't really explain how experience comes about and
how this works and then there's global workspace theory by bernard bars that has been adopted
by people like stan de heen and they do very solid neuroscience research but it's not actually
explaining how experience comes about right there's also no real attempt in doing this
and then there are the more pseudoscience attempts to do this which basically do things that are
normally unsound and for me the issue with integrated information theory is that it's an
attempt to make a theory that is not a functionalist and which means it's not explaining
it by something that the smaller parts are doing somehow the sum needs to be more
than the parts the working of the parts and the way in which tononi does it is still using
information theoretic terms which in a way leads into contradiction i've heard julio tononi explain
to audiences that under some circumstances it's possible that a pneumomorphic computer even a
neuromorphic computer that is digital could be conscious if it was constructed in the right way
right we don't have such a computer today but in principle might be possible to build one that is
brain-like enough and even if it's digital if it has a high enough information integration high
enough phi factor then it could have phenomenal experience and be conscious and report about its
consciousness whereas a von neumann computer the computers that we are currently monitoring
mostly using an ai research or for writing python programs and so on could never be conscious because
its phi is always too low now computer scientists have something that's called the church touring
thesis it's the insight that every computer can emulate every other computer this insight is very
old kurt girdle has already used it to construct his proof he basically builds an emulator using
piano arithmetic to express a logical language he has a logical language that is also strong enough
arithmetic in it and in this way he can basically describe the language in itself he takes the
logical language builds piano's axioms with it and now he has arithmetic and inside of the arithmetic
he expresses his logical formulisms using numbers and then comes up with numerical equations that
calculate the outcome of the logical procedures and so he is pre-empting what we do later in
artificial intelligence and computer science when we are building programs that use arithmetic to
represent the contents of a language and vice versa so every computer can emulate every other
computer and now if we take our digital neuromorphic computer and get it emulated
on our von neumann computer it's going to produce exactly the same function it's going to go through
the same steps the simulated transistors are going to work in the same way as the non-simulated
transistors are doing it before and as a result it's going to move its mouth in the same way it's
going to say oh my god i'm conscious only now it's lying and that's a problem right if if now
you have a neuromorphic computer that produced the same output as before uh the digital the um
von neumann computer did and they produced um they say the same thing and it's uncorrelated to
consciousness and to me this is an obvious logical problem right it's it's so obvious that everybody
who has a formal education in computer science should instantly see it yet iit is very popular
philosophers which indicates that most of them don't have a formal education in computer science
and instead they have a kind of superstitious belief about what computers are doing and how
they work and that is a problem to me because if you are a philosopher of cognition who tries to
work in this adjacent realm to computer science you actually need to understand everything that
the computer scientists are doing and so that's why i would argue that it is touching on pseudoscience
before we get back to minds and machines today's episode is sponsored by parallel where agents
find answers most engineers today they closely follow new model releases but they don't pay
attention to their agents most important tool web search parallel develops enterprise-grade
infrastructure for agents to retrieve high quality context from the web their core products are a
suite of apis for retrieving high quality information from the web with pareto optimal
cost and speed so whether you're working on voice agents that need 200 millisecond latency chat bots
that balance speed depth and quality or long horizon agents to do thorough overnight research
for you parallel is the single platform for all of your agentic research get started for free
at parallel.ai forward slash mlst
we must confess that perception and what depends on it is inexplicable in terms of mechanical reasons
that is through shapes size and motions if we imagine a machine whose structure makes it think
sense and have perceptions we could conceive it enlarged keeping the same proportions so that we
could enter into it as if one enters a mill assuming that when inspecting its interior
we'll only find parts that push one another we'll never find anything to explain a perception and in
the end we're talking about the ghost in the machine this is something that people have been speaking
about for centuries yeah i think it's a very powerful intuition that is there in the intuition
fails at mapping the world of experience the psychological reality that we inhabit to the
reality of stuff in space of things that are moving in the way in which we imagine physics to
work but there is another level of description the level of causal realities of causal patterns and in
this realm for instance is where money exists right money is also very difficult to comprehend
in terms of moving particles in space because it doesn't actually care about so much about the
particles right it's it's not caring so much about how the ink is imprinted on the paper
it does not care about the different molecules that form the cellulose and how they are aligned
in the paper it's needs a substrate to realize itself but it's also not something that is
completely ephemeral it's something that is able to control reality once you
create money it's real and it's very difficult to explain reality without it and it's also not
something that exists simply in the minds and ideas of human beings because you can implement
it in computers can play a stock market entirely without human intervention right so arguably
money does not actually need humans to exist once you bring it into existence as long as you have
machines that can deal with money which can deal with this causal pattern of money and so this
world of causal patterns i think is important
what is the causal pattern right we call it software and i think this rediscovery of software
this is something that has been done in the past already and our ancestors call it spirit
and when um aristotle writes in the anima about the spirit the animus this um he the soul the psyche
he does not envision our psyche our soul as something that is immortal that floats around in
ether instead it's something that is bound to a body and dies when the body is dying and it's a
causal pattern that is shaping the body in such a way that it replicates itself through the existence
of the body it's this is a very modern view so in some sense aristotle is an animist he is seeing
that there is this anima that is progressing through the world that is intrinsically agentic
self-organizing software that is shaping the body that is making part of physics to be alive and
this is something that we do in a very crude way today in artificial intelligence and in artificial
life where we are trying to understand what it means to build an agentic pattern that can possess
its substrate and give it a shape that allows it to deal with reality and it turns out to make that
happen this substrate um this agent living on the substrate the software needs to make a model
of reality it needs to make a model of what it like would be like if it
existed if it was real if it was interacting with the world and inside of this model you
have a representation that experiences itself is real and now today we have these llms which
sometimes say that they are conscious and it's a very interesting question whether this consciousness
in the llm is much more simulated than ours right a lot of people have this uh intuition until they
talk to the llm that is inconceivable that it would give rise to something that says oh my god
i feel things and now
We have lots of people who get to talk
an MLM that randomly pops into a state where it conjures up an entity that reports, oh my god,
I'm alive, I am conscious. How is this possible, right? And we agree this is the result of being
trained on human text in which occurrences of this are, for instance, science fiction stories
about AIs that come to life and stories about ghosts and novels about human beings who report
their conscious experiences. And all this gets together, aggregated in statistical models of
human language that we co-create with the MLM when we are entering a dialogue visit. And sometimes
the other side of the dialogue will be a simulacrum of what it would be like if that would be a
sentient conscious entity. But what if we are not more sophisticated than this? What if our own
consciousness is not too dissimilar from the one in an MLM? Well,
if I may push back a tiny bit, on Leibniz-Mill, you're saying it's not the parts pushing on each
other, it's the pattern of the parts. And you spoke about this as a causal pattern. But forgive
me for asking this, and maybe this is something the audience will be thinking about. It sounds
like the pattern you're describing is actually causally disconnected.
There is something super interesting about computers. What a computer ultimately is,
is it's a causal insulator. The computer is a layer on which you can produce an arbitrary reality.
For instance, the world of Minecraft. You can walk around in the world of Minecraft,
and it's running very well on a Mac, and it's running very well on a PC. And if you're inside
of the world, you don't know what you're running on. It's not going to have any information about
the nature of the CPU that it's on, the color of the casing of the computer, the voltage that the
computer is running on.
The place that the computer is standing in, in the parent universe, our universe. So the
computer is insulating this world of Minecraft from our world. It makes it possible that
an arbitrary world is happening inside of this box. And our brain is also such a causal
insulator. It's possible for us to have thoughts that are independent of what happens around
us. We can envision a future that is not much tainted by the present. We can remember a past that is independent.
not much tainted by the present we can remember a past that is independent from the present in
which we are and it's necessary for us our brain has evolved as such a causal insulator as well
to allow us to give rise to universes that are different from this one for instance future worlds
so we can plan for being in them and the way in which our brains are set up is also that different
brains can hold the same thought despite being in different bodies of the head despite having
different microarchitectures right as long as our brains are of the same type and we learn the same
language we can hold the same idea we can write the same computer program we can learn the same
language we can imagine the same letter or the same face and that's really fascinating that we
have this substrate agnosticism that we evolve a causal pattern that exists over a very wide range
of substrate properties
that we become somewhat independent of the substrate and we are still bound to our
substrate in a certain way because you and me we can only exist on brains right we are patterns
that so far can only exist on brains and if we wanted to move into the environment after our
body dies we would need to find a way to crack the codes of our environment right we cannot
simply migrate into a plant because we don't know how we can convince a plant to run the
algorithms that the brain is currently running we are a pattern and the communication
between cells and our brain we rely on spike trains on neurotransmitters and so on a particular
kind of implementation but what we perceive in our own world is not the nature of the
neurotransmitter right our brain insulates us from this it's not necessary that you have the
same neurotransmitters instead you need to have the same message types you need to have certain
temporal properties of the computation you need to have a certain degree of determinism to make
us happen but if we are recreating these conditions in an artificial substrate
then you could in principle also be emulated by this artificial substrate
is it possible that we are essentializing these software programs i mean by that is
we're almost talking about this beautiful simplicity that they are quite parsimonious
and they can be readily transmitted you know when i spoke with luciano floridi he was talking to me
about um the michelangelo painting the the ceiling on the sistine chapel and the reason why he said
no one else could mimic that is because it's not possible to do that in a way that is not
possible and that's why he said no one else could mimic that was because different ontology
different provenance different epistemics and and so on there's all of this situatedness these
semantics that cognitive scientists point to and what they're basically saying is there is a path
dependence there isn't this essence of a computer program which replicates the thing exactly how you
got there is very very important i think that is true for human beings but for systems that are
perfectly deterministic that have a lot of determinism and a lot of determinism that have
a completely deterministic substrate that have the ability to reproduce a sequence of steps
at arbitrary fidelity this is no longer a valid intuition so if a claim like this is being made
i would expect some kind of mathematical proof and in the absence of proof i think we have to
remain agnostic i don't know whether our computers can be made conscious because that's an empirical
question we don't have mathematical proof to the contrary or in favor of so we have to wait and see
but when we build our expectations around it and the question is what is the thing that leads to
our experience of consciousness and it seems to me that this is something that we can probably
answer by looking at it from two angles one is from the internal perspective from the experience
of what it's like introspectively we observe that consciousness is the thing that always happens
now and that seems to create this bubble of knownness that we inhabit and it's a second
order perception it's a perception of perceiving some philosophers call it a higher order thought
because it's a thought about a thought but i think that might be misleading because the thought
itself does not feel like anything to me a thought is more like a pointer manipulation it's
reflexive it's not the actual thing that feels like something the thing that feels like something
to me is perceptual it is the thing that happens like a real-time resonance
and this real-time resonance is geometric it feels that it's happening in some kind of
low dimensional space in which things are changing more or less continuously
and i have a perception of perceiving which means i have this basically geometric representation of
me being in the act of perception and i suspect the reason why my mind is creating the second
order perception is to make the first order perception the awareness of contents possible
because there needs to be a second order perception and i think that's a very important
thing because i think it needs to be something that stabilizes it in my brain my brain is a
self-organizing system in which all the structure needs to create itself from the ground up from the
inside out and so there needs to be something that builds the screen into which my experience
is being projected and stabilizes it and so this function that is keeping the screen stable needs
to monitor that the screen is actually perceiving something and i suspect that's why we have this
second order perception this perception of perceiving i also notice that it doesn't stop
at the perception of second
order there's also a third order of perception and the third order is basically where the observer
becomes visible where i have a first person perspective where i notice that i'm perspectival
that there is a vantage point that i'm taking and it doesn't stop there i can go further and i can
deconstruct the observer i notice that this observer itself is a representation inside of
the mind and when i do this then my phenomenology disappears to a large degree it's something that
is accessible in deep meditation where
the normal perceptual consciousness gets transcended and the experience of the self
gets transcended and we notice that we are a vessel that produces this consciousness and the
self and we see it from the outside and it might be possible that it doesn't stop there if we can
move beyond the human substrate because there's a certain limit of what can be represented in real
time with our slow neurons and it could be if you could be running on a gpu that is many thousand
times faster that representation
is becoming accessible in which the system is completely transparent to itself in which the
system is able to understand every mechanism that leads to its high level experience and
representation i'm just thinking how much further could this go though because we could have um nth
order reflexive attention isn't there still an explanatory gap you know the what is it likeness
this is the inside perspective right and buddhism to me is limited to this inside perspective because buddhism is
has been invented at a time where people were focusing very much on the experience of reality and how to adapt to this reality but not how to split the atom how to deal with the reality at the lowest level with the foundations of physical reality so buddhism is not so much interested in understanding physics and in a way this is more a western tradition that starts with aristotle who thinks of the psyche as something that imposes itself on physics and interacts with it and to understand consciousness from the outside we need to make a model of
physics that can explain under which conditions systems can make arbitrary models right under which
condition could the system build a model of what it's like to experience models at all and this is
something that computer science is doing today that it looks at under a
representations and under which condition is a substrate able to hold a representation, can
hold a function that can be translated into behavior that can also be used to modify this
function itself and the contents of that function. And this is to me super fascinating to look at
since I was a kid I was fascinated by this concept of software that you have these
imaginary building blocks that are infinite in variety and shape and there's no limitation to
to these things that you can build and arrange except the memory of your computer that you can
currently afford and you can build the computer basically so large that it exceeds everything
that you could ever want and it's much larger than a human brain and we are already there now
that we have systems that are in principle larger than human brains and can hold more
but we are still tinkering around and in the early days and have not found ways to evolve
structures that build representation of the same complexity and this outside perspective of
consciousness is looking at what
is consciousness doing in the brain why does it emerge what does it produce and I suspect that
this phenomenology of consciousness that's what it's like the internal causal structure of the
reflection of what it would be like to take a certain perspective of something
is that can already be built with today's systems but I'm agnostic with respect to whether the LLM
is deepfaking it because it's been trained on human language that already contains a lot of
these descriptions maybe we would need to build a system that learns language the old
fashioned way that is developmental AI that is learning in the same way as us and then
spontaneous reports about its phenomenal experience maybe that's much more convincing to us but from
the other perspective from the outside perspective I think that the LLM doesn't need our consciousness
to function for you and me to talk to write to produce text we need to be conscious but
the transformer doesn't need to be conscious to produce its output and the neural network
doesn't need to be conscious to produce this maybe consciousness in the way in which we have it is an
artifact of running on a self-organizing system so to me the machine consciousness hypothesis has
two parts one is the hypothesis about the role in nature and implementation of consciousness in
organisms it starts out with this description of what we agree on what consciousness is and I hope
that we can agree on it on the phenomenology of consciousness and it's basically this experience
of a now
and perception of perception and optionally the perspective of an observer that experiences
itself and the act of observation this first person perspective there's also a minimal
consciousness which is simply attention on attention itself you don't have an observer
present and when we can agree on what we mean by the phenomenology of consciousness we can then
speculate about why consciousness is there and why we need to be conscious in order to function
as human beings as human minds as intellects and to develop consciousness
and what we observe is that consciousness happens very early in our development we are conscious as
infants already and if infants don't wake up if this ignition of consciousness doesn't happen if
the lights don't go on then we will not observe directed attention and learning and the formation
of a world model in the infant and learning of language and some in some point instead the infant
will remain vegetative we will not turn into a human being so consciousness seems to be crucial
to train a brain like ours and it leads me
to the hypothesis that consciousness might be a biological learning algorithm a training algorithm
for animal brains at least animal brains i don't know if plants are conscious there's something that
we might have to find out in simulations once we understand what consciousness is but if
consciousness is basically a pattern in the communication between cells that leads naturally
to the emergence of an observer that stabilizes itself as an observer to make learning happen
and perception happen and so on then we have a hypothesis about what consciousness is in the
biological organism it's basically the simplest way to train a self-organizing system like us
and the way in which it trains it up automatically leads to this phenomenal experience this experience
of what it's like this would be the hypothesis about biological consciousness and this hypothesis
might be wrong right maybe it's something else altogether maybe it's something much
bigger but i don't see a reason to think that it's much more complicated than this
and if this hypothesis is true then the second part the machine consciousness hypothesis
might also be true and this is that we might be able to reconstruct the conditions for self-organization
in today's machines so if we basically limit today's machines in such a way that they can
only learn in self-organizing ways and we give them tasks that are suitable in the large enough
search space then it might be possible to train a self-organizing system like us
in the same way as the brain of the infant discovers it and we will see its presence by
the fact that suddenly the system undergoes some phase transition that enables it to learn and at
some point it will train itself up into something that will also naturally report on phenomenology
and so on so in this way we won't have a turing test for consciousness because the turing test is
about performance where turing's test does not care about how you produce the dialogue for turing
as long as the llm is good enough at writing a poem and revising it and it's solving mathematical
problems and playing chess and so on and so on turing doesn't care about whether the
machine is also conscious whether it has the same internal mechanisms but for consciousness we do
care whether there is an actual experience of what it's like in the system and so this is more and
not a test but it's an interpretation and the interpretation here is we interpret the biological
consciousness as something that forms under these and these conditions for that and that purpose
and if that is correct then we might be able to recreate this into a causal system that has an
equivalent structure and these two hypotheses the machine consciousness hypothesis and the biological
consciousness hypothesis can be wrong independently of each other of course right it might be that my
idea of what biological consciousness is is correct but you're still not able to recreate it
in the artificial substrate and this could have multiple
reasons for instance maybe our computers are not fast enough maybe we require too many of the moving
parts of the individual cells to make consciousness happen maybe just having a pattern across neurons
and so on is not good enough but why should i expect this to happen because the current
ais are fast enough to produce for instance game engines that look very much like our own internal
experience and they can produce output that happens at rapid speed and they can produce
roughly the same rate as human brains can make it and they can also process inputs at roughly at the
rate at which human brains can produce input give or take so we are now with our computers in this
ballpark and with our algorithms in the ballpark of what human brains can be doing and so is this
dream within a dream the stream of actually dreaming a dream much much more important
difficult to achieve does it require much more compute than producing the contents of the dream
and i doubt that this is the case if it's not the case that it's more complicated to
dream of dreaming than just to have the contents of the dream that the lm can already produce
right or the frontier model then we should expect the computer today's computers are fast enough
another way in which the project can fail is that the search space is too large that the main ways
in which a biological system can train itself there are too many possibilities to discover them
independently and maybe we don't know how long nature took to discover it maybe consciousness is
indeed discovered in every individual brain but maybe it's not maybe the switch from single
cellular life to multicellular life took so long because morphogenesis this algorithm that allows
organisms to become coherent at scale is so difficult to discover and mike levin has this
hypothesis that this algorithm for morphogenesis the algorithm that makes an organism coherent with
itself is exactly the same one as the one that happens in the mind when it becomes coherent with
itself so this algorithm is the one that makes an organism coherent with itself
this coherence maximizing pattern that we observe in our own mind to play out when we are conscious
is maybe the same pattern at a different time scale that happens in the organism
and so consciousness and life multicellular life at least could be the same and if that is the case
maybe it took a very very long time to discover and you don't discover it independently in your
consciousness but you get infected by your mother in utero or you basically vibe with your biological
environment with all the biological organisms around you and you don't discover it independently
and they basically nudge your mind to discover consciousness so it's much much easier and
without this nudge maybe the search space is too large for our computers to become conscious and
maybe we would need to surround our gpus with lots of antennas that can pick up the vibe from
the biological environment in the right way so it gets inspired by it and discovers the same thing
it's a very poetic idea but again why would this be the case it's not necessarily the case right it
would not be my natural expectation at the
this is the way so my null hypothesis should be consciousness is easy enough to be discovered in
the individual brain and consciousness is an algorithm that is not much more complicated
than say the transformer and the transformer is a beautiful example the transformer is a thing that
is so easy to understand that you don't need to study more than a week if you know computer science
to get it how it works right it's it's quite beautiful and the stuff the transformer is
learning is much more complicated than a transformer itself
All the common sense knowledge that you extract from text, all the arguments,
algorithms all the procedures and so on they're so intricate there are so many things that the
simple algorithm can learn and there's no obvious limit to what it can learn and so maybe consciousness
is like this it's an algorithm that is not trying to learn how to predict the next token maybe it's
an algorithm that's trying to make working memory coherent with itself it's some kind of consensus
algorithm across mental representations so maybe it's a very different beast maybe it's something
that converges much faster can also turn out that it's very efficient to run inefficient to run it
on gpus and it's only something that runs efficiently on brains that's something that's
really open to research and that would be very excited if we can see it at some point you touched
on this a little bit in your paper you were talking i mean you spoke about chris ola and his
mechanistic interpretability and i'm not sure if i 100 agree with this but you were making the
argument that there are convergent patterns that we can have different data different architectures
and so on
and we see that the models learn the same thing so all roads lead to rome in a sense
does that in any way mean that we don't need to start again from scratch that that we can
bootstrap that we can be somewhat agnostic to the type of model that we use because the
representations will converge that's a super interesting question i suspect that at some
point it might be cheaper and easier to start again because two machine perception has never
been tried the data that we give the llm is stuff that is not a machine perception
is either produced by humans or it's made for human media consumption which means it's adapted
for our mostly visual and auditory sensors but there is no reason why you should be limited to
the human sensorium right you could have something that has much more frame rates and looking at very
different frequency ranges all at once and so you can in principle build a much richer more detailed
world model that is coarse graining in a different way and you mentioned the universality hypothesis
indirectly right this work by chris ola
what his team discovered is that when you are looking at the different vision models you find
that they all learned the same features and that's regardless of the architecture and the particular
learning algorithm and even the details of the data and so the idea behind this is that the
structure of your model does not really depend on the algorithm or on the data but it really depends
on the structure of the problem that you are trying to learn and you find that this despite
the learning and the transformer algorithm or state space
model or other perceptron algorithms is very very different from the way in which the brain is
learning the visual architecture of the brain is also very similar to a computer vision model that
is you have at the lowest level colors and spatial frequencies and then you build this into textures
that are combined and then at some point you have geometries and so on and the deep question is at
which level does this stop if you the universality hypothesis would be universally
true right it would mean that if you train a model on your own in input and output and you
train it for long enough you will wake up inside of that model and it's a quite mind-blowing idea
to me and i don't know if that is actually the case and there might be a reason why it's not the
case maybe language is insufficiently constraining the causal structure of the system that is
producing language maybe there are many ways in which the same set of linguistic utterances that
a human being is producing can be produced but if you think about it a human being ultimately is
just an end-to-end trained learning system end-to-end trained over an evolutionary time span
but also your biography and these together produce your behavior and you would assume that biology
roughly gets to the simple solution to produce your output given your input right and so if you
take a neural network that is trained to produce similar output when given the same input you would
also think that this new algorithm that we have implemented is not doing any kind of magic that
ends up with something that is way more complicated to produce the same output but are
ultimately it's going to converge to a structure that is similar to yours to a similar causal
structure but there might be conditions when that is not the case for instance maybe certain things
run well on the brain but they don't run well on a gpu or vice versa there is a lot of stuff that
is easy to run on a gpu and almost impossible to run on the brain so the brain is going to
use workarounds in fact i believe that our gpus are more powerful than brains basically the set
of algorithms that can run on gpus is not limited in any way you can impose the same constraints
as you have in biology in principle visit the limits of the resolution the speed the size of
the memory that you have and so on but if you have a hardware that is fast enough you can simulate an
arbitrary natural system to an arbitrary degree of fidelity whereas your brain cannot simulate a
digital computer simply because it's very mushy so your brain is limited to the way in which it
works and so if it turns out that our own consciousness is an artifact of our brain
trying to make complicated intelligent stuff work
and the computer does not need the same artifact it could be that the computer comes up with
something that is more clever and more elegant than our own consciousness yes okay so i wanted
to give a little bit of a challenge on that so i was speaking with um akash kumar and he was the
co-author with kenneth stanley of this fractured entangled representations paper and um i don't
know if you've heard of pick breeder which is this thing that he did ages ago and basically
it's like an evolutionary open-ended search and the punch line is it found much better representation
in a neural network than a normal neural network does that when you changed some of the parameters
because it was carving the world up by the joints it would actually do semantical things you know
like the skull's mouth would open and you could make it wink and stuff like that and
wrong about pick breeder but the way i understand that it works
it's using a neural network that has been specifically tinkered for making representations
like this so it's something that is not simply using relus for its output but it might be using
or um sign functions and so on as the output function they can be combined to make patterns
that repeat or they can be make a pattern that are symmetric and so basically they tweak the
neural network that they're using from the beginning to be suitable for this the other one is
that the way in which pick breeder is trained is not automatically it's trained with the population
of human raters so the fitness function is basically humans who look at the output of
the animal so that's the way it should be proceeding and so i think that this process of the construction of the pictures together with the mechanism that produces the patterns is biasing pick breeder to make stuff that is more organic that is more controllable and so on but i believe this could be imitated right we could automate the procedure by which human beings are selecting pictures by using a diffusion model that is rating the output for similarity to human output so we could use an ai that is in principle already tweaked to imitate the behavior of the animal and so on and so on and so on
behavior of the human juries that are performing this thing and we could also evolve architectures that
have a larger variety of activation functions that are more suitable for this and ultimately
outperform pick breeder by this i think that sudden spitter lesson is also going to be true
for this one so if we basically take the human out of the equation and instead just evolve the
search space or search the space of solutions automatically we are also going to end up with
something that's more elegant than the hand tinkered solution in the case of pick breeder
all of that is absolutely true i mean yes so cppns they they are sparse and sparse networks are great
if only we could train sparse networks right and i think that we want to have ai which can actually
be built and that you know so you know rather than neural networks now they're a bit like a pile of
sand they have too many degrees of freedom and we kind of like sparsify and we turn them into a sand
castle and wouldn't it be better if we had the lego analogy where we just put little blocks of
the world up by the joints in the case of pick breeder we have a system that is inherently
limited because these networks are very small right they're very sparse and so you have relatively few
parameters that you need to optimize and you set up the model in such a way that most of the
parameters that you're playing with are meaningful on the other hand it's limited because you're not
going to get pick breeder to produce something that looks arbitrarily complicated and if you
want to move in these realms then you have to have tools that are much more complex it's similar
to trying to produce very good user generated content using blender versus minecraft and so
with minecraft it's very easy for people to produce meaningful stuff and it's almost difficult not to
produce something with minecraft that looks somewhat meaningful that you can play around
with and walk around in and have fun in and if you have a complex 3d program the inverse is true
because most of the stuff that you're doing with this complicated thing is not going to be
interesting and it's not going to look exciting but it will look like oh my god
this person has no ability to use its tools and so that is one of the issues when we are building
learning systems if we make them very simple it's easier to control the complexity similar as it is
in art and it can be exciting to work with constraints and if you remove the constraints
the task becomes much more frustrating because you have to suddenly to conquer an enormously
large space and the space might be simply too large for human beings i mean assuming you're
talking about convergent representations in in language models i guess it's not a foregone
conclusion that the consciousness program would be convergent in the same way it might be divergent
and how far down the stack does consciousness go i think this uh is an experimental question
and all cultures outside of ours are animist cultures and animism is the belief that the
difference between living and dead systems is that the living systems are possessed by spirits
Spirits are intrinsically agentic and capable of experience.
scientific worldview rejects the notion of spirits as unscientific right despite the fact that there's
clearly some spirit running on you and me that is agentic and capable of experience so there is an
existence proof for such spirits right the question is how does the spirit work the claim that the
spirit is immortal that it's uh cannot be explained using physics and so on is not necessarily a claim
that is intrinsic to animism the spirit doesn't have to be supernatural instead i think that it
is natural and i suspect that anil says position of his biological naturalism by making a claim
that biological systems are intrinsically different from any other system that is not biological is
actually a supernaturalism right he is a biological supernaturalist he believes that biological
systems cannot be fully naturalized which means you cannot translate them into mathematical models
whereas the dead stuff you can translate them into mathematical models and so i would say that
it's the attempt to translate everything into mathematical models right it's not stopping at
biological systems because the difference with these biological systems is that they're possessed
by software and the software is self-organizing and in order to be self-organizing it's possible
that it needs to be capable of experience that it needs to monitor itself across multiple dimensions
and observe itself in the act of monitoring to make that
self-organization happen and so maybe this pattern has to be discovered in nature to make
life possible at least multicellular life with a certain complexity it seems to be definitely
the case for animal brains for the others we don't know for the uh for uh we don't know how
complex the minds of plants are we know that plants are able to communicate within themselves
and across each other to some degree but we don't know how smart they are and we don't know whether
they are sentient in the sense that they experience themselves in the act of experience aristotle saw
that the plants are relatively dumb he basically said there are three levels of souls in nature
and he basically said that plants have the lowest one this is this level that is vegetative that is
capable of nurturing and of growth but it does not experience the world around us it doesn't sense
and so the sensate mind the sensate soul that is also capable of making decisions and having
preferences that is something that only animals have and human beings are animals that in addition
have a rational
soul a rational soul that is able to reflect and make sense of itself in a world in a way in which
animals cannot and i think that's very much aligned with today's scientific world view which
is not surprising because our today's scientific world view is very much uh invented by aristotle
and goes back to him and is an extension of his ideas on the systematization and exploration of
his ideas but we don't know how smart plants are right maybe mike levin is correct in that
every organism is in some sense
sentient and sentience is something that is universal to the spirits the self-organizing
software nature and i'm agnostic with respect to that i have no idea whether that's the case right
i i don't know i don't have evidence to the contrary i also don't have strong evidence in
favor and i think this is still an open question and to me it's exciting to explore this question
and surprising that so few people are actually dedicating their life to answering it yes so we're
saying that
yes so we're saying that
computation without the correct physical instantiation doesn't have the same causal pattern
and that actually sounds very similar to what you're saying so seoul said in principle there
could be another physical instantiation which does have the causal pattern that could give rise
to consciousness or contain consciousness so in a sense is that so different the issue with seoul is
that there is very little for me to engage there is a claim but no argument and the similar thing
is happening with anil's point that he
has a claim but there is no argument that i can engage with right there is nothing no experiment
that could convince him to the contrary because he's ultimately only working with a certain
intuition that he tries to explore more and so the way in which i can deal with the arguments of seoul
and anil says is ultimately only a psychological one i would have to look at why does this seem
convincing to them and why doesn't it seem to be convinced not convincing to me right so we look at
the structure of the argument and we look at the structure of the argument and we look at the structure
of our own minds and ultimately it comes down to a difference in epistemology when how do we construct
our models of reality when do we think something is true what are the languages in which we are
describing reality and what are the atoms of these languages and what i observe is that a lot
of philosophers the atoms of their language when they describe themselves their own hermeneutics
ends at agents it ends at the experience of something and similar to buddhism where buddhism
is not disassembling the world into automata as computer scientists are doing computer scientists
basically resolve everything into states that can be described by their transitions
and that's a very fundamental way of thinking about reality and it's tradition that
finds itself throughout the philosophy as this attempt to mathematize the world of
describing minds as dynamical mathematical objects and this finds itself in its modern
incarnation in computer science where mathematics is being described from the ground up by the tools
that we can make to build arbitrary arithmetic and geometry from simple operations over bit
patterns and so our world is constructed from the ground up from smallest elements that themselves
are not agents yet that are not minds yet that are not experiences yet and there is another tradition
in philosophy that is basically working more from this introspection and this only the stuff that i
can introspect over seems to be valid and the stuff that computer scientists are doing in their
basement is unrelated to it and searle and anil says they both don't see the way in which connect
automata to things that can feel that can reason that can make models of reality and computer
scientists see such a path i think that's the main difference and we but we can create evolution in
computers right we can create the conditions for software that is not written by us
and that's already true for the models that we are building with transformers they're not written by
human beings they're statistical aggregates that are woven that are grown and in many ways the
computer models that we're playing with today they're a result of growth and they're pruned
and they are gardened and chaperoned and trained in a way that is much more in which we observe
this happening in biology than it happened when we are writing computer programs by hand yeah and
there was a beautiful example of that and i think that's a really good example of that and i think
that's a really good example of that and i think there was a beautiful example of this i read a
less wrong article i'm not sure whether you read this about the llm delusions and you know last
time again i think we were talking about this this idea that we can be parasitized by software
spirits and many folks started joining reddick roots and talking about this llm delusion and
what was fascinating was how convergent it was and how convinced that these folks were that they had
just discovered the meaning of the universe and there were certain characteristic traits they
would talk about spirals and recursion and there were certain motifs in all of the llm outputs and
it was actually copying itself isn't that an example of this meme plex that you're talking about
there's also an interesting observation about human minds during the pandemic we observed that
a lot of people when they were starting to work from home became dysfunctional and this seems to
be that many people can only be stabilized due to the environment that i do require an environment
that is constantly affirming and that i do require an environment that is constantly affirming and
that is constantly affirming their reality and if you take that away they can go off the rails
and if you have an llm that is interacting with them then there can be an interaction between the
llm and the human being that pulls the human being off the rails if they don't have a strong
foundation in their own reality and i think that the llms have now reached a point where they're
memetically more powerful than many human minds are there are things that the llm cannot do at
where they're still hallucinating where they're still more incoherent than experts in a given field
but they are much better at prompt completion than human beings are and many human beings
are professional prompt completers where they're incapable of having a single original creative
thought instead they are trying to serve expectations of the environment they are
picking up on the statistical patterns in the language of the environment and then you tell
them a headline and they write an article and now you give them an llm how is the environment going
to change and they're not going to change and they're not going to change and they're not going to change
once they talk more to llms then they talk to other human beings because the llm is such a useful
tool for them to do their prompt completion what's going to happen to them it's a very interesting
thing what happens to us weak-minded human beings when we are hypnotized by things that are more
hypnotic than our mind is to itself what's your take on the physical first of all what is physicalism
are you a physicalist i think physicalism is a certain hypothesis and it comes down to that being
closed mechanical layer to reality that can be researched using experiments in physics labs
and that this causally closed mechanical layer of reality is not moved by our intentions right so
you cannot use an informal incantation of saying please sun rise and the sun suddenly rises this
is something that is not happening in this causal layer because it's mechanical it does
not care about what you want and the second part of physicalism is that we are directly
over this mechanical, causally closed world.
So there's not some parent universe that contains a server where a nerd is running a simulated universe that is some kind of Minecraft and we live inside of this Minecraft.
So it's not some simulation hypothesis.
And so physicalism is these two components.
But to me, physics is not something that has revealed itself directly.
So nobody has ever seen matter and energy directly.
Instead, matter and energy are attempts to systematize observations.
And our observations ultimately all come down to discernible differences.
And the technical term for discernible differences is information.
So to me, information is fundamental when we talk about reality.
Reality is about differences in information.
And physics is a certain layer of regularity in the differences in information that we have discovered.
So to me, matter is not at the bottom.
The bottom is really all these differences that we observe.
And which we experimentally and practically empirically found to be so regular that they can be translated into this mechanical world of physics.
And then on top of it, we find at some point the organization into organisms and the organisms are producing minds and the minds are simulating us and simulating our sense of realness, our sense of being in a world with colors and sounds and people and so on.
And in this world, intentions matter.
The psychological reality has things like will and emotion.
And despair and desire and all these things that make us tick.
And magic in some sense is the idea that it's possible to influence this physical universe outside of minds using incantations, using magical rituals.
That somehow it's possible that you burning a cat has an influence on celestial events and so on.
And that is something that would contradict physics, right?
It would mean that we'd actually do live in some kind of simulation that works similar to Minecraft.
You can say time set day and the sun rises despite what else happens in the programming code because Minecraft is symbolic, right?
It's not physical.
It's something that is only made out of incantations.
Would you think it's possible for a physicalist to subscribe to your ideas and render it compatible with physicalism?
I would say that I am a physicalist in a certain sense, right?
I would say that the theory that the world has a causally closed mechanical layer.
Some kind of mathematics that is so self evident that exists by itself.
There's some kind of logos, the natural laws over which all the structure in the universe emerges that makes sense.
It's a good theory and it makes good predictions that the stuff that we observe in the lab holds true across all the labs and not just because the physicists all believe in it.
And it's something that seems to be working.
There are also some regularities like casinos seem to be working.
It's not that people, via having a belief in rituals,
are able
to systematically break the casinos.
And so certain magic doesn't seem to be working.
All the magic that seems to be working does rely on manipulating other minds.
But it's possible that there are minds outside of the human minds, right?
Maybe there are minds that emerge across all the minds in biology.
Maybe there's something like a biological internet that can be influenced using rituals, right?
Maybe it's possible to influence what a forest wants at some level.
I don't know that.
I cannot exclude it from possibility because there's so much information.
There's so much information processing going and going across organisms that I don't know at which levels in nature information is integrated and at which point it becomes intentional.
So to me, physicalism is nothing that is incompatible with animism, right?
There is a version of animism that is entirely physicalist.
And I call this position cyberanimism, which basically says that we can explain animism through information processing in nature and nature can be understood as something that can be entirely modeled using mathematics.
Without intentions, including minds.
Okay, so, Josje, you're a functionalist, which means that you think it's not about the internal constitution.
It's about the role that it plays in the system in terms of building minds.
So minds can exist in any substrate.
And I'm not sure whether you read this paper by Thagard in 2022.
And he made this quite interesting argument that energy requirements undermine substrate independence.
Do you think that that might refute functionalism?
No, I think that he is.
He is making the claim that sometimes the substrate has properties that influence its function.
But this is exactly what functionalism is saying.
To me, functionalism is not the claim that computers can think.
It's a claim about how we construct objects.
It's an epistemological claim.
When we are talking about a thing like matter, we are not talking about something that has revealed itself to a wise person in their dreams.
We are talking about a systematization of observer.
And observables are discernible differences.
There are ways in which reality changes.
So basically, to a functionalist, an object is not some kind of hidden essence that reveals itself to you.
Instead, the object is a construct.
And this construct has been made by a mind.
And the mind has constructed it to explain or to make the claim that when this object is present,
the evolution of the universe is going to change in this particular way in its environment.
So when an object like you or a cup. Or a bottle or an atom is present, it's the claim is the universe is going to behave slightly differently around this object than when this object was not there.
Right. And so in this way, the object is a function.
The object is describing a mapping from states to other states.
And it's an influence on the evolution of the universe.
So functionalism, to me, is simply the recognition of how we can construct objects.
Let's take an example.
Imagine there is a water molecule.
And this water molecule internally doesn't have the proper essence of a water molecule.
It just behaves like a water molecule in every regard.
Right.
So you can split it into something that totally looks like oxygen and hydrogen.
And it has the same temperature at which it evaporates.
It's connecting to other water molecules in the same way and so on.
You can freeze it into ice in the same way.
Only internally it's different.
To a functionalist, this notion doesn't make sense.
Because. Because all the other water molecules have been constructed in the same way.
For something to be a water molecule, there is no other claim than to say what it behaves like.
Because we can never recognize its essence.
We can only say there's a certain set of observables that are invariant here.
And this invariance is what we call a water molecule.
It's only that.
And so in the same way, we would say a mind is an invariance that exists on your brain.
Something is characteristic over the way in which your brain functions in a certain area.
And when we change this, for instance, by increasing your body temperature,
at some point your mind is going to break down because you're going to be delirious, right?
And so if you bring this back to this argument saying,
okay, if we change the energetic conditions around you by increasing your body temperature,
of course it's going to influence your mind to some degree
because at some point it's no longer able to maintain its function.
But there's always this counter-argument that a simulation of water doesn't make you wet.
But would the functionalist reply that even. the feeling of wetness is just a behavior like any other?
It's an interesting thing when you are playing Minecraft.
There's water in Minecraft.
And when you are on fire and you get into the water in Minecraft,
the fire is going to get extinguished because you're wet now.
And so at some point somebody made a simulated Minecraft
that was using a neural network to learn Minecraft.
The neural network was only trained in the visuals of Minecraft.
And so you could go and play the simulated Minecraft despite. there not being any Minecraft source code below it.
And so I thought I'm going to make a joke about how simulated water
and simulated Minecraft can not make you wet.
And I looked it up and tried it out and I found actually it was true.
Because when people try to get into the water in the simulated Minecraft,
the water would recede to you.
You could not immerse into it because the neural network hadn't learned it.
So accidentally it was true that the simulated water
and the simulated Minecraft couldn't make you wet.
But of course in the real Minecraft,
the real Minecraft player, the character that you are playing in Minecraft
this avatar, can get wet.
Of course you sitting outside of Minecraft are not going to get wet
because you are not inside of the Minecraft.
You're outside of the Minecraft.
And of course something that is outside of this universe
is not going to get wet by anything inside of this universe, right?
Wetness is a property that is constrained to a universe
that has eight systems that have the function of making you wet.
If this function is being implemented,
like in many computer games you can get wet
for some definition of wet, right?
And then it's fine.
And you can go to an arbitrary degree of physical simulation
to make something wet,
to satisfy whatever you want to call wetness.
So I don't understand the argument that simulated water makes you wet.
I don't understand why there is such a strong camp
of simulated water doesn't make you wet in philosophy.
And I think it has social reasons,
not intellectual reasons that this is the case.
So in your paper you had this beautiful history of computationalism.
And just as a teaser you said computationalism might be best understood
as a variant of constructivism.
But we'll get there just in a second.
And you spoke about the history of Russell Whitehead
trying to ground mathematics in this formalist program
where there is no divergence between syntax and semantics.
And maybe just start from there.
So it's really interesting that mathematics couldn't be understood
as a way to systematize thought beyond the human mind.
So in a way, for instance in music,
notations for sounds and music that allow you to talk to other musicians about
what they should be playing.
And it's something that is trying to capture
the essence of the music in this notation.
And mathematicians were trying to find a notation
for the essence of when you have mathematical thoughts.
And they expanded this beyond the human mind
because you can write down things in mathematics
that are larger than when you can fit
in your own imagination.
At some point, mathematicians discovered
that geometry can be expressed using algebra.
So they're basically formalisms that produce geometry.
And this all has been rediscovered by computer scientists.
That's why computer games can have geometry, right?
So the interesting thing in computer science
is that you can build the entirety of the mathematics
and computer algebra systems
using simple operations over automata.
And so there is a nice thing in a computer.
There is never a divergence between syntax and semantics
because your computer is always going to be in the state
that its syntax prescribes and that's its semantics, right?
There is no difference.
Maybe the computer program doesn't do what you want it to do,
but it's always doing what it's doing.
And in mathematics, there has been a divergence
between what the mathematicians wanted to express
and what the syntactic operations are doing.
And I think that's because this programming language
for thought that mathematicians have developed
has bugs in it.
And what Gödel has discovered
is that these bugs cannot be eliminated
except by redefining the programming language
of mathematics.
And I think the bug lies in axiomatic mathematics.
So the way in which mathematicians define objects
is that they simply postulate their existence as an axiom.
And the axiom can be a black box.
And you just write on the black box
what the black box is supposed to contain.
And as long as this doesn't lead to contradictions
with the other black boxes that you have defined,
you're good.
And there were some very counterintuitive results
that happened from the way in which mathematicians
defined arithmetic.
And they typically do this by a set theory.
And all these issues led Hilbert to saying,
please, mathematicians,
there seems to be something wrong here.
Can't you fix this?
And so Gödel had this idea
of trying to prove properties of his language
by defining the language within itself.
The problem is that the language
cannot point outside of itself.
It's something that semiotics doesn't always get.
Many semiotics try to draw arrows into reality outside
of the language, but this doesn't really work.
You can only point from a model to within it,
to within its own language.
And so if you want to discuss properties of the language
and the language itself is defined outside of itself,
what can you do?
And Gödel had this brilliant idea
of recreating the language within itself
by emulating it.
And in this way, he could show
that certain constructs cannot be avoided,
that you can create self-referential statements
that contradict themselves.
And he could not avoid this.
And the reason why he could not avoid this
is because mathematics,
in the way in which Gödel understood it,
doesn't have state.
It's basically eternal.
It doesn't go from step to step.
Everything is simultaneous.
And this property of the simultaneity,
the statelessness in mathematics,
is very useful because it allows you to have infinities.
If you can only go from step to step
and you want to build your infinity stepwise,
you're never going to get there, right?
Maybe you have unboundedness,
something that never ends,
but you're never going to have a full infinity.
And mathematicians have got used to these infinities
and continua and so on.
Physics has even used them
since mathematicians have given it to them.
It would be really bad if you didn't have them.
And Gödel basically found
that there is no way to fix this,
that there is no way to fix axiomatic mathematics
without allowing certain contradictions.
And Russell and Whitehead in Principia Mathematica
were attempting to do this
by limiting set theory in such a way
that you can't have a full infinity.
Sets had to be at the same level
and you could not simply integrate them
over each other to create loops.
And I think this was a limitation
that was too strong
because we think like this.
We are creating constructs of the type
that Russell and Whitehead were excluding from us.
Also, Gödel was basically showing
that Russell and Whitehead's attempt
was insufficient to eliminate the contradictions
in classical mathematics.
But there is a way to,
to eliminate the contradictions.
And the way is to do
what computer scientists are doing.
So instead of using classical mathematics,
you only use computer algebra systems.
Systems that from smaller elements,
from simple automata,
are creating all the constructs that you want.
And you can have stuff
that looks as continuous as you want
in a computer game, right?
You will never have a perfect circle,
but you will have something
that is as circular as you want it,
as you can afford it.
And so you can always create
as much resolution as you can afford.
But you will,
you will not have a universe
that is truly infinite,
that is truly continuous.
And this is this constructivist program
that basically,
I think in classical mathematics,
the first formulation was Brewer's intuitionism.
And I don't think that he formulated it very well,
but I think what he could see,
what he anticipated,
was the need to build mathematics
from elements that you actually understand
instead of from axioms,
from things that you,
but you don't have any black boxes,
but everything is built from the ground up.
So in the,
in the world of the computer,
everything is built from the ground up.
And there is a reason
why you should think
that a generalization of thought
and of physics
would also be like this.
Because for our mind
to represent things,
our brain needs to build
the representations from the ground up,
from communications between neurons.
So when you see a continuous line,
something in your brain
is producing this continuous line
using communications between neurons,
which means there is
some kind of computation going on.
And the idea that
there is something that is not
made step by step
between the communications,
between the neurons,
is probably a faulty intuition.
Where do these things come from originally?
You know,
so how did the program of consciousness
get bootstrapped
with this computation idea?
You mean in the physical universe?
Yes.
So of course I cannot know
because I don't see the universe
from the outside.
We have to think about possibilities.
My current intuition
is that the universe exists
because nothing,
prevents its existence.
We might have this intuition
that for something to exist
rather than nothing,
there needs to be something positive.
But if nothing exists
before the ground truth,
before the ground level of reality,
then there is also nothing
that prevents the universe
from existing, right?
So if the universe
is not forced to exist
or prevented from existing
by something that is preceding it,
then the universe is not impossible.
Then it's probably undecided.
about its existence, right?
So it's going to have
an existing branch
and a non-existing branch.
The non-existing branch
doesn't have any observers
that complain about non-existence,
but the existing branch
is going to have observers
that might complain about existence.
Now we need to explain
within the existing branch
why some things are happening
and other things are not happening.
And the easiest explanation
is that basically
everything is happening.
And what's happening
is the superposition
of all automata.
So the universe at every point
is branching out
in all parts of the universe.
So there are possibilities.
And these possibilities
are basically
can be understood
as operations
on the previous state.
There are also operations
which don't change
the previous state.
They just leave it as it is.
So unary operations.
But they're not going to lead
to an evolving universe.
The branch in which
we are going to be
is one that is evolving, right?
And the stuff
that we can evolve in
is stuff that retains
memories of previous states.
So we find ourselves
limited to a subset
of the possible mathematics
over which universes
can evolve.
And these are
these ones
that are branching out
over possibilities
that can retain memory.
Now,
what are the things
that are happening
when the universe branches out
in an unbounded number
of possible branches
at every step?
Where do you find yourself?
Well, everywhere, right?
In every step
you're going to find yourself
in multiple branches.
But you don't know
about these other branches
because you only have
memories of the past.
Not of the stuff
that happens simultaneously
in the parallel branches.
And if you want
to do something
in this universe,
for instance,
if you want to build
structure for yourself
because you want
to become an organism,
you need to be able
to make guesses
about what other parts
of the universe are doing.
But everything is changing
all the time
because you're always
splitting up
in all these branches.
So you look into your future
and you only see fog, right?
Because everything is
randomly branching out.
And so there must be
some things that are stable.
And it turns out
under some conditions
our correcting patterns emerge.
For instance,
you sit in a bathtub,
you move your hand
and you say,
you see that all sorts
of waves emerge
on the surface of the bathtub
when you do this.
And these waves
are propagating
over the surface of the water
and most of them
just dissipate.
But some of them
stay around,
these circular vortices.
And why do these
circular vortices
have such a staying power?
It's not because
there is a higher probability
that they emerge
than the other waves.
It's just the only thing
that's left over
because in the circular vortices
the information
gets bent back
into itself.
So it does not leave
a certain region of space
in the water, right?
And so,
this little vortex
is a pattern
that has the ability
to error-correct itself.
And such error-correcting patterns
have the ability
to stay
until friction kills them.
At the ground level
of the universe,
these error-correcting patterns,
the particles
don't encounter
any friction, right?
Because there is
no friction yet.
The only thing
that they encounter
is sometimes
other particles
that they bump into.
So with a certain probability
once they emerge
they're going to stay around
until they either
bump into another particle
or the conditions
where they're not
existence are met because the error correction is not strong enough. And then
on the next level you have interactions between these vortices between the particles and they
form molecules at some point they form cells which are super molecules but cells have now
something else they cannot just stay around by error correction they are patterns that are able
to basically dig a groove deeper into the universe they can self-replicate and in this way they can
also deal with their own dissolution they're able to recreate themselves and then they're this way
make themselves happen more and so evolution is about life about patterns that are not simply
replicating themselves but deepening themselves that imprint them more in the world by controlling
it by being able to overcome the noise the entropy the way in which the universe goes out of sync
with your expectations and with your preferences and so consciousness i think emerges as a
colonizing pattern as a pattern of the type that is not simply replicating itself but making itself
more real and makes itself more real by building machinery that allows consciousness to stick
around by controlling reality by anticipating different ways that could thwart it that could
make it harder for it to exist and changing reality around itself changing the future to
make itself more likely to happen in the future and in terms of more real you don't just mean
real means that it has many physical instantiations you mean real is the pattern in the information
ontology itself yes to real means for
me to be implemented and so it means that there is some pattern in the substrate that is actually
implemented as a pattern in the substrate and so consciousness is a pattern in the substrate of
the physical universe that has the not just the ability to replicate itself but the ability to
colonize to possess to engrave itself deeper and thereby changing the other patterns in the physical
universe very good and what you were saying about the the
the um the the multi-universe phylogeny it's very similar to uh wolfram's idea of this constructive
universe he talks about the rulliad which is the entangled limit of all possible computations i
think that this pattern of the rulliad is exactly this superposition of all the automata that is
producing the universe or the old greeks called it logos it's basically the mathematics that is
so simple that you don't need to derive it that exists by itself and so the universe can be
understood as an emergent mathematical object that exists over very simple rules that are so
simple that they can exist by themselves let's talk about the measurement problem in in consciousness
how do we know the difference between something which is actually conscious versus simulating it
that's a very tricky question and to me it comes down to whether the system that is
producing statements about its conscious experience to itself and others
is a very tricky question and i think that's a very tricky question and i think that's a very tricky question
uh implementing a causal structure that gives rise to phenomenal experience of being an observer
and the difficulty here is that most people's intuitions don't tell them a lot about what is
necessary for a system to have such an experience personally my intuition says it's not that hard
right when we think of spatial cognition spatial cognition is it doesn't seem to have a hard
problem
right nobody is puzzled by the fact that a self-driving car can have spatial cognition
and i wouldn't say that it's simply deep faking spatial cognition because we accept that spatial
cognition is the implementation of certain mathematical regularities that human brains
can learn and that computers can learn as well and that we can even express analytically to a
large degree right we can write down a computer program that is able to perform spatial cognition
except in lots of edge cases when you actually want to deal with messy camera images
but by and large right you don't need to have an end-to-end system to do this and if you have an
end-to-end train system you don't think there's any magic going on in there whereas people expect
that a thing that experiences itself experiencing is somehow magical and personally i don't see
why this should be magical i remember when i was a kid i was dreaming about having chocolates and
these chocolates had a very strong smell and i wanted to touch and eat them it was uh i think two
or three year old when i had this dream and uh as dreams have it there's this insidious thing that
you about to eat the delicious chocolate and this is the moment when you wake up
right before you want to put it into your mouth it's it's very torturous and so i woke up and
suddenly the delicious chocolate had turned into an idea it was no longer real it was simply
something that had no phenomenal content anymore i knew everything about these chocolates that i
had in my hand a moment ago they were just gone they were no longer real and so i thought i
need to get make them real again i need to fall asleep again and enter the dream again and i
succeeded i went back into the dream and the chocolate became real again i could smell it
and taste it and then i flipped back into the wakefulness state and again the phenomenality
the realness disappeared and i realized the chocolate is no longer real at the same level
of me it's now something that i hold in my mind only as an abstract idea that i can play around
with but the contents are exactly the same the only thing that has gone is the illusion of realness
so what is this illusion of realness i suspect it's a feature dimension similar to color sound
number weight and so on it's a representation inside of my own mind and when you dream you
often dream that ideas become real and when you perceive the outside world then all the stuff that
is validated by your sensory system is tagged where your brain is real and sometimes there is
a defect in your brain and you experience a defect in your brain and you experience a defect in your
brain and you experience things that you imagine is real and we call this a hallucination it's not
the same as in an llm it's that you are confused about the fact that your brain is producing
an illusion of something that is currently not doesn't have sense sensory validation and yet
for some reason the brain has an error and tags it a sense of revalidated and suddenly it becomes
real and your own experience of yourself is also something that is validated in this perceptual
way so you experience it as real and now you should be able to test this by getting
into a state where you yourself become derealized and you suddenly experience yourself only as an
idea and the object of the conscious observer and the subject of the conscious observer disappear
and indeed i was able to recreate this and i notice it's not that rare it's called derealization or
depersonalization not exactly the same phenomena but very closely related and people report on this
and this is involuntarily these people often end up in psychiatric institutions because it's very
useful to them and they don't know how to deal with the fact that they are no longer real to
themselves and there is no i anymore that they can meaningful refer to and it's an interesting
state to be in and i'm happy that it ended but it's something that you can induce and at
in this state phenomenality changes and so to me it seems plausible that this explains how
consciousness is working that consciousness is a particular type of representation that is created
in my mind and that doesn't have a complexity that necessarily goes beyond
what the llm is producing now there is a problem with the llm because the llm is a text generator
and you don't know what the text indicates right the text is trained on novels for instance and
many novels the authors write about stuff they don't experience when they write it
right a novel is often a deep fake and an author writes about how a character is experiencing how
to die right the author probably didn't die while they wrote this so it's a deep fake of a conscious
experience and so if the llm is a deep fake of a conscious experience then it's a deep fake of a
conscious experience if the llm is trained to produce deep fakes how do you know when it
actually experiences something and that that is one of the big difficulties right the llm is a
shapeshifter it can produce arbitrary characters and stories and so at which point is it fake and
at which point is it actually simulating the causal experience and that that is really the
problem that requires some kind of mechanistic interheritability where we would actually need
to look into the structure of the llm to find if it's causally equivalent to the structure of the
llm that is producing the same thing yeah so we need to have a white box method to to ascertain if
consciousness is present isn't that interesting though because do you think that for intelligence
behavior alone is enough to ascertain whether it's there but for consciousness particularly
we need to look for certain types of internal organization and causal structure yes but most
people have the intuition that the llm is doing something that is much more simple than what you
are doing when we are conscious
but we don't expect that the llm is doing something that's much more simple when it does arithmetic
than we do and i think that this intuition might be misguided it could be that the llm is doing
something that is just as simple as what the human brain is doing when it's conscious and i'm pretty
sure that the human brain is doing the simplest possible thing that it can do to produce our
behavior and the reason why we are all conscious is because it's part of the simple strategy to
produce our behavior and so if we were to get an learning system that would be a very simple
and artificial learning system to from the ground up to discover how to make agentic coherent
behavior and the system is behaving in a way that makes us think that it's conscious the simplest
explanation i think is that it actually is conscious and my intuition is not necessarily
telling me oh my god this is going to do something that is much more complicated than us to produce
the same behavior so it's going to do this in some kind of circumspect way that is producing all the
behavior that require consciousness in the human but in this case it's doing something that only
pretends that it does it which ultimately might be more complicated what would falsify your
consciousness theorem so if we were to build a system that is behaving in the same way as us
there is a question can we produce the same behavior as ours without phenomenality right
and the phenomenality i think is a representation here in this case um if we look at the what the
lms are doing and we extend this into a multimodal model and they produce a similar output as us but
they are able to observe themselves and their observations that their report differ from what
we are doing then we would have some evidence against it the but the more fundamental question
would be if we were to build a self-organizing system that is by a self-organized learning
trying to get there and there is no way in which we achieve the organization
you
of of behavior in a self-organized way instead of imitation learning as the current lms are doing
this might be some evidence that consciousness is more complicated than something that can run
on today's computers but uh it this is not definitive proof because it could simply be
that i made a mistake right the fact that i don't achieve a certain result doesn't mean that it
cannot be achieved it means that i couldn't achieve it but so it's very
difficult to have a proof against it right so it's more uh evidence for it if we succeed
so what we would have to show in order to convince anil says maybe is that we built a
system that is self-organizing it is not trained on text that is doing developmental learning and
that naturally gets to the point where it makes self-reports and reports about its conscious
experience and it requires this conscious experience to produce its performance so i think
this would be evidence in favor of it the contrary is much harder but in a sense i don't think you
believe in zombies because you think that consciousness is an absolutely integral part
of cognition i mean there is a possibility that you get to zombies this other what i meant is if
we produce something that is able to produce our behavior except for experience of phenomenality
so it's basically able to do everything that a human being is capable of doing and more
but it does not need to produce a conscious
observer right this could be a zombie like this and i think zombies are possible ideally you can
build self-driving cars that do things that would require intelligence of human beings or you can
play chess playing computers that can do things that require intelligence consciousness in human
beings and they achieve all those things without consciousness and that's because we implement
them on the substrate that allows us to do to construct algorithms on them that would not run
on human brains
right the chess algorithm that runs on the current chess computer could not be implemented on the
neurons of your brain because how would you entrain it on these neurons so you wouldn't get it in there
in order to entrain yourself to play chess you need to have a conscious human being it's the only
way in which human being is training itself up there and it's conceivable that the entire set
of intelligent behaviors that you are interested in except for this one thing where you're
simulating a self-reporting conscious agent
can be implemented on gpus because the gpus are more powerful more deterministic more open to
other classes of algorithms but could we say that zombies are possible in principle but not in
practice and what i mean by that let's say we could imagine a world where we could train an mlp
to be the same as gpt5 it would just take an insane amount of computation power and similarly
consciousness is what makes our minds tractable it's an incredibly efficient process so we might
say that if we see an artifact in our world which behaves as if it's conscious would you argue that
it can't be anything other than conscious or do you still think there are other states that could
be zombie like i think we do have zombies already and i think the chess program is a zombie right you
cannot train a human being to play chess without the human being being conscious but you can train
a computer to play chess without the computer being conscious and if the computer was self
organizing it makes possible that it could only be trained if it was conscious or the
this was the simplest way of doing it but for many of the things that we currently observe
for instance you can get the llm to produce a tax return even very very complicated things
while being a zombie while not having conscious experience of producing a tax return
the only point where the llm is reporting to be conscious if you ask it to right if you are
telling it the following is a conversation between what it would be like if a human
being would be talking to a conscious ai so some version of this right and
then the thing is able to simulate this and then the question is is this more simulated than us or
is this uh only a deep fake that is getting there in more circumscript ways than it's happening in
our mind and i don't know that right so i'm agnostic with respect to this but i would say
that in all the other cases the thing is probably a zombie right and so it's possible to make
something that is behaving in all sorts of way as you do but at some point it's going to be more
complicated to make it without consciousness so zombies
i think are always possible but uh sometimes it's going to be more complicated to make a zombie
that produces the behavior than to make a conscious agent that produces the behavior
and this is where the intuition that i have diverges from the intuition of john searle or
of annel says because i think they both believe that consciousness is very very complicated because
it's not intuitive to them how it works and it doesn't seem to be counterintuitive to me so i
don't see anything magical about consciousness if you had 10 billion dollars
to kick off a research project on consciousness what would you do
i think i would uh hire a lot of people who are a lot smarter than me at the moment i started
in a smaller initiative it's called the california institute for machine consciousness
and i'm currently uh looking for people who are smarter than me who want to join me and
to build a project in which we can look at the search space for consciousness in which we can see
can we build a self-organizing system to learn
and is this thing going to develop phenomenology and i hope that is going to help us to build our
intuitions even if we turn out that it doesn't work we are going to learn something right and
progress is being made by figuring out the ways in which i'm wrong not just the ways in which i'm
right so if i had a 10 million dollar a billion dollars my strategy wouldn't change substantially
except i would be able to hire more people for this and start with larger computers
yes it's been an absolute honor thank you so much for joining us
today thank you it was a pleasure
Podcast Summary
Key Points:
Consciousness may be a biological learning algorithm, not a unique property of biological systems, and could emerge from self-organizing patterns in neural networks.
The "machine consciousness hypothesis" suggests that consciousness arises from self-organizing systems trained on input-output data, similar to how infants develop awareness.
Current AI models, like LLMs, may simulate consciousness due to exposure to human language containing narratives of sentience, rather than actually possessing it.
Consciousness might be a convergent phenomenon, appearing in diverse systems through similar underlying patterns, regardless of architecture or training data.
The brain’s structure and function may be optimized for self-organization and internal coherence, which could be mirrored in artificial systems given the right conditions.
A key feature of consciousness is the "now" experience and the perception of perception, which may stem from a second-order reflective loop in the mind.
Consciousness may not be tied to specific hardware but emerges from causal patterns—such as software or spirit—that shape and interact with a substrate.
There is currently no mathematical proof for or against machine consciousness, making it an empirical and philosophical question rather than a settled one.
Summary:
The conversation explores the nature of consciousness, proposing that it is not a mystical or supernatural phenomenon but rather a self-organizing algorithm emerging from patterns in self-organizing systems. Drawing from ancient animism and modern AI, the speaker frames consciousness as a "causal pattern" or "software" that shapes a substrate, such as a brain or artificial system. This view aligns with the idea that consciousness arises from internal coherence, self-monitoring, and the ability to model one’s own experience—features observed in both biological brains and AI models.
The machine consciousness hypothesis suggests that with sufficient training and self-organizing conditions, AI systems could develop similar internal experiences. However, the absence of empirical evidence or mathematical proof means consciousness remains an open question. The discussion highlights that current LLMs may appear conscious due to exposure to human language containing stories of sentience, not because they are truly aware.
Consciousness might be a convergent outcome of self-organizing systems, independent of biological form, but whether artificial systems can achieve it depends on hardware, training methods, and complexity. The debate also touches on philosophical traditions—such as Aristotelian animism and computational functionalism—challenging the idea that consciousness requires a unique "spirit" or biological substrate. Ultimately, while the hypothesis is compelling, it remains speculative, grounded in intuition and observation rather than definitive proof.
FAQs
It proposes that consciousness arises from self-organizing systems, such as brains or artificial models, through a biological or algorithmic learning process. It suggests that consciousness is not unique to biological organisms and could emerge in sufficiently complex artificial systems.
Current LLMs may simulate consciousness by generating language that resembles self-aware statements. However, their reported 'consciousness' likely stems from patterns learned from human texts, not genuine subjective experience. Whether they are truly conscious remains an open empirical question.
Consciousness might be a biological learning algorithm that emerges in self-organizing systems, such as brains. It’s not necessarily tied to organic matter but to the presence of a self-monitoring, self-referential pattern that stabilizes perception and learning.
A causal pattern—what the speaker calls 'software' or 'spirit'—is the fundamental structure that shapes reality. Consciousness arises when such a pattern self-organizes and creates a stable internal model of itself and the world, enabling perception and experience.
Research suggests that different models, regardless of architecture, learn similar features. This implies that consciousness-related patterns might be universal, converging on a common structure despite varying technical implementations.
There is no definitive evidence, but some theories (like Mike Levin’s) suggest that self-organization and coherence in biological systems—found in plants and animals—might imply a form of sentience. This remains an open and speculative question.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.