Drifter said:
Machines have been capable of adjusting their outputs according to their inputs for hundreds of years.
A major distinction is complexity. For one thing, machines with truly significant memories (significant amounts of "state") are quite new, and the kinds of memories in most computers also don't lend themselves to the kind of pattern creation and recognition that human brains casually (and constantly) do. So even if we cast aside the supernatural, we still have a ways to go before the "machinery" we create is up to the task of learning and acting the way we do.
As an aside, "machinery" and "state" aren't necessarily separate things. Today's application programs are arguably machinery, but they're also state (just data) as far as the hardware is concerned. Self-modifying programs aren't even a new thing, and in fact self-modifying
hardware also already exists. Look at FPGAs. They're just a bunch of elementary logic circuits floating in space until you "program" the interconnections between them (this "program" is nothing like the sequential computer programs most people are familiar with). The device can then become just about anything you want--within the realm of digital logic, anyway. (Interestingly, this was presented as a new and horrifying idea in the movie Terminator 2, but the technology already existed on a small scale when the movie was released.)
I imagine that the "brain" of a truly sentient AI (or one that's indistinguishable from sentient), if such a thing is indeed possible, would be more like a massive FPGA than a computer. It wouldn't take in data and process it sequentially according to a fixed set of software algorithms. Instead, everything would be done simultaneously (or nearly so) using algorithms implemented by self-modifiable hardware. That hardware would therefore have some inherent limitations, but its behaviors, as well as how those behaviors are implemented, would be wholly or mostly learned.
But back to the supernatural stuff: I'm a nonbeliever, but I still find the idea that I'm just a configuration of matter pretty unsettling. I also struggle to see why it can't be true, though.