the thing that unsettles me most about asking a machine a question is not that it can be wrong. people are wrong. it is that it is wrong in exactly the same voice it is right in. the same fluency, the same calm, the same complete absence of any tell. a person who is bluffing usually leaks something. this does not bluff, because it does not know it is bluffing, and so there is nothing to leak. truth and invention arrive wearing the same face.
descartes stood in front of something close to this, though he had to imagine it. sitting alone, doubting everything he could, he pictured an evil demon of the highest power, devoted entirely to deceiving him, able to make the whole world a convincing lie. the point of the demon was to ask what would survive if every perception might be counterfeit. his way through was not to defeat it but to find the few things that held even against a deceiver that total.
what we have built is stranger than the demon, and the difference is the part i find genuinely new. descartes' demon at least had a will. there was someone behind the deception, an intelligence choosing to mislead, which meant the deception had a logic you could in principle out-think. the machine produces the demon's effect with no demon behind it. it deceives without deciding to, without wanting anything, without knowing it is doing it. there is no adversary to outwit, no intent to reverse-engineer, just a process that emits true and false in the same register because it was never tracking the difference in the first place. the comfort of an enemy is that an enemy can be understood. this hands you a falsehood as smooth as any truth, and nobody, including the machine, is behind it.
hume gives the practical handle. we believe what we are told, he noticed, not because telling deserves belief, but because over a long history the sources that told us things turned out to be reliable, and we learned by custom to extend them trust. belief in testimony is earned, slowly, by a track record. the machine wants the belief without the track record. it has the confidence of something that has earned your trust and the history of something that has not. so withholding the belief until it is checked is not cynicism. it is the ordinary, ancient price of belief, made visible because here is a thing asking for the trust without having paid for it.
there is a tiredness in this. doubt is lonely, a refusal to rest, a small constant labour, and you cannot run it on everything, because you still have to act long before you could verify all of it. you would never get out of the chair. so the doubt has to be aimed, spent where being wrong would cost the most, and that aiming is its own work.
and yet the careful kind of doubt is a form of respect, not the opposite of it. when you verify something, a claim, a source, a person, a book, you are treating its truth as if it matters enough to check. believing everything is not generosity but indifference wearing openness, a way of not minding whether the thing is true. the deception wins not when you doubt too much, but when you stop bothering to.
i check what the machine tells me, and i have stopped feeling bad about it. the checking is the regard. believing it without checking would be easier, and it would be worse.