The text as a crime scene
We had been in Paris for three days. Holmes had agreed, not without reluctance, to lend a hand to the French authorities in a forgery case whose details I shall not disclose here. We were staying in a Left Bank apartment placed at our disposal by the Sûreté. It was narrow but comfortable, with a fireplace that drew well and a window overlooking the rooftops.
That evening, Paris murmured under a fine rain. From the street below came the rumble of cabs over the wet paving stones and, now and then, the distant cry of a street vendor. Holmes had not yet lit the large oil lamp. A low glow from the fire slid across his lean and sharply drawn profile, across the cluttered desk, the cooling teapot and the piles of files. He sat sideways in his armchair, fingers joined beneath his chin, eyes half closed, in that stillness which always deceived superficial minds, who mistook for rest what in him was only concentration carried to its highest pitch.
I was holding a few loose sheets. Holmes, without turning round, startled me:
“You have tonight, my dear Watson, the stiffness of a man divided between curiosity and embarrassment. Put the object of your trouble upon the table and stop trampling on my carpet.”
I started. How on earth had he known?
“It is not, well, exactly, a crime. It is a text.”
Holmes opened his eyes. A glint of interest passed through them, brief as lightning.
“Ah. Crime by language. It is often subtler than the other kind.”
I smiled in spite of myself and went on:
“It may not even be a crime. A friend sent me this document together with a question. I thought it might interest you, and perhaps pull you out of the morose humour in which I have seen you all morning.”
Holmes raised an eyebrow but made no comment. He took the sheets, held them up to the fire as if to test the grain, sniffed them briefly, an old habit, and laid them down again. Then he sprang to his feet with that abruptness peculiar to him and lit the lamp. The room immediately assumed that air of an intellectual anatomy cabinet which it wore in hours of serious analysis.
“And what is the question, Watson?”
I moved towards the table, my heart beating a little faster despite myself. I knew I was about to set Holmes on a new trail, and such moments always carried something electric within them.
“Holmes, how can one tell whether a text was written by an artificial intelligence?”
He looked at me with that faintly ironic, almost cruel crease which appeared in him only when a question had been badly framed. I felt myself blush.
“My dear Watson,” he said, folding his arms, “you must first rid yourself of the childish need for certainty. In this matter one almost never knows in the sense you mean. One suspects, compares, weighs, eliminates, and then concludes in degrees of probability. Whoever claims to recognise machine-assisted writing with absolute certainty from a single sign deserves less the name of investigator than that of fortune-teller, or... charlatan.”
I lowered my eyes, somewhat mortified. Holmes must have noticed, for he softened his tone:
“That said, there are indeed clues. Not proofs, mind you, but clues. And a disciplined mind makes better use of them than an impetuous one. Come closer, Watson. We shall examine the matter together.”
He sat down before the sheets, smoothing the first with the flat of his hand with an almost tender delicacy, and added:
“Where would you begin?”
The question caught me off guard. Holmes rarely asked for my opinion.
“I... I do not know,” I confessed. “With the meaning, perhaps? With what the text says?”
“A classic error, though a pardonable one. No, Watson. With the visible. Always with the visible. Language has this admirable trait: it betrays itself even in its clothes. Look here.”
He pointed to the beginning of the text, tapping the paper with his forefinger.
“Observe these quotation marks. They are not French. They are straight quotes, born of international usage, convenient for the machine, common in Anglophone environments, and very frequent in certain assisted productions.”
I bent over the page. Indeed, the quotation marks resembled those I used on my American typewriter.
“I do not say that a French author cannot use them,” Holmes went on. “I know some negligent enough, or digitised enough, to do so without remorse. But it is a first small pebble on the path. Then here, look closely: this stiff punctuation, these overly uniform spacings, this impeccable presentation, almost hygienic. Nothing spills over, nothing lives, nothing resists. The page looks as though it had been combed.”
“Combed,” I repeated, struck by the image.
“Yes. Like a formal French garden where not a leaf would dare fall awry.”
“Is that enough to awaken your suspicion?”
“To awaken it only. Not to satisfy it. A good secretary, a meticulous journalist, a practising lawyer may all produce such neatness. One must go further.”
He read a few lines in silence. I watched him, as ever fascinated by the intensity of his concentration. His eyes moved to and fro across the lines with a speed that made me dizzy. Then he lifted his chin and smiled, that hunter’s smile of his when he scents the trail.
“Now listen to the lexicon.”
“The lexicon can be listened to?”
“In your case, my dear Watson, it generally contents itself with being heard.”
I could not suppress a grimace. Holmes, magnanimous, continued:
“To the investigator, it reveals its provenance. In this text, the words are not wrong. That is their cunning. They are exact, polished, appropriate. But they inhabit a region of language where many terms seem serious without being very tight. They promise rigour before quite delivering substance.”
“I am not sure I understand.”
Holmes rose, took a few steps towards the window, and gazed for a moment at the rain through the pane. Paris glimmered faintly in the night.
“One encounters there,” he said without turning round, “well-dressed abstractions, words that promise more than they commit, connectors that follow one another with remarkable docility. One reads: ‘moreover’, ‘furthermore’, ‘from this perspective’, ‘it should be emphasised’. One senses less a thought drilling into the matter than a formulation apparatus busy keeping itself properly upright.”
I took the sheet and read a few passages under my breath. Holmes was right: everything was correct, and yet something was missing. A flavour, perhaps. A grain.
“You mean that it sounds right, but in too general a way?”
Holmes turned sharply, his eye gleaming.
“Precisely! You astonish me, Watson. The vocabulary gives the impression of seriousness before quite supplying its substance. It is like those men who wear evening dress admirably and whose conversation, after ten minutes, leaves no trace.”
He returned to the table and perched on its edge, legs crossed, in that careless pose he adopted only when fully engaged.
“There is sometimes, too, in machine-assisted writing, a faintly exotic lexicon. The French is correct, but it does not entirely dwell in its own house. It has something imported, smooth, transferable about it.”
“Like a translation?”
“At moments, yes. Not a bad translation, which would be simple enough to detect, but a language that seems to have passed through several offices before reaching us. A transit language, if you will.”
I nodded, beginning to glimpse what he meant. Holmes turned the page.
“Now let us come to syntax. There is an even more instructive terrain.”
He bent over the text with renewed intensity, like a physician auscultating a patient.
“The sentences here are well made. At times too well made. Their length varies, certainly, but with a regularity that smells of balancing. They advance neatly, answer one another, close without leaving any jolt behind. The sway of the clauses is almost constant.”
“Is that not a quality?”
“It may be. But look at the paragraphs.”
I looked. They were all of comparable length, like soldiers at attention.
“They have that average length so convenient for the contemporary eye,” Holmes continued. “Not too short to seem superficial, not too long to risk density. One would think them calibrated so as to offend no one.”
I set the sheet down again, troubled.
“So you condemn clarity?”
Holmes gave a brief smile, almost warm.
“Never. Clarity is a virtue. Mediocrity is its caricature. A living mind may be lucid and remain singular nonetheless. What interests me, Watson, is the absence of fertile accident.”
“Fertile accident?”
“The human mind resumes, branches, insists, hesitates, or corrects itself in the turn of a sentence. It leaves small tensions in its syntax. Here, everything slides. And what slides everywhere ends by lacking any grip.”
He rose abruptly and seized his violin, which rested against the wall. He plucked a string, let the note vibrate for a moment in the damp air, then laid the instrument down again with a sigh.
“The same applies to style. It is possible, Watson, for a text to be intelligent and yet fleshless.”
“Fleshless?”
My voice betrayed my astonishment. Holmes nodded gravely.
“Yes. It may explain everything, connect everything, balance everything, and risk nothing of itself. Machine-assisted writing, when it has not been powerfully reclaimed by a true hand, often has this defect: it produces reasonable prose. Fair. Continuous. It avoids sharp asperities, overly personal angles, sallies that might commit a temperament. It prefers general agreement to an accent of its own.”
“That does not necessarily make it a bad text,” I objected.
“No. It often makes it a disembodied one.”
The word struck me. Disembodied. A text without a body. I sat opposite Holmes, feeling that we were touching something important.
“Now that you say it, that is exactly what strikes me. The document I brought you contains nothing false, nothing ridiculous, nothing frankly clumsy. And yet, when I reached the end, I had not the impression of having met someone.”
Holmes inclined his head approvingly. His eyes, usually so cold, held an almost affectionate light.
“You are progressing, Watson. Detecting a possibly machine-assisted text does not consist only in counting outward signs. It consists in sensing what the text does, or does not do, in the reader’s mind.”
He got up and began pacing the room, his hands clasped behind his back, a sure sign in him of thought organising itself.
“Some machine-assisted texts seem composed by the aggregation of plausibilities. They answer expectations, reformulate deftly, synthesise admirably, but possess no inner necessity. They resemble those very handsome faces from which no precise memory remains an hour later.”
“Forgettable faces,” I murmured.
“Exactly. Handsome, but forgettable.”
“Would the suspicion then be strengthened by the relation to reality?”
Holmes stopped short and fixed me with intensity.
“Very often. Ask whether the text dares a situated detail, a singular memory, a roughness born of experience. Does it observe closely, or does it distribute well-behaved generalities? Does it leave something implicit, or does it feel the need to explain every step of its staircase?”
He walked a few paces towards the fireplace and contemplated the flames for a moment.
“A machine tends to wrap the parcel well. A human being, especially when he is really thinking, sometimes lets you glimpse the crumpled paper.”
I took up the sheets again, turning them between my fingers.
“And yet, is it not unjust to suspect a text simply because it is so well kept? You yourself have written articles of great correctness, Holmes.”
Holmes turned with a sardonic smile.
“Thank you for reminding me of my virtues. It saves time.”
I rolled my eyes, but he went on at once, serious again:
What Holmes looks at first
“You touch on an essential point there. The absence of errors is nothing, absolutely nothing in itself. A good author, a rigorous academic, an experienced writer, a piece of prose long revised, may produce an impeccable copy. Likewise, some machines today produce awkwardnesses or preserve irregularities; others even imitate the small human mistake with suspicious skill.”
He raised a finger, as if to underline what was to come.
“One must therefore never say: ‘There are no mistakes, so it is AI.’ That would be consummate stupidity. One must distinguish natural correctness, professional correctness, and that artificial homogeneity which gives the impression that everything has been washed at the same temperature.”
“I begin to see your method,” I said, with an enthusiasm I no longer tried to conceal.
“Then let us finish it.”
Holmes sat down again and crossed his legs. Outside, the rain had intensified. From the boulevard came the crack of a whip and the neighing of a horse.
“Look at the overall structure. That is often the most eloquent terrain of all. Assisted texts willingly present themselves with a polished introduction announcing what it will say, a development neatly cut without surprise, and a conclusion that recapitulates exactly what the reader had already understood. It is comfortable, often useful, and at times even effective.”
He paused, eyes fixed in vacancy.
“But a genuinely strong human thought shifts the ground beneath its own feet. It discovers as it advances. It does not merely execute a plan; it is sometimes altered by that plan.”
“So a text that is too well organised may awaken suspicion?”
“Too well organised? No. Too exactly functional? Yes.”
Holmes leaned towards me, lowering his voice as if in confidence:
“There are paragraphs that resemble model employees: each fulfils its task, none oversteps its brief. In some cases this is simply the work of a competent professional. In others it betrays template production. The investigator does not decide at once; he compares. He asks: is this regularity the sign of human discipline, or of mechanical and statistical fabrication?”
I kept silent for a moment. A cab passed in the street, its wheels hissing over the wet stones. Somewhere an hour struck ten.
“Suppose,” I said at last, “that a teacher, a journalist, a lawyer writes in this very neat, structured, measured manner. Do we not risk treating him unjustly by suspecting he has turned to the machine?”
Holmes leaned back slightly in his armchair and joined his hands beneath his chin.
“We do risk it, Watson. And that is precisely why one must resist at all costs the intoxication of diagnosis.”
His gaze grew grave.
“There are excellent false positives. A non-native speaker may punctuate oddly. An experienced writer may employ highly standardised turns of phrase. A translated or heavily revised text may acquire that smooth surface you too hastily associate with artificial productions. And do not forget hybrid texts: human lines revised by machine, or the reverse. We are entering a zone of mixture in which the question is no longer ‘machine or man?’ but ‘in what proportions, at what moment, with what effects?’”
“So it is an inquiry in degrees,” I murmured.
“Exactly. A graduated inquiry.”
Holmes straightened and counted on his fingers:
“I shall summarise it in five gestures, which even you, Watson, may retain without disturbing your sleep. First, observe the visible marks: punctuation, quotation marks, page layout, typography. Then note the habits of the text: its lexicon, its connectors, its discreet redundancies. Next, listen to its movement: syntax, transitions, regularity, the absence or presence of jolts. After that, replace the document in its context: who writes, for what use, after what revision, in what possible source language. Only then formulate a prudent hypothesis.”
He paused with solemnity.
“Not: ‘it is AI’, but: ‘this text presents several signs compatible with machine-assisted or heavily normalised writing’. Nuance is not a luxury, Watson. It is the honour of intelligence.”
He took the last sheet, regarded it as one looks at an insect pinned beneath glass, and then laid it down delicately.
“And your verdict on this one?” I asked, my throat a little tight.
Holmes shook his head with a half-smile.
“My verdict? I do not give one. I give an assessment. This document presents several convergences: imported typography, a lexicon that is serious yet vague, syntax balanced to excess, smoothed style, a highly programmed structure, and little singular anchoring. All of that makes the hypothesis of assisted writing plausible. Plausible, Watson. Not certain.”
“But then...”
“It might also be the work of a human author very concerned with correctness, somewhat displaced in his own language, a little too polite in his thinking.”
“So you leave the doubt standing.”
Holmes looked straight into my eyes.
Suspicion is not a verdict
“Naturally. Doubt is not the enemy of inquiry; it is its discipline. Fools want a culprit every minute. The rigorous mind prefers a cluster of clues to an illumination.”
I took the sheets, folded them slowly, and felt, a rare thing, that they interested me less now as objects than as symptoms of a larger phenomenon. It was no longer simply a matter of knowing whether a text came from a machine, but of understanding what a machine was gradually doing to our way of writing, perhaps even of thinking.
What the inquiry reveals about language
Holmes, who had been watching me, guessed the thread of my reflection. He nodded slowly.
“Yes, Watson,” he said more softly. “The question is not merely a police one. It is literary, intellectual, almost moral. The more these instruments write correctly, the more we shall have to learn to distinguish not the true from the false, as one separates an authentic coin from a crude counterfeit, but inhabited speech from speech that is merely plausible.”
He bent to relight his pipe, whose bowl had gone out. The flame of the match lit his angular face for an instant, hollowed by shadows. Then, in a cloud of fragrant smoke, he concluded:
“In matters of suspicious writing, my dear friend, the investigator does not hunt for a magical signature. He seeks the convergence of clues, and beyond that the rare thing no mechanism can feign for long...”
He drew a long puff and let the silence settle.
“A language that is truly inhabited.”
Outside, Paris slept beneath the rain. And I, Watson, thought that we had perhaps, in that small Left Bank apartment, laid the first stones of an inquiry that would occupy generations to come.