Can AI models reliably break historical ciphers like Napoleon's 1809 letter?

A more skeptical reading treats striking cipher demonstrations as hypotheses until the original artifact, exact transcription, tools, transcript, search path, and failure cases are available. Plausible language can be overfit to an incorrect or incomplete problem, so a claimed solution to the Marmont letter should be judged by reproducibility and historical consistency rather than fluency alone.

• A claimed AI cipher victory is weak evidence when the ciphertext, model transcript, tools, reproduction package, and failed paths are unavailable.
• Historical cipher demonstrations can produce an apparently coherent answer to a problem whose underlying artifact or premise has not been independently confirmed.
• Even favorable commentary on a model solving a historical cipher later expressed uncertainty about whether the result was correct, underscoring the need for external checking.
• The Marmont account offers a reproducible human-facing solution package, but the cited description does not by itself show that a general AI model independently solved and validated the letter.

Featured sourceA cipher win is not an eval without the working - Ken Ashe | AI Application Builderkenashe.ai5 sourcesView sources