Re: Cipher vs Language

[Jorge Stolfi <stolfi@xxxxxxxxxxxxx>]

    > [stolfi:] Still, it is strange that people find the Crypto
    > Theory more likely than the Chinese Theory. Scores of
    > professional and amateur cryptographers have tried to crack the
    > "code" for almost 90 years, and have made absolutely *zero*
    > progress. Worse, the crpto camp cannot even explain away the
    > many arguments that point to the VMS *not* being a code.

    > [Glen:] I had an instant "gut reaction" to this, until I
    > realized that this is simply your gut reaction to people poking
    > at Chinese all the time.
    
Email is a great tool for making enemies, it seems. I am sure that you
would have none of that "gut reaction" if you heard me say that in
person --- say over a pizza and beer. (Not even if you had to pay for
the pizza. 8-)  

I am still too aware of my own glass roof to poke fun at other
people's efforts.  I wrote that as a statement of fact, and I am
puzzled indeed.

No matter whether it is Chinese or code, the VMS text has many
peculiar properties that have become known over the last 2-3 years: 
the word-length distribution, the word frequency-rank relation (Zipf
plot), the internal token structure, the similarity between labels and
text words, and so on. These properties impose certain constraints on
the encription method (if code) or on the language (if plaintext).

Those constraints appear to have ruled out all plausible encryption
systems, except for codebook; just as they appear to have ruled out
all plausible natural languages, except monosyllabic ones.

I am aware that many quite reasonable people, like Glen and Philip,
find a non-European origin so unlikely /a priori/ that they would
rather believe in impractically complicated codes, Byzantine decoys,
and secretive communities of herbal conspirators, just to avoid it.
That's OK. My spiel about "Human Rights and Bayes Theorem" was
meant to be humorous, but not in the least condescending: I *do*
believe that /a priori/ probabilities are a matter of faith and 
"gut feelings", not science, and should be respected as such.

Still, I had expected that the crypto camp would at least take notice
of those very real constraints, and either try to find holes in them,
or start discussing the codebook theory in earnest, or start looking
for other encryption schemes that would meet the constraints. But
while there were some attempts at the latter (such as Gabriel's "daiin
daiin" code and Rene's sorted-letter anagrams), they apparently
weren't followed up. It would seem that the cryptos who are still
working on the manuscript have simply ignored the existence of those
constraints, and that some are still pursuing character-based cipher
schemes that simply *can't* be right. Shouldn't I feel disappointed?

    > I guess to you Crypto is one of your favorites [shooting
    > targets]. Why else would we have "crypto-kooks",
    > "crypto-wierdos", and other slanderous terms that group a
    > certain set of philosophies into a single derisive label?
    
Please note that I have NEVER used these terms. Even though I don't
find criptography a particularly exciting field, I do respect
"cryppies" (as I believe they like to call themselves) as much as any
other scientists, and I admire their remarkable successes --- e.g. the
recent exploits of Jim1 and Jim2.

    > I could easily take your own words and rewrite them briefly thus:
    > 
    > "Still, it is strange that people find the Language Theory almost as
    > likely as the Crypto Theory. Scores of professional and amateur
    > Linguists have tried to crack the "language code" for almost 90 years
    > 
    > This has every bit the validity of your statement
    
Sorry, I disagree. The "scores of linguists" who looked at the
manuscript in any depth were basically one person --- Jacques Guy.
IIRC, Friedman's group included a linguist at first, but he dropped
out quite early. Miguel Carrasquer was around for a few months but
hasn't been read since. Did I forget anyone else?.

I am a computer scientist, not a linguist. I have been playing the
amateur linguist recently because the statistical evidence made me
believe in the language theory --- not the other way around. And I
don't seem to have made many converts...

    > and [linguists] have made absolutely *zero* progress.
    
Not so: the language camp now has good evidence that the physical
words (space-delimited strings) are indeed linguistic words (units of
meaning), and that their internal structure is syllable-like. The
possible natural languages have been narrowed to a fairly restricted
set. Many "unnatural" properties of the VMS text have been shown to
occur in such languages. Finally we now have a questionable but
definite interpretation for the two symbols on f1r, which points to an
even smaller subset of those same languages (those which used Chinese
characters at the time). And, finally, we have a plausible
who/why/how/when scenario which seems to be at least compatible
with all the available data.

    > Worse, the Language camp cannot even explain away the many
    > arguments that point to the VMS *not* being language."

This is not a fair claim. I offer the word structure, Zipf plots, and
label structure as arguments against character-based ciphers; and the
size of the text as argument against codebook-based ones. 
How do you get around them?

The arguments against language seem to be (1) it can't be an European
language because the statistics don't match, and (2) it can't be a
non-European language because the text and illustrations have an
obvious European look. I accept (1), but (2) is a non-sequitur (see
below).

    > It's fair in my mind to say that the VMS has historically been a
    > hot potato no one of any credibility wanted to handle for very
    > long.
    
Still, several very good cryptographers worked on it for many years,
even decades. Those were people who had permanently established their
credibility by breaking state-of-the-art codes, given much less data,
in much less time. Some of them gave up when they got convinced that
it was plain language, and thus there was no code to be broken. Others
simply gave up in frustration at the complete lack of progress.

    > With every identification of a western plant or astronomical
    > diagram or instrument, the Chinese argument falls further in the
    > hole ... Look at D'Imperio's list of clothing worn by VMS
    > characters ...

The Chinese Party barks back that modern professional botanists
could not identify any strictly "western" plants (if there is such
thing) with certainty. Neither could Georg Baresh's doctor friends ---
even though they were presumably quite familiar with those same
western plants AND with western herbal drawing conventions of the
times.

Moreover, considering *why* the modern experts failed, it is clear
that most of the claimed "identifications" are merely "best matches":
"ASSUMING that this is a piture of a western medical plant, which one
would it be?" If you call that an "identification", then I demand the
same privilege for my reading of the big f1r weirdos --- and the case
is thereby closed.

As for people's faces, hairdos, clothes, buildings, etc.: there are
many examples out there proving that such details were often supplied
by copists/translators according to their own background, rather than
reproduced from the original book. (Consider for example all those
European pictures of Christ as blond and blue-eyed, or the
black-skinned statuette of Mary which is Brazil's "patron saint".)

As for astronomical instruments: before claiming them as "western",
you must show that they were not known in East Asia too. Which is a
rather difficult task, considering that Chinese astronomy is old and
vast, and much European medieval astronomy was imported from the
Arabs, who had extensive contacts with East Asia.

Finally, about the astronomical diagrams: I note that almost all of
them are still without recognizable European models. Even the European
Zodiac symbols and month names are inserted in diagrams that do not
match the European sign/month lengths.  

On the other hand, they seem to match the Chinese agricultural year
with 360 degrees (not 365 days), divided into "months" of 15 degrees
(not 28/30/31 days), and starting in February.

I would very much like to see a copy of that "Transit Star Catalogue for
Time Determination by the Ming Dynasty scholar Zhou Silian", for 
example... 

    > Okay, so it was somebody exposed to Chinese that chose to use
    > Chinese in a western herbal, without including Chinese plants or
    > characters.
    
Has anyone seriously looked for Chinese plants in the VMS? 
    
    > The system is written left to right, and the Zodiac is western.
    > Certainly Bayes gives you the right to arbitrarily ignore a
    > certain amount of opposing evidence, but I submit that this
    > information is the first evidence, the most crucial, and not to
    > be arbitrarily dismissed. ... in order to pursue a Chinese
    > connection, you must first arbitrarily ignore all the
    > pictographic and character information available that
    > unanimously suggests a western source and intellect.
    
Please, the Chinese theory does *not* ignore this information, quite
the opposite. It specifically claims that the VMS author was either an
European, or a native who embraced European culture and set out to
produce an European-style book. (Again, history records plenty of
examples of this sort thing.)

In fact, the assumption of a western or westernizing author is not
only a trivial explanation for the western look of the VMS, but also
provides a good explanation for why the VMS was written in the Chinese
language but not in Chinese characters. See my recent message on that.

    > Instead he chose to base his character set primarily on mirrored
    > Latin and astrological shorthand.
    
And why not? See above...

But there are better explanations for the symbol shapes (see my 
previous reply to Nick Pelling).

    > If it were meant for a Chinese audience, shouldn't at least one
    > of the annotating decipherers have used a familiar character,
    > such as the oladaba on f116v?
    
I did't get this one. 

In the Chinese theory, the VMS was most likely taken or sent to Europe
by the author himself. Attempts at "decipherment" would have started
only later, in Europe, after its origin had been forgotten. If the
author was a native, the "oladabas" text may have been his
(pathetically failed) attempt to write a "cover letter" or dedication in
some western tongue. If the author was a westerner, <insert your
favorite oladabas explanation here>.

    > No language I know of including Chinese requires the systematic
    > exercise of a specific mathematical rule to render it
    > intelligible.  
    
I suggest that you look at Gwoyeu Romatzyh (GR), an alternative system
for spelling Chinese with Roman letters that was standardized by China
in 1928. GR tries to encode tones without using numbers or diacritics;
instead it uses spelling changes, as shown in these tables:

    http://www.elgin.free-online.co.uk/read_gr_tones.htm
    http://www.elgin.free-online.co.uk/write_gr_3steps.htm

Note that GR is different from the "old" (Wade-Giles) Chinese
romanization system used in the West, which had "Mao Tse Tung" instead
of pinyin's "Mao Zedong".

If you think GR is not bad enough, please check the native spelling
rules and the Romanization proposals for Tibetan, Burmese, Thai,
Cambodian, ...

Even a few hours of reading about Chinese romanization should convince
you that (a) there are pretty good reasons to forget Roman letters and
invent a special-purpose phonetic alphabet (as Taiwan did); (b) there
are a zillion and one possible ways to encode syllable tone in
writing, all of them fundametally wrong; and (c) even if we know which
East Asian language we are dealing with, unraveling the author's
spelling system may be harder than cracking your average medieveal
cipher.
    
    > That you and I are discussing this subject gives question to our
    > credentials.
    
One of the few - very few - advantages of being a professor at a
second-rate third-world university is that I don't have to worry about
preserving my scientific reputation.

Just to give you an example, not long ago a professor at the
Mathematical Physics department across the street found a way to send
signals faster than light -- by a "scissor wave" method that is
dismissed in one sentence of Feynman's textbook. He was featured on
the cover of the university's official newspaper as "the man who
proved Einstein wrong", and was eventually elected Dean of the Maths
Institute.

As for me, I just got a full professorship here, in spite of, or
perhaps thanks to, the Chinese theory. (One board member *did* ask
about that "mysterious manuscript stuff" at my examination.) Had I
embraced the Martian Theory instead, I may have been University
president by now... 8-)

    > I just want you to know up front why I'm going to be the one to eat
    > your pizza, so there won't be any hard feelings when I don't offer you
    > a slice. In appreciation I will do my best to learn the word "no" in
    > Chinese. :-)

OK. I already know the word in Latin and English. 8-)

All the best,

--stolfi