Barry Simon's Cities Experiment - An Overview
In May 1998, I formulated an experiment to test for correlations between the names of famous Rabbis and the cities in which they lived. I did so for two reasons. First, Mr. Gans had stated to me privately and also numerous times publicly that he had found a positive result involving Rabbis and cities although he has not publicly released the details. As you will know from other writing of mine at this site, I am concerned that claims about the validity of the codes are connected with wiggle room, so I decided to try to make an experiment with as little wiggle room as I could arrange to see if there was an effect.
Secondly, I wanted to respond to the proposal by Prof. Haralick that I consider a joint experiment with him and Prof. Rips. I have almost 100 co-authors and a lot of experience in joint research projects. It is clear to me that given the fact the we are geographically separated, a considerable time and expense would be involved. To justify this effort, I felt there had to be some substantial prior indication that the codes phenomenon was real. Accordingly, a preliminary experiment seemed the way to go.
In a nutshell, the experiment proposed using the method of the original Witztum, Rips and Rosenberg paper (WRR) and the exact list of appellations they used (for the union of the their two lists of Rabbis). The proximity to the cities in which they lived were to be measured by the WRR method. The only new element of the experiment as compared to WRR (or the Gans experiment) is the use of cities and their source.
To minimize wiggle room, I choose the Margaliot encyclopedia that WRR used to decide on which Rabbis to include. I designed the experiment by myself and made only minor changes after consulting my wife, Alec Gindis and Brendan McKay.
To avoid any possible claim of my choosing the data in a biased way, I asked Alec Gindis to have three Yeshiva students from Jerusalem extract the data. The raw data is supplied at this site so the reader can compare with the encyclopedia if they wish.
Only after they accumulated the data was it forwarded to Brendan McKay who has the software to apply the WRR method to the data thereby completing the experiment that I proposed. His report of the results is also at this site.
I proposed that a significance level (as defined by WRR) of 1 in 1000 be the borderline to take a correlation of cities and appellations as substantial enough to warrant a complicated research project. In understanding this significance level, I note that for the original WRR experiment but with the combined Rabbis lists the significance level was greater than 1 in 1 billion so the threshold level of 1 in 1000 is quite modest. It is also the significance level that Diaconis proposed for the original experiment.
In fact, using the WRR method (of multiplying the smallest of the four P levels by 4), the significance level is smaller than 1 in 2 (!!).
I conclude that no apriori case has been made to justify the time of an elaborate joint experiment, although I encourage Prof. Haralick to continue his work on the original Rabbis experiment.
I also conclude than any effect in the Gans version of the cities experiment is deja vu all over again: it is entirely due to the effect of arbitrary spelling rules - in other words, a consequence of the use of wiggle room.
The remaining links in the discussion of this experiment are: