February 8, 2012
Jewish Community Charting Errors Add Up
This blog ends with an attempt to simplify with a parable. Whether the U.S. Jewish population is 5.2 million or 6.4 million may seem like a demographers’ squabble (previously in 1, 2, 3, 4, 5) . The primary issue may be: Does the organized Jewish community continue to field an admittedly expensive National Jewish Population survey which it has historically done for the past twenty years or is there a cheaper and adequate alternative? A group at the Cohen Center for Modern Jewish Studies Brandeis publicized a much cheaper, and they hoped to demonstrate, as good a tool as the previously fielded National Jewish Population Survey. I wish that Brandeis had succeeded in their claim of pioneering a new estimation method that would save the Jewish community significant resources, now even scarcer due to the economic downturn.
The 5,148 Jewish households found by the 2000-1 National Jewish Population Survey by calling hundreds of thousands random digit dialed (RDD) telephoned American households produced the last estimated 5.2 million number of American Jews.
The Brandeis group achieved their 6.4 million American Jews 2010 estimate by creating a panel of a bit over a thousand Jewish households by using a pre-existing survey panel frame of fifty thousand American households maintained by a firm called Knowledge Networks. Knowledge Network rents out the use of their panel which it argues mimics the US population and is not subject to the weaknesses brought about by increasing cell phone use that affect the utility of the RDD method.
The second stage of the Brandeis group’s method was taking over a hundred national surveys done in the past decade which asked: “What is your religion,” often with less than two thousand adult household respondents per survey and standardizing these surveys to the Knowledge Networks’ demographic characteristics to derive what the percentage of respondents answering Jewish was in the surveys. From this the Brandeis group reported that 1.8 percent of all the adult respondent in the standardized 150 surveys were Jewish. 1.8 percent of the US census yielded the estimated adult Jewish population estimate and the under age 18 children seemed to be estimated from what was learned from the thousand Jewish households from the Knowledge Networks panel which was then determined to be an overall estimate of 6.4 million U.S. Jews.
Every survey has some error. What’s important is if the error goes in a specific direction or its just random and may be just a wash. I have found that using Random Digit Dialing introduces randomization which better controls for directional error which can magnify or diminish a population characteristic. Jewish surveys depend on reliably identifying Jews for which the “Jewish screener questionnaire” has long been the gold standard. The Brandeis group was able to use a Jewish screener questionnaire only on the Knowledge Networks households panel but not on the other surveys on which their population estimate was based.
I wish that the American Jewish population, using the same way of defining Jews as in the past, was growing, or even remaining stable at 5.2 million. I haven’t seen evidence of significantly increased birthrates, longevity and/or in-migration from abroad among Jews in the U.S. In my mind, I just can’t account for the Brandeis group’s estimated increase of 1.2 million Jews, or 23 percent, in the last decade. I don’t think that the last two RDD based National Jewish Population Surveys missed a million Jews in 1990 and 2000.
I’m sure there are cheaper ways to derive a US Jewish population estimate because in 1996 I began testing one component of a method that could be adapted for more valid Jewish population estimates and subsequently published a paper about the findings in 1997.
Here’s an attempt at explaining the Brandeis research by creating a parable:
As the waters of American surveying have gotten murkier because of increased telephone RDD refusal rates and decreased telephone land-line penetration, the use of the reliable SONAR of Jewish population study, the RDD Jewish screener process, has been growing in cost because of the increased number of soundings needed to discern the sea floor terrain and characteristics of the Jewish community.
The ship’s pilot, the organized Jewish community, is tired of paying for SONAR soundings for those expensive charts and has been muddling through using outdated charts, instinct, memory and luck to navigate channels that it knows are shifting. Let’s say that the seafloor has coral reefs and one type of coral is the “Jewish” coral.
Along comes a chart-maker who says “I have a shiny new and inexpensive way to make charts for you. I’ve found the company with a big powerful glass bottom ship that I can use to look at the sea bottom and find the Jewish corals and then I can use a fleet of inexpensive little glass bottomed boats, built by others for other purposes, and adjust their glass bottoms on the basis of my measurements of Jewish coral from the large glass bottomed ship to create a chart of the Jewish coral community. The chart-maker’s best efforts don’t take into account that the glass on the bottom of the big glass bottomed ship and fleet of glass bottomed boats is more convex, therefore, magnifying the Jewish community. This is an attempt to stencil on the glass bottom that “Jewish Coral May Look Larger Than They Actually Are.”
Pini Herman, PhD. has served as Asst. Research Professor at the University of Southern California Dept. of Geography, Adjunct Lecturer at the USC School of Social Work, Research Director at the Jewish Federation of Greater Los Angeles following Bruce Phillips, PhD. in that position (I was recently notified that with 40,000 visitors this year the 15 year old study of the LA Jewish population was third most downloaded study from Berman Jewish Policy Archives in 2011) and is immediate past President of the Movable Minyan a lay-lead independent congregation in the 3rd Street area. Currently he is a principal of Phillips and Herman Demographic Research. To email Pini: firstname.lastname@example.org To follow Pini on Twitter: Follow @pinih