Pénombre, la vie publique du nombre

--

Jetlagged statisticians, is there a pilot in the computer ?

July 20, 1998, 8:20 p.m., the news report on France2 (one of the main French TV channels). According to the ILO, France has the highest rate of workplace violence : 11.2 % of men and 8.9 % of women are allegedly victims of violence each year. France also tops the list for sexual harassment (again, at the workplace), with an annual frequency of 19.8 %, while figures for the other Western European countries range from 7.6 % (Sweden) to 0.8 % (Austria). There is never much time for in-depth analysis on TV, but the commentator does add “maybe that’s because our statistics are compiled better !”. The citizen member of Penumbra, hearing that, can only react with a : good for that, newscaster ! It is in fact always problematic to compare surveys between different countries, or at least, the comparability should be carefully questioned. It is worth noting and praising the journalist who, although unable to check on the information, is on the lookout.

Since I myself am in a better position to make that check, I decided it was a good idea to look into the subject, if only to report my conclusion to that vigilant journalist. I thought I would discover that the formulation of the questions, in different languages and different sociological contexts, could hardly yield comparable answers. The feeling of having been victim to an aggression is highly subjective and context-linked. I therefore asked to see the questionnaires. This was a “victimization” survey conducted in some thirty countries under the auspices of the United Nations. It covered attacks of all sorts (affecting individuals and property), and the workplace was only one particular aspect, isolated from the overall survey for the needs of the ILO.

To my surprise, the French questionnaire contained no questions about the workplace. The reference questionnaire (in English) did contain such questions, as did the French version of the Swiss questionnaire. The problem was radically different, then. It was no longer a matter of the comparability of a question, but rather, of discovering how it had been possible to establish findings although the question had not been put. Are some statisticians extraordinarily smart ? I will spare you the details of my investigation of the investigation. My conclusion was as follows. This was the third time this survey had been conducted : the workplace had been introduced for the second version, and since France had not participated in that second version, it had kept the same questionnaire as for the first. No-one had noticed the omission. Subsequently, in its automatic analysis of the responses for all of the countries, the central computer had replaced the missing question by the next one. So the question did get answered, after all, but the answer was meaning-less.

 
Two lessons

There are at least two lessons to be drawn from this story :

 – first, that the organization of this type of surveys, involving a number of institutes, elicits a loss of control over the technical aspects of the process (checking of the questions and checking of the data collected) ;

 – secondly, that regardless of the technique, it is of some concern that all those individuals who had an opportunity to see those results accepted such a tremendous difference in the order of magnitude without a wink. No-one was surprised. The experts in violence all got busy explaining with the utmost seriousness the whys and wherefores of the phenomenon, and the dangers involved, with no astonishment over its magnitude.

No-one. Except the commentator on France 2, who had some suspicion. And it was the accidental hearing of that commentator by a curious statistician that let the cat out of the bag.

If such is the case, then why conduct a survey, if the same speeches are to be heard irrespective of the findings ?

The discovery was made too late to prevent a publication such as Le Monde – Initiatives (a thematic weekly) dated October 7th from reproducing the piece. One imagines that absolutely everyone is prepared to pick up such findings, even when they are that extravagant. What, then, should we imagine, and fear, when the findings are only slightly warped !

The next riddle is : on the ILO list of prize-winners, Argentina is vice champion for sexual violence, with 16.6 % (third prize goes to Rumania, with 10.8 %). If France is disqualified, what’s the story for the challenger ? I sent my report to a colleague in Buenos Aires, and am waiting to hear from her. Is Argentina’s survey defective as well ? or are Argentinians definitely hot-blooded ?

René Padieu
April 1999

 
Penumbra, 2001 June