Qualimetrics Contributions to Research Methodology
Preface by David Boje to RECHERCHE EN SCIENCES DE GESTION: OBSERVER L’OBJET COMPLEXE – APPROACHE QUALIMETRIQUE by Henri Savall and Véronique Zardet, Lyon 3 & ISEOR
Henri Savall has spent 29 years developing an alternative to the sociotechnical systems paradigm, which he calls ‘socio-economics.’ This book, with Véronique Zardet (his long time accomplice), is a detailed account of socio-economics philosophy of research (See Savall, 2003 special issue in Journal of Organizational Change Management). Their challenge to traditional sociotechnical research and management science research is that neither one focuses adequately upon the financial context of change. Organizational change, for example, often explores social and technical, but leaves financial to the imagination. Management Science, while more open to triangulating qualitative and quantitative methods, has been less able to link these to financial methods. This is fundamentally a book about methodology, one that proposes to work the middle between qualitative, quantitative, and financial methods. They situate their research paradigm between empiricist and constructivist, refusing to dualize. They do not want to choose one and exclude the other. They invent a new term for their betwixt and between approach: ‘qualimetrics,’ a bridge from qualitative to quantitative, as well as financial methodologies. Since I think this is a new metatheory, as well as new method; I would like to give it some moorings.
There is no position between qualitative, quantitative, and financial methods; rather there is instead, an ‘arriving’ on the bridge, a kind of middling, a refusal to take sides. The word “arriving,” for Savall/Zardet, also implies a history of interaction between qualitative, quantitative, and financial phenomenon; qualimetrics is dynamic; Savall and Zardet never complete their quest to let one amplify understanding of the other two. To many of us in management science, quantitative and financial mean the same thing. For Savall and Zardet, the differences matter.
The distinction between quantitative and financial, is that the former is about numbers, and the later is concerned with analysis. And Savall and Zardet are keen not to exclude the qualitative. At its simplest expression, there is an ‘arriving’ at the qualitative validation of quantitative (the numbers) with an exploration or tracing of the translations and exchanges that construct financial statements. In management science, we know that the quantitative side has a qualitative evaluation, and vice versa; we know far less about how to integrate qualitative with the financial methods. Savall and Zardet move us deeper still, daring a look at ways one becomes the other two, and ways one traps, even tortures the others, not merely in academic research, but in organizational practice. And they provide a way out of this torture chamber by looking at the stories behind the numbers, and the social passage points in erecting analyses and accountings of the firm.
Savall and Zardet’s work is related, I think, to that of Robert Gephart’s (1988) ethnostatistics, to Jacque Derrida’s (1991) deconstruction, Norman Fairclough’s (2003a) role of language in socio-economics of “new capitalism,” and to Kenneth Burke’s (1937/1957) “excluded middle.” I will dance each one with Savall and Zardet’s qualimetrics so that you can see how profound and important it is.
Ethnostatistics, for Gephart (1988) is the study of how statistics are constructed, displayed (or used), and interpreted by researchers. While the context of ethnostatistics is primarily in social scientific research, it can also apply to intervention research. “Ethno” means “actual behavior” of an “informal subcultural, folk” and looks at the activities of “producers and users” of numbers (Gephart, 1988: 10). Qualimetrics, like ethnostatistics is “concerned with mundane, everyday life practices, with lay and professional knowledge necessary to implement and use statistics” (Gephart, 1988: 10).
Qualimetrics is an analysis of how organizational members as well as ‘intervention researchers’ construct, use, and interpret statistics, in such practices as accounting, evaluation, production, and finance. There are contexts involved in producing numbers for use in accounting and management reports. There is a hegemonic reductionism, where individual stories are reduced to numerical displays. And the numbers mean different storied experiences to different people. Qualimetrics seeks to recover this loss through root cause analysis, a search for the intertextuality of attributed causes and effects (See Boje, 2001, chapter on causality). This is done by a combination of observation, interview, and document study. Savall/Zardet assume that information, be it qualitative, quantitative, or financial – is always partial. For example, if people skip the interpersonal (one-on-one interview) or the observation of social behavior, and rely on the document – they find the document leaves out many things. You find you need to interview people, and they will say “and you also need a copy of this and a signature from that other person.” Similarly, staring at a chart of accounts, you find many of the numbers, with further study, are wrong, that shortcuts were made in producing the numbers. It requires interviews and further document study to get a more complete picture of the “hidden costs and hidden revenues” depicted in financial statements.
Gephart’s (1988: 20-21) “expansion analysis” is similar to
Savall/Zardet’s idea of “hidden cost and revenue analysis.” Both are ways to recover the contextuality of
statistics as well as the social construction of causal interpretations of
numbers. The numbers reported by accountants to managers, for example,
oftentimes do not recount the technical aspects of how the statistics were
produced. Transparency becomes difficult
as there are many intermediaries in collection methods between customers,
communities, vendors, workers, investors, regulators, engineers, and many
others. In some (dysfunctional) settings, such as the recent scandals with
Enron and Global Crossing in the
Accountants, to continue this example, are socialized to produce and display statistics in ways that may not always match the decision makers’ needs for information. Environmental indicators, to list another example, may not be available, since numbers are not being collected that could represent processes that are toxic or wasteful. There is selectivity in what measures, peoples, constructs, and processes get selected for expression in charts of accounts.
Statistics can manipulate raw data in ways that invent distorted images of human and organizational performance. This is what Savall/Zardet refer to as the hidden costs and hidden revenues, which they hope to recover through qualimetrics. The users and the producers of statistics at work have differing interpretations and assumptions. This in ethnostatistics is the ‘social context’ of measurement and the limits of particular quantitative practices and procedures (Gephart, 1988: 29).
Qualimetrics is defined, then, as an intervention into the way in which numbers are produced, analyzed, displayed, and interpreted. The aim is to be more sensitive to the socio-economic context of measurement and the (dysfunctional) outcomes of how numbers are produced, analyzed and interpreted.
While it is assumed that the researcher has technical knowledge in measurement, the practitioner has technical knowledge and experience in understanding how numbers are being generated and used. Together they can coproduce more or less sense, yet miss the embedded context of the game. For example, Fairclough (2003b) looks at how discourse imposes a new order, which achieves its reflexive process by imposing new representations of the world, such as in the ‘new economy’ discourses.
The capitalist mode of production is historically distinctive not only for its crisis-tendencies but also for its capacities to periodically renew the bases of its economic expansion and, in so doing, to re-articulate and re-scale the relations between the economic, political, and social (Fairclough, 2003b: 1).
Statistics is partly rhetorical, part of the business of impression management practices; it accomplishes its re-articulation through changes in discursive practices. Indicators and databases are part of creating an impression that is inherently strategic. In interventions and in practice, qualimetrics, like ethnomethod, refers to the interpretation and construction used in write ups, such as tropes, metaphors, metonym, synecdoche, irony, and storytelling (Gephart, 1988: 49). And qualimetrics can be a way of looking at how capitalism re-articulates and re-scales relations between social and economic (Fairclough, 2003b). This is because qualimetrics extends qualitative and quantitative practices into their financial implications.
Qualimetrics as intervention also recontextualizes how numbers are produced and read, exploring the subjectivity of their construction, the selectivity of operations applied and not applied, ways in which some content is quantified, while other content is ignored. The hidden costs and hidden revenues that result from manager and worker action are recovered in audits and investigations involving interviews, document study, and observation.
Gephart uses the term “next stepping,” how researchers reference older research (journal and books) to establish their own contribution. Similarly, in qualimetrics, the ways in which changes are a next step to prior action can be researched. Actors in organizations constitute a collective memory, which next steps upon what gets documented in written and oral resources.
Accounting and reporting practices are indexical to meaning. Indexical is a word from ethnomethodology (Garfinkel, 1967), which means to recontextualize the meaning of words, numbers, or actions. Action, for example, is meaningful in context, but aspects of that context (e.g. socio-economic) are emergent. Meaning is bound to the context of social action. Savall/Zardet (like Gephart) are tracing and deconstructing the construction, interpretive, and rhetorical steps of the statistical producers and users. Interpretations are coproduced by the interactions of researchers and practitioners, their actions are meaningful in the particularities of context. It is through our research and intervention work that reality gets constructed.
Social and economic (socio-economic) contexts are dynamic and changing. They are emergent, both constructing and deconstructing situations of interpretation. Stories amplify not only qualitative but quantitative context; variables rip away context needed to interpret stories; qualimetrics traces the bridging that is in the arrival.
Qualimetrics also looks at how cost and revenue production gets fudged (manipulated), ‘arriving’ in reductionistic and selective ways. There is also a fudging of the interpretation accomplished by the collectivity. There are systemic ways in which entropy occurs, ways in which the processes of numeric production and interpretation are self-deconstructing, becoming less or more than the context.
Qualimetrics cuts across disciplinary boundaries; it is interdisciplinary and intertextual. This intertextuality and interdisciplinaryness is also entropic, and self-deconstructing processes rival constructing ones. A great deal of absurdity, nonsense, and illusory attribution inheres in accounting, performance, quality, and other reporting systems. What on the surface is represented as entirely rigorous, after more grounded research, turns out to be a highly subjective process. Qualimetrics questions the signature-story, the story of the firm reflected in its financial and performance accounting. Qualimetrics traces the ways in which numbers torture, fragment, abbreviate, and invent stories. Qualimetrics sets many alternatives stories against the dominant story of the firm, and in this way intervention researchers and participants coproduce a new story, one bridging qualitative and quantitative practices.
I see parallels in Savall and Zardet’s work and that of Kenneth Burke. In Burke's Attitudes Toward History (1937/1957), for example, the "Law of Excluded Middle" is developed. Burke's approach involves looking at the terminological clusters on both sides of an ongoing dialectic (1937/1957: 232-236). In qualimetrics the “excluded middle” is a trilectic, the terminological clusters between qualitative, quantitative, and financial.
Savall and Zardet are not proposing a “symbolic merger” of qualitative, quantitative, and financial. Nor, is it a compromise. Rather, by charting clusters, they trace trilectic clues to important ways in which one deepens and tortures the others. Three opposed logic systems (qualitative, quantitative, & financial) can be true simultaneously. What matters here is their contextualization and interaction.
Fairclough’s (2003a, emphasis mine) work on ‘the language in the new capitalism’ applies, as well, to Savall/Zardet’s work on the socio-economic:
The common idea of new capitalism as a ‘knowledge-based’ or ‘knowledge-driven’ socio-economic order implies that it is also ‘discourse-driven’, suggesting that language may have a more significant role in contemporary socio-economic changes than it has had in the past.
If this is so, discourse analysis has an important contribution to make to Savall/Zardet’s socio-economic research on the transformations of organizations.
Since dialectic and discourse-analysis are many-faceted concepts, I will try to be more precise. The point of dialectic (or trilectic) logic and thinking is to break out of the coercive and hegemonic aspects of dualistic-logic, as it is registered by discourse analysis. Merely juxtaposing point and counter-point (quantitative, qualitative, or financial) is not dialectic, it is just another form of dualism. What is important is not the synthesis, but the context of the trilectic.
Adrian Carr (2000a), for example, talks of the triad of thesis-antithesis, and synthesis. “Each triad represents a process wherein the synthesis absorbs and completes the two prior terms, following which the entire triad is absorbed into the next higher process" (Carr, 2000a: 213). Qualimetrics traces important interplay between the triad and the higher-order processes that construct the new working reality of arriving. The word “arriving” has special meaning, the dynamic interplay, that never becomes static. In deconstruction and critical discourse work, this is the “resituation” of the duality beyond its situated-hierarchic hegemony (Boje, 2001). This means that neither qualitative, quantitative, nor financial is put at the top of a hierarchy of knowledge.
Therefore, my reading of qualimetrics is that it changes the rules of the research game (the logic that embeds actors in their social and economic context). This is done through long-term intervention research that keeps integrating financial with quantitative and qualitative traces of how information is produced and distributed, and how systems of reporting distort the processes they represent. The implications of Savall/Zardet’s work is that researchers need to be trained in multiple methodologies and in ways to keep expanding the nexus of qualitative, quantitative, and financial epistemology. In practice, the implication for management science is that practitioners have to wrestle with this problem. The purpose of this book, by my reading, is to help management and other social scientists, as well as practitioners, to work between and beyond methodologies.
Boje, D. M. (2001) Narrative Methods for Organizational
and Communication Research.
Burke, Kenneth (1937/1957). Attitudes towards history.
Derrida, Jacques (1991). A Derrida Reader: Between the
Blinds. Peggy Kamup (Ed.). NY:
Fairclough, Norman (2003a). Critical Discourse Analysis in
Researching Language in the New Capitalism: Overdetermination,
Transdisciplinarity and Textual Analysis.
Fairclough, Norman (2003b). Representations
of change in neo-liberal discourse.
Garfinkel, Harold (1967). Studies in Ethnomethodology. Cambridge: Polity
Gephart, Robert Jr. (1988). Ethnostatistics: Qualitative
Foundations for Qualtitative Research. Qualitative Research Methods Series #12.
Savall, Henri (2003). Socio-economic approach to management. Special issue of the Journal of Organizational Change Management. Vol. 16 (1): 1 – 128.