Chaos and Complexity

Complexity theory and management practice – Jonathan Rosenhead

 

Concepts:

 

Management, complexity, Stacey, complexity theory, chaos, science, patterns, findings, common, practice, creativity, planning, strange attractors, evidence, equilibrium.

 

 

Summary:

 

There is a growing popular literature on chaos and complexity (e.g. Stewart 1989; Kauffman 1993) authored by scientists of high reputation writing about research fields in which they are themselves active.  There is also a burgeoning literature (e.g. Wheatley 1992; Stacey 1992, 1996; McMaster 1995, Merry 1995) which draws on this work to address management concerns and practices, but whose authors are experienced in management and management education rather than in the substantive scientific fields whose findings they report and interpret.

 

There is some evidence of managerial take-up of 'complexity' as a framework for informing organisational practice.  How novel are the management prescriptions which are derived from complexity theory?  The systems of interest to complexity theory, under certain conditions, perform in regular, predictable ways; under other conditions they exhibit behaviour in which regularity and predictability is lost.

 

The most graphic example of this is the oft-quoted assertion that the flapping of a butterfly's wing can in due course decisively affect weather on a global scale.  That is, we can take some quite simple (and entirely deterministic) equations, compute the values of some variables of interest repetitively using the outputs of any stage of the calculation as the input to the next, and get results which skip around as the calculation proceeds.

 

The indeterminate meanderings of these systems, plotted over time, show that there is a pattern to the movements.  Such a pattern of trajectories (and a whole range of different ones have been identified by trying out interesting ideas in the branch of mathematics called topology) is called a strange attractor.

 

Complexity itself is centre-stage, rather than an emergent property of research in particular disciplines, at the Santa Fe Institute.  Such accounts include the occasional straw person set up the more resoundingly to be knocked over, but there are also palpable hits on juicy targets.

 

Given that the key finding claimed for complexity theory is the effective unknowability of the future, the common assumption among managers that part of their job is to decide where the organisation is going, and to take decisions designed to get it there is seen as a dangerous delusion.

 

Yet agility of thought based on the fostering of diversity is a prerequisite for the organisation's longer-term success.  For an organisation to seek stable equilibrium relationships with an environment which is inherently unpredictable is bound to lead to failure.  Even the dominant 1980's approach to strategy, which distanced itself so emphatically from the strategic planning paradigm of preceding decades, at base maintained the aim of strategic management as the realisation of prior intent.

 

Rather than trying to consolidate stable equilibrium, the organisation should aim to position itself in a region of bounded instability, to seek the edge of chaos.  Instead of a perfectly planned corporate death, the released creativity leads to an organisation which continuously re-invents itself.

 

If you accept the relevance of complexity theory to the managerial condition, then you must also accept the package of systemic categorical imperatives which are embedded in it.  Competent ordinary management is necessary if the organisation is to deliver cost-effective performance.  Extraordinary management, by contrast, is what is required if the organisation is to be able to transform itself in situations of open-ended change.

 

In the necessary absence of hard evidence, arguments in favour of new assumptions and directions will be analogical and intuitive, and the process of decision making will be political as champions attempt to persuade others to their point of view.  Stacey does not propose that ordinary management should drive out extraordinary management.  For him rationality is fine, and necessary, for handling routine business, but is just not up to the job of sense-making in poorly structured situations.

 

In their demotion of pre-existing systems insights, and in a number of other respects, they rest explicitly on the authority of science.  We should start with the least problematic element -- the solidity of those natural science results in their own domains.  Stewart (1989) provides a good source of such examples -- the weather (of course), ecological cycles, fluid dynamics, chemical clocks...

 

What follows from this is that complexity theory is a field within which some surprising and diverse results have been found, leading on to some further intriguing conjectures.  Typically some simple laws of behaviour and interaction are postulated, and the computer is used to see how the operations of these laws would translate into long-term development or macro-behaviour.

 

For example Kauffman (1993, 1995) models how an organism might evolve through an 'adaptive walk' of mutations across available alternatives, depending on the degree of cross-coupling of the organism's component parts.  Krugman (1996) shows how aggregate patterns of land use (eg the formation of multiple business districts, racial segregation) could result from individual responses to purely local conditions.

 

It can be seen from this that scientific authority is an unsafe ground for asserting that specific results from complexity theory necessarily apply to organisations, or that complexity-based lessons constitute imperatives for management practice.  Both tendencies can be amply illustrated from a single work -- Stacey (1992).

 

Whether we get chaos or not depends critically not only on the form of the equation but also on the parameters defining the strength of the feedback loops.  These severe limitations on the practical relevance of chaos/complexity have evidently not deterred the rather inclusive claims made for its sphere of operation.  This is the route of metaphor or analogy.  This account of attempts to apply ideas from complexity theory to management practice has been broadly critical -- critical of claims for the authoritative status of what would be better presented as stimulating metaphors.

 

Unnecessary, because management complexity has indeed generated metaphorically based insights which are novel and instructive.  Long-term planning has taken such a battering that the complexity-based view that it is impossible anyway can hardly classify as startling.  He makes a good case (Stacey 1992, pp 126-144) that a single vision to serve as intended organisational future, motivator of behaviour and guarantor of corporate cohesion is a thoroughly bad idea.

 

I will highlight just one of these here -- the notion that organisations 'should' operate at the edge of chaos.  This has meant that the natural science results from which management complexity theory might borrow have in effect been accumulating only since the 1960's.  The full title of Darwin's great work was "On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life".

 

Against this background he published in 1798 his influential essay "On the Principle of Population" in which he argued that, since population tends to increase geometrically, but food supplies can grow only linearly, increasing population would always tend to outstrip the means of subsistence.

 

The past 20 years have not only seen the mushrooming of interest in complexity theory.  They have also been years in which a hegemonic common wisdom has developed, in favour of the market rather than planning as the way to order our affairs; in which the demise of the Soviet Union has consolidated this victory in geo-political terms; in which mass production of standard products has been replaced by a Post-Fordist fluidity in promoting and responding to changes in taste and fashion; in which influential thinkers have mounted an assault on the pre-eminence of reason in social decision-making (see Rosenhead 1992); and in which postmodernism has made good progress in sweeping such notions as 'progress' off the agenda of respectable discourse.

 

There is a precedent for this process in the response to Darwin, which was not only scientific and theological in character; his work also provoked an influential current of social theory. Two hundred years after Smith, in a period when there is once again an almost mystical belief in the beneficial properties of market forces, we are told on the authority of science that non-intervention is best.  The general posture seems to be that managers who realise the importance of creativity in a world of complexity will have concerns for which analysis is an irrelevance.

 

The choice, the reader is invited to believe, is between this, and a more fluid, creative and political process.  For he presents a powerful argument (Stacey 1992, p 101-2) for the inadequacy of informal mental models -- the same case incidentally that operational research advanced thirty or so years earlier -- only to leave managers with no tools to supplement them.  On the handling of uncertainty, Stacey thinks that this can only be done by predicting the future, which complexity theory claims to be meaningfully impossible.

 

Yet scenario planning is precisely predicated on this proposition.  This common ground will often provide scope for the use of one or more of the family of problem structuring methods (Rosenhead 1989a, 1996).  Aids to rational discourse on organisational issues fit awkwardly within the world-view promoted by management complexity authors.  Indeed they provide rational arguments against rationality, as well as forecasting with great confidence the impossibility of forecasting, and planning for the absence of planning.