AI is the topic of the moment. In companies and in our daily lives. In tomorrow’s wars (and already today’s) too.
Some dream of getting rich quickly, others fear for their jobs. And we can all question what it means for our freedoms and – in our streets, or even on the battlefield – the risks to our lives. As well as the truth of what we read and the images we see, now that generative AI is here.
In companies, leaders are asking themselves about productivity gains, about the future of certain jobs, about their room for manoeuvre and decision in the face of what “big data” brings: those astronomical quantities of data now analysed with these powerful tools. And they are concerned about how their teams will adopt AI, as with any major change they want or that seems inevitable to them, but which simply cannot be imposed.
But what is not explicitly addressed – and should be taken far more seriously by everyone who has a say in acquiring and using these tools – is the question of decision-makers’ power. And of their responsibility. Decision-makers who sometimes, without thinking, agree to give up this power and neglect their responsibility. Most often for lack of time to think about it. Sometimes, unfortunately, because it is easier.
The power of suppliers, or omniscience at the click of a button
As at every stage of the “digital transformation”, companies and their leaders are bombarded with miracle promises from suppliers of products and services. A classic, with broken promises and sometimes abuses, which over the decades have brought a certain discredit on consulting – or rather, one should say, on service provision.
Last century we had ERP systems, which promised to put all the data about their company into the hands of leaders alone, in order to guarantee its good “governance”: finance, human resources, procurement… Everything would be steerable at a single glance.
Later, to address the eternal challenges of internal communication and social cohesion, we were sold enterprise social networks: digital platforms that would make it possible to animate communities, contribute to collective intelligence and social dialogue… while doing away with managerial leadership, exchanges between peers, and even the coffee machine…
We also saw a whole range of videoconferencing and real-time data-sharing tools flourish, with explosive growth, as a technological answer to the temporary needs of the pandemic lockdowns and, more generally, to the never-ending search for savings on property costs. They too would guarantee productivity while reducing physical interactions, seen as so much “friction” – and therefore so much wasted energy and resources.
In industry, construction and healthcare, data sensors have multiplied to enable preventive maintenance of machines and buildings and to improve people’s health. The aim was – and still is – to create “digital twins”. But the consequence of this proliferation of sensors and collected data was the creation of unfathomable and unusable “data lakes”.
These innovations, and so many others, were promises only partly kept, playing on fantasies of omniscience or even omnipotence – or simply on passing fashions.
The point here is not to reject the new industrial and societal revolution brought about by the arrival of digital technology in our lives, but only to criticise its false promises, which, through sheer fascination, lead people not to ask the right questions.
Today, the computing power of machines and the development of tools for analysis, statistical processing and even “reasoning” provide astonishing answers to a great many situations.
Listening to the promises – since today’s achievements are nothing compared with what tomorrow will bring – and faced with the abyss this new situation opens up, we are running… To chase illusions, or so as not to be left behind?
Whatever the answer, when allocating resources that are by nature limited, or making irreversible choices, decision-makers should take the time to stop for a moment. And do what they are paid for: think before deciding.
AI, a new FOMO?
A sense of urgency, as we know from the online scams we are all targeted by, is one of the most effective levers for selling a solution, whether technological or human.
It is in fact one of the recommendations of John Kotter, one of the “gurus” of change management – more often quoted by non-practitioners than by those whose expertise and practice draw on broader foundations, and who see the limits and excesses of his recommendations every day. Trained in the best practices of American business management, and therefore above all in selling, he knew how to formulate memorable “recipes” that are still recommended today. In particular: creating a sense of urgency is, in his view, an essential condition for success – note the word “sense”, rather than reality. FOMO (“fear of missing out”) before the term existed.
This sense of urgency is one of the keys used by sellers of solutions and services, who use it to promote virtuous change and a bright future. Solutions and services indeed, because IT services companies thrive both on selling solutions whose development costs have often already been recovered, and also, above all, on services dedicated to getting those same tools adopted. Because if the magic tool is not easily adopted, it must be because of that wretched human factor… which has to be trained and convinced through large-scale managerial communication campaigns. Not because the tool, being imperfect, will need adjustments, time, and arguments like those of the tailor in Fernand Raynaud’s famous French comedy sketch, who insists that a badly cut suit fits perfectly if only the customer would stand differently…
A great classic, then.
Handing decision-making power to a black box?
With AI, however, there is a new dimension that goes beyond the tools.
Whereas the previous generation promised you a panoptic view of all your company’s indicators to help you decide, the new solutions offer to help you decide by taking the decision themselves – or at least some of the decisions.
It is no longer humanly possible to cope with the avalanche of data captured in your machines and across your company. All the more so as it is now supplemented by data from your environment – physical, informational, cognitive. Because, so it seems, to decide you need to know everything… A senseless quest, certainly, but a fascinating one.
Faced with the proliferation of “indicators” last century, methods and tools had to be developed to sort them: visual management, prioritisation…
Now, since data can be captured anywhere and at any time, because every event must be quantified and computing power grows every day, you are promised that you can give up this frustrating prioritisation (since to choose is to give something up) and finally gain access to exhaustive analysis… No more fog of war, no more business complexity… You become omniscient. And when AI really acts in the physical world, through the machines it will control, you will be omnipotent…
What is really missing from this promise – and it is one of the blind spots in the adoption of AI by individuals and organisations – is that in reality you do not know on what basis this analysis is carried out…
What are the selection criteria? And what are the “possible flaws”?
The safety sciences have long shown that every programmed system has flaws… Either from the outset, or emerging over time. And these flaws do not always come down to the “human factor” alone – that ideal culprit which technologies used for social, or even cognitive, control might also aim to “tame” more effectively.
In reality, as everyday events in our socio-technical environments prove, there is no such thing as a perfect system. That is why a system must be resilient even more than resistant. What matters is not being infallible and indestructible, but continuing to function and deliver the expected services, despite the inevitable failures and accidents.
If this statement shocks you, have you never wondered why our digital systems are “updated” so regularly? And for reasons that are only very superficially explained?
It is therefore essential that suppliers of products and services are explicit about how their “black box” works… assuming they know themselves.
And above all that decision-makers think carefully about this transfer of their power to extensions they do not really understand… A transfer of power, or even a transfer of sovereignty, at a time when that word is increasingly used.
This question of principle, almost a philosophical one, can be translated into more concrete questions:
- Your organisation is now deploying AI tools that you did not build, that you cannot inspect, and whose updates can change the behaviours on which entire processes have been built. Without notice, without negotiation, sometimes without your knowledge.
- Who in your organisation has actually read the model providers’ terms and conditions?
- Who is able to audit what these tools do with your production data?
- What reversibility clause have you negotiated in case the supplier changes its pricing model or ceases to exist?
These questions are not trivial. They define the real degree of your decision-making autonomy, of your strategic power. And in the vast majority of cases, the honest answer is: no one, no one, none.
This is not a criticism, it is an observation. AI providers enjoy a considerable information asymmetry, technical expertise that is hard to assess from the outside, and a market position that makes negotiation very unequal – particularly when, because of the size of the company or the skills available, there is no truly qualified person to face the seller. At least to ask the right questions.
So, to sum up this transfer of power, the central question is not “which tool should we deploy?” but “what are we willing to give up in order to use it?”.
It is high time – as collectively as possible, but also as precisely as possible, and therefore for each company and each organisation, since each one is different – to take the time to formulate the answers.
The end of the managerial relationship, with surveillance that does not speak its name?
By promising a way to process the data lost in the “data lakes” I discovered in industrial companies that were embarking on digital twins more than ten years ago, AI revives this quest for omniscience – the promise that comes before the omnipotence of a “perfect” creator.
But as so often, this omniscience has a dark side.
Dreaming of controlling machines is one thing. But in our world made up of humans – and because these humans are always all too human, and therefore “imperfect” – some have not given up the post-Taylorist fantasy of turning human operators into the perfect cogs of a well-oiled machine.
In the field of industrial safety – and particularly in the most sensitive and dangerous areas of our world, where people’s lives are at stake – the “human factor” has long been regarded, and is still regarded by too many, as “the weakest link”.
In many fields, some have multiplied defence mechanisms, rules and procedures, to the point of paralysis and suffocation… While others have chosen the new approaches to safety, which rely precisely on the “human factor” as a pillar of organisational resilience.
With AI comes the potential for greater control over those who carry out the work: all it takes is to analyse the data, all the data. The data produced, the data captured… from the speed of a mechanical gesture to the force used to strike a keyboard.
And more commonly: case-processing times, automated quality scores, monitoring of customer interactions, analysis of internal exchanges… This data now exists, it is or can be collected, and it can feed dashboards that managers did not always ask for and that employees are not always aware of.
This panopticon is no longer reserved for home-delivery platforms, which can track a courier to the nearest metre, whereas GPS tracking of lorries used to be the preserve of companies alone.
It is now available to support functions, legal departments, HR teams and sales functions.
The limits that human analysis placed on the monitoring of others are now, with AI, being pushed back…
This “algorithmic management” is therefore poised to take hold in organisations, across every profession. A tool supporting traditional management, or a voluntary or involuntary surrender of power – and of the managerial relationship too?
With AI, will your managers become the executors of decisions taken by systems they do not understand? Some will no doubt be happy about it, if they were not willing, or not prepared, to take on these delicate roles that call for tact, courage and relationship skills…
But is that what you expect of them? And is it what you expect of yourself?
Giving up decision-making power: the algorithm as an alibi
This is not a new phenomenon. Some “leaders” sometimes call on “external authorities” to justify decisions they have already taken, and which they have no intention of adjusting or supporting. These “authorities” may take the face – or rather the brand – of consulting firms.
Relying on an outside view to confirm an intuition is one thing. But one must also be prepared, if necessary, to adjust it (the “lateral project” strategy), or even to abandon it.
Asking for cosmetic support is a manipulative act, a waste of time and of human and financial resources, and a form of cowardice in the face of adversity… An adversity that will reveal itself anyway, if the expert support that was requested identified it and proposed possible solutions that were then dismissed out of pride.
This managerial and decision-making cowardice may be amplified by the use of AI, since AI is reputed to be infallible – despite the warnings of the most cautious and most honest, and despite the lessons of industrial safety and of the “accidents” that punctuate and shape life in our world, which show that there is never such a thing as an infallible system.
AI will be able to dress up decisions already taken with an appearance of objectivity. An executive committee will be shown an algorithmic recommendation that “confirms” the desired direction. An automated score will be used to justify a reorganisation that was going to happen anyway. The model will be left to “decide” so as to avoid owning a difficult decision.
Once again: this is not new. It was a temptation, sometimes a drift, that relied on third parties. AI will industrialise and accelerate this drift.
But it will also make it less visible, and therefore more dangerous. Because the algorithm has no face, no apparent interest, no declared bias. On the face of it.
This is probably the most subtle, and perhaps the most dangerous, dynamic of this transfer of decision-making and managerial power – or even its abandonment – to algorithms.
Giving up the power of social dialogue too
This silent erosion of managerial power and responsibility will also show in the quality of social dialogue – well beyond its formal bodies and procedures.
When we speak of power, we naturally think of “power over”. The power Max Weber described more than a century ago: that of people, organisations, norms…
But we too often overlook the fact that power is also what Hannah Arendt described: “power with”… For the worse, admittedly, but also, fortunately, for the better.
If decisions are taken without anyone really owning them, it will soon show, and trust within the social body will erode. Not necessarily as a consequence of rejecting automated “social control”… but quite simply because “power with” means leaving room for the conversations that connect people, and that slip into the uncertainties, the imprecisions, the misunderstandings. Ways of working that require time to build and maintain mutual trust.
And beyond this risk of a loss of trust, there will also inevitably be the risk of distrust: well beyond the absence of a relationship, confrontation. Because that is the natural evolution of mutual ignorance.
So, as far as AI is concerned: have your employees been consulted, informed and involved in the introduction of these tools? And in the changes this will bring to social dialogue, in all its dimensions (formal and informal)?
And rather than seeing the need to discuss it as a constraint, remember that resistance is not all you can expect from their contributions. As always, collective intelligence can usefully enrich a change project, and fostering it can secure, accelerate and sustain its implementation.
It is therefore high time to anticipate these questions in order to structure the new way your teams will be led, rather than having to put up with whatever emerges spontaneously, with the inevitable tensions that will arise.
And losing the power to improve collectively
In the logic of “the human factor as the weakest link”, this substitution of the machine for human decision-making power will also lead companies to lose their ability to learn from their mistakes.
Because of the belief in an infallible technical system, errors – and even more so accidents – will be attributed by default, even more than they are today, to the human factor as the culprit, rather than to an emergent property of the complex socio-technical system formed by AI and humans together.
This capacity to learn is an essential asset for companies and organisations facing complex environments.
In a mechanistic, predictable environment, you can build walls (and even then, it is not always enough).
But when the environment is complex and shifting, as today’s world is, you need adaptive systems. Systems and approaches that are not based on hunting down faults, but rather on what enables resilience and improvement.
Keeping the ability to understand what is happening (“situational awareness”). But above all keeping the power of individual and collective initiative. To cope with the unexpected, as the military do with “mission command”, which leaves people on the ground the responsibility and the ability to decide and act. And to innovate, always.
Looking for new culprits, or working better together?
It took the safety sciences a long time to convince organisations that an accident is never the fault of a single operator. They continue to make progress, despite the permanent temptation to look for culprits. And it must be acknowledged that industrial systems, although far more complex than they were decades ago, have also become much safer.
I am betting that we can do the same with AI in organisations. Here again, the idea is not to point the finger: at greedy suppliers, at IT departments tempted by this transfer of power, at managers overwhelmed by tools that replace them, at teams that are incompetent or reluctant to change…
But to look clearly at the effects these transformations can produce. And to ensure that human decision-makers keep the means to truly steer them, by making them their own.
This is not an anti-AI argument, but a plea for human organisations and their decision-makers.
One that I offer to help you put into practice, concretely, in your organisations.
Leave a Reply