Abstract
This work was written in Turkish. The summary on this page is a translation; the citation gives the original title in parentheses.
Ever since I was a child, my greatest interest has been space travel and, in connection with it, science fiction novels and stories. I think it would not be wrong to say that this interest has directly influenced my fields of work and my career. It was entirely this curiosity that led me to work first in IT law and then in the field of artificial intelligence. For me, reading what has been written in these fields and reflecting on it is, beyond work, an effort to satisfy an inexhaustible curiosity. Above all, I cannot put into words the pleasure of seeing that yesterday’s science fiction has become today’s reality. I deliberately chose to call this article a “law-fiction essay” in its title.
For what I am about to set out is partly law, partly science and partly science fiction. Moreover, there are as yet no definite conclusions or judgments in this field; there are more questions than answers, which is why the title of the piece says “essay”. To avoid misunderstandings, let me say this at the outset: what is written in the field of science fiction is a prediction of the future on the basis of scientific data, usually told within the framework of a story. Yesterday’s science fiction story is therefore today’s reality. The clearest proof of this is the stories of the famous writer Jules Verne. Starting from the scientific discoveries of his age, he predicted the journey to the moon and submarine voyages, which lay beyond his age; these are familiar events of our lives today. And a new name is about to join these writers: Isaac Asimov.
Full text
This text is a translation of the Turkish original, prepared for this website. Numbers in square brackets are the page numbers of the original; sources in the footnotes are given as in the original. For citation, the original publication (PDF) is authoritative.
Assoc. Prof. Dr. Murat Volkan DÜLGER, Academic/Attorney
A Law-Fiction Essay:
THE LEGAL RESPONSIBILITY OF THE ARTIFICIALLY INTELLIGENT BEING (CAN THERE BE ONE?)
“IF AN ELDERLY AND DISTINGUISHED SCIENTIST SAYS THAT SOMETHING IS POSSIBLE, HE IS ALMOST CERTAINLY Right. WHEN HE SAYS THAT IT IS NOT POSSIBLE,
Ever since I was a child, my greatest curiosity has been space travel and, connected with it, science-fiction novels and stories. I do not think it would be wrong to say that this interest has directly shaped my fields of work and my career. It was this curiosity alone that drove me to work first in IT law and then in the field of artificial intelligence.
For me, reading what is written on these fields and reflecting on it is more than work; it is the effort to satisfy a curiosity
that never runs dry. And I cannot put into words the pleasure of seeing that yesterday's science fiction is today's reality.
I deliberately chose to call this article a “law-fiction essay”. For what I am about to relate is partly law, partly science and partly science fiction. Moreover, there are as yet no definite conclusions or judgments in this field; there are more questions than answers, which is why the title speaks of an “essay”.
To dispel misunderstandings, let me say at the outset: what is written in the field of science fiction is a forecast of the future made on the basis of scientific data and usually told within the framework of a story. Hence yesterday's science-fiction story is today's reality. The most concrete proof of this is the stories of the famous writer Jules Verne. Starting from the scientific discoveries of his age,
HE IS VERY PROBABLY Wrong.”
*ARTHUR C. CLARKE
he foresaw the journey to the moon and voyages by submarine, which lay beyond his age and are familiar occurrences of our life today. A new name is now about to join these writers: Isaac Asimov. • Asimov, who lived between 1920 and 1992 and was both a professor of biochemistry and a science-fiction writer, was the first person to write about robots, what they might be able to do and the moral consequences of this, in his book I Robot (Ben Robot), first published in 1950. Asimov is one of the most iconic authors of science fiction and popular science writing. In this book the author also wrote the three laws on robots that are today regarded as the foundation of this field. According to these, the three basic laws that robots (possessing artificial intelligence) must obey are as follows: (1) A robot may not harm a human being, nor may it, even by omission, allow a human being to come to harm; (2) A robot must obey the orders given to it by human beings, except where such orders conflict with the first rule; (3) A robot must protect its own existence, as long as this does not conflict with the first or second rule.
What Asimov wrote was something whose realisation could not even be dreamt of, not merely in the 1950s but even in the mid-1980s, when I read these stories. In fact, the inventions that would make what he wrote come true were being made in precisely those years. Today we refer to all of these inventions in general as the “information revolution”. The invention of the digital computer, the constant, exponential growth of processing and storage capacities (Moore's law), the increase in their functions despite their ever-shrinking size, and wireless connection to the internet paved the way for these developments. Today we cannot live without information systems and what they have brought to our daily lives. In fact, even what I have described above has in a sense become history; for very different topics are now being discussed in the IT sector and their applications debated: Technology 4.0, the internet of things, artificial intelligence and so on.
Today, in many sectors, production takes place in factories equipped with devices that run on information technology; one can connect from a mobile phone to a car or to the oven at home, give commands and obtain information; a semi-autonomous car can park itself. Advertisements on subjects that interest us arrive constantly on our mobile phones. When we leave home, information on the traffic on the route we use to go to work, and on how many minutes it will take us to arrive, reaches our phone before we have even asked. In fact, without noticing it, we are all becoming part of this new world.
All of these concepts are interconnected; but the one that fascinates me most is artificial intelligence. The concept of artificial intelligence encompasses, alongside information technologies, many other fields such as philosophy (ontology/the theory of being), ethics and law. It is precisely at this point that we now see what Asimov wrote on the verge of coming true. Today we see that autonomous or semi-autonomous devices and software are used in almost every field, from the automotive industry to medicine, from the military to science, from banking and finance to computer games and even to law, and that artificial intelligences, albeit of limited capacity, are coming into play. Average estimates on this subject are that artificial intelligence software and artificially intelligent robots will have entered every area of our lives before the year 2050.
The first step that must be taken in order to understand artificial intelligences and to discuss whether they can bear legal responsibility is to define what intelligence is. Unless it is established what an intelligent
being is or, in other words, unless it is determined which characteristics allow a being to be described as intelligent, it is not possible to call a designed being intelligent.
An important point to be stressed before setting out these characteristics is that, for some years now,
there has been a notable debate as to whether artificially intelligent beings can really exist.
Futurologists are announcing the birth of a new species which, as an intelligent creature,
“UNLESS IT IS ESTABLISHED WHAT AN Intelligent Being IS OR, IN OTHER WORDS, UNLESS IT IS DETERMINED WHICH CHARACTERISTICS ALLOW A BEING TO BE DESCRIBED AS Intelligent, IT IS NOT POSSIBLE TO CALL A DESIGNED BEING Intelligent.”
takes its place on earth alongside man: “Machina sapiens”. Those who criticise this view, on the other hand, argue that a thinking machine is a contradiction. According to them, machines, including computers, with the cold logic that is their basic building block, will never be able to possess insight or creativity as human beings do. This debate is conducted around a number of fundamental questions concerning the existence of humanity and of artificial intelligences: Can thinking machines
function like human beings? Can there be thinking machines?
The five characteristics that an intelligent being must have (I use the word intelligence here to mean the average level to be expected in beings of normal reason) are as follows. First, they must communicate. It must be possible for any person to communicate with the intelligent being. The easier it is to communicate with a being, the more intelligent it appears. One can communicate with a dog, but one cannot discuss Einstein's general theory of relativity with it. One can communicate with a small child about Einstein's theory, but in order to discuss it the child must grasp the subject. The second characteristic is that it must have internal knowledge. An intelligent being must have some knowledge of its own existence. The third characteristic is that it must have external knowledge. An intelligent being must have knowledge of the world outside it, learn about it and be able to use what it has learnt. The fourth characteristic is the ability to act purposively (teleologically/finalistically) – and in criminal law, too, we define an act as human conduct directed towards a purpose. An intelligent being must be able to act towards the goal it pursues. The fifth characteristic is creativity. An intelligent being must be creative to a certain degree. In this context, creativity is the ability to perform an alternative act where the first act proves mistaken. For example, when a fly tries to get out of a room, it keeps repeating the same movement in vain although it hits the window pane every time. An artificially intelligent robot, by contrast, tries to leave by the door once it has hit the glass. The great majority of artificially intelligent beings possess all five of the characteristics given in this definition. The characteristics possessed by some twenty-first-century types of artificially intelligent beings, moreover, enable them to behave in far more sophisticated ways.
Today there also appears to be confusion as to what should be understood by the concept of artificial intelligence. Everything from machines that can calculate quickly to devices that genuinely • have the ability to learn is described as artificial intelligence.
Yet it is wrong to call these artificial intelligence in the absence of the characteristics set out above. Accordingly, such devices are: devices directed by a human being or by an artificially intelligent being (for example, robotic arms that weld in a factory, or surgical robots that move under the surgeon's direction); semi-autonomous vehicles or software (for example, cars that perform certain movements by themselves, satellites, spacecraft carrying out research on Mars); and, lastly, artificially intelligent beings that act fully autonomously, can learn by themselves and can make choices. One step beyond these are cyborgs, which can produce their own kind and also have biological characteristics.
And perhaps, in a more distant future, artificially intelligent humanoid beings that have acquired a wholly biological structure and are extremely difficult to distinguish from humans (like the biological Cylons in the second version of the series Battlestar Galactica).
The nerve cells of advanced biological organisms, called neurons, work through the passage of electric current; in other words, on top of our bodies there is an extremely advanced biological quantum computer (not a digital one, since it does not work solely with numbers in the binary system). What I understand by an artificially intelligent being, and whose legal responsibility I am discussing, are artificial intelligences that act fully autonomously, can learn by themselves and can make choices, and the beings that lie beyond them.
I must mention one more point here. There are different types of artificial intelligence today.
In fact, the ideas put forward in the 1950s, when artificial intelligences first emerged, and the ideas supported and used today by the companies investing in this field are quite different from one another. These days, where technology is concerned, artificial intelligences are spoken of across a very wide field. Almost every week the technology and business pages of newspapers and magazines carry news about artificial intelligences. While major IT companies such as Microsoft, IBM, Google and Apple
www.hplusdergi.com
invest large sums in the research and development of artificial intelligences, many other companies follow these investments in order to keep pace with this process.
For example, Toyota announced in November 2015 that it was investing one billion dollars in this field, and in December 2015 Elon Musk announced that he had provided a further 1 billion dollars in funding to the non-profit company named OpenAI (Open Artificial Intelligence). What does all this mean?
If one looks more closely, one sees what these millions of dollars are expected to lead to. It is apparent that the technology is now emerging in a form different from the types of artificial intelligence found in the science-fiction books and films of the past. Very little is now said of intelligent humanoid robots (as in the film Blade Runner) or even of high-powered artificial intelligences without a body (like HAL 9000 in the film 2001: A Space Odyssey). In fact, it is more accurate to call the kind of artificial intelligence being developed by technology companies “adaptive machine learning” (AML). AML is, very generally, the creation of computer systems that can learn to perform certain intelligent behaviours by working on very large quantities of data. Indeed, the current motivating factor on which work on artificial intelligence
rests is seen to be, predominantly, the potential applications of big data. The elderly scientist in the words of Arthur C. Clarke quoted at the beginning is in fact reasoning about the feasibility of artificial intelligences and proceeds from a rule concerning scientific progress. If this rule is followed, the likelihood that artificial intelligences will become reality is close to certainty. This is a view that I, too, hold. Nothing we have learnt to date is sufficient to say the opposite. But if the realisation of artificial intelligence is truly possible, why has it not been achieved so far? The year 2001 has come and gone, yet we are still in no respect close to realising the artificially intelligent HAL of the film 2001: A Space Odyssey. There are two basic answers to this question. The first is perfectly plain: we are well behind those who made predictions on this subject (including Marvin Minsky, who acted as adviser on the film 2001), because we have not yet been able to understand exactly what is required. There are still many significant scientific difficulties to be overcome. Beyond that, even if all the scientific difficulties were resolved, there would still be an enormous engineering challenge in making a machine learn enough and behave fully intelligently. The second reason is perhaps more controversial: to date, there has not been sufficient demand for artificial intelligences to bring about their realisation.
Humankind has generally made its great scientific and technological leaps in order to meet the needs created by wars. Perhaps, had the Cold War continued, more progress would have been made in this field (humankind becomes extremely eager and creative when it comes to destroying one another!).
There is no difficulty in determining legal responsibility in respect of the devices in the first and second categories mentioned above (remotely directed and semi-autonomous devices). For those who develop the software or, even where there is no error in the software, those who do not use the device in accordance with the rules are responsible under both private law and criminal law for the act committed, either alone or in proportion to the wrongfulness of their conduct.
Indeed, even where there is no wrong either in the software or in its use, it is possible, from the standpoint of private law, to apply the principle of strict liability. Hence the universal principles of law that we accept today, the theories of the various legal disciplines and the legal provisions currently in force are suitable and sufficient for resolving the legal problems raised by these two categories of devices.
The real problem that needs to be discussed lies in artificial intelligence in the true sense and in artificially intelligent beings; that is, in those that can learn by themselves, take decisions and carry them out.
Although beings of this kind are initially developed, programmed and produced by humans, it is possible for later generations to be programmed and produced by artificially intelligent beings like themselves.
Even though the one who first develops, programs and produces them is human, beings possessing this kind of artificial intelligence will, because they will have the ability to learn and to develop themselves, go beyond the behaviour for which they were initially programmed; indeed, in some situations they will be expected to do so. In other words, once these beings have been programmed in a basic way, they will take their own decisions themselves
and, indeed, to the extent that they develop themselves, the variety of the decisions they take (good or bad) and of their behaviour will increase. It is precisely at this point that the basic question confronts us once again.
Who will be responsible for the decisions taken by these artificial intelligences and for the conduct they engage in pursuant to those decisions? Those who produce the being? Those who use it? The being itself?
Today there is no complete answer to this question. There are many views on the subject, but there is no single right one. There are views holding that those who program artificial intelligences, those who use them, the artificially intelligent beings themselves, or all of these in a mixed form, should be responsible. Frankly, I do not have a fully formed view either. The reason is this:
Our present legal system, which has its sources in Roman law, is inadequate to answer this question.
To explain:
I have stated that a being with artificial intelligence in the full sense must be able to learn and to be
THE REAL PROBLEM THAT NEEDS TO BE DISCUSSED LIES IN Artificial Intelligence IN THE TRUE SENSE AND IN Artificially Intelligent Beings; THAT IS, IN THOSE THAT CAN LEARN BY THEMSELVES, TAKE DECISIONS AND CARRY THEM OUT.
conscious of its own existence, and must therefore also have the ability to develop itself. I also said that we humans, too, are biological computers and that our neurons work by electric current. Human beings start life with the basic codes that come from the genes they receive from their mothers and fathers; that is, they open their eyes to life with an operating system. They then go through a long process of upbringing and education both in the family and at school; that is, application software is installed. In the end they go on with their lives as independent individuals on their own. Now, do we hold people's mothers, fathers or teachers legally – and at times even ethically – responsible for the fact that their genetic codes are bad (for example, that they came into the world with the potential to develop a hereditary mental illness that paves the way for crime), that they are prone to crime because of the bad upbringing they received in the family and/or at school (for example, through constant exposure to violence), or that they commit offences in adult life? The answer is clear: no. Under the legal system we have accepted this is not possible, nor should it be. For everyone must be held responsible for his own wrongful and culpable act. What, then, changes when it comes to artificially intelligent beings that go through similar processes?
What changes is this: in the legal system in force, artificially intelligent beings are not a subject, that is, the holder of a right. They are merely an object, that is, things constituting the subject-matter of a right. Yet these beings represent far more than a table, a cupboard or an armchair. But the “person – thing” distinction bequeathed to us by Roman law leaves us no possibility of placing them anywhere else. The same holds for medical science, which has made incredible advances particularly in artificial fertilisation and embryology. It is possible to act upon cells that are not yet even a foetus but have been fertilised and have begun to divide (and which are the basic building blocks of the human being); yet in the eyes of the law they are merely objects • (and they are not protected by criminal law).
Yet these cells, in which all our basic codes are contained, are the first moment of existence of the human being, who is far more than an object.
As can be seen, the “person – thing” distinction that we have inherited from Roman law, and the consequences attached to it, are inadequate in this branch of science as well.
Accordingly, we need a third, a fifth and even a sixth kind. The content and limits of their rights, obligations and responsibilities must be determined. In short, our classical understanding of law, which has persisted for two thousand-odd years, must now change. Humanity needs a revolution in this field.
The greatest obstacle to this, however, is once again humankind itself. For man has the narcissism to see himself as the most intelligent, and the only intelligent, species in the entire universe. Yet research in this field shows that we are not the only intelligent species, that in the past we lived at the same time alongside many human species, but that somehow (to explain this would take too long and is not our subject) we remained as the only human species.
Moreover, the discoveries made by the Hubble telescope show that there are countless planets in the universe (even here our conceited attitude persists: we look for carbon-based elements for life, as if every living thing necessarily had to resemble us; could there not be methane-based life forms? Why not!). On these planets there may well be
www.hplusdergi.com
countless intelligent life forms that resemble us or do not (in my view, the miracle would be not the existence but the absence of other intelligent beings in the universe; the only problem is one of communication, arising from the vastness of the distances involved). What, then, prevents the existence of other intelligent beings in our own world? What stands in the way of their being both holders of rights and bearers of responsibility? We do, that is, human beings. Do we not constantly speak of universal principles, even in law?
“WHAT, THEN, PREVENTS THE EXISTENCE OF OTHER INTELLIGENT BEINGS IN OUR OWN WORLD? WHAT STANDS IN THE WAY OF THEIR BEING Both Holders of Rights and Bearers of Responsibility? WE DO, THAT IS, HUMAN BEINGS.”
I am very curious: do “strict liability” and “the principles of legality of offences and penalties” apply in the order formed by the intelligent forms of life that exist in the Andromeda galaxy or on the planets of the constellation Orion? Even this word “universality” shows how self-centred we are. As long as we fail to break this, we shall remain extremely far from solving even the problems among ourselves as humanity, let alone the legal responsibility of artificially intelligent beings. Before the question whether artificially intelligent beings can bear legal responsibility, therefore, the questions that must be asked and answered are these: Question (1) Is it possible for artificially intelligent beings to enter our lives as a new species? (In my view, yes.) Question (2) Should they be regarded as subjects or as objects? (In my view this legal system must change, and they should be defined as a different kind and have rights and responsibilities equal to those of humans.) (3) Can artificially intelligent beings bear legal and criminal responsibility? (In my view, of course they can, but only if we regard them as holders of rights and responsibilities on an equal footing with a human being.) (4). Will today's law-fiction ideas one day come true? Will they be phenomena that legal science discusses, that are written about in law books and that form the subject of legal provisions? (I have not the slightest doubt. Whether I shall be able to see it, I am not sure. I hope that in that future what I have written here will not be forgotten, and that the artificially intelligent beings of science of the future will have the courtesy to cite this mortal biological being of the past.)
Assoc. Prof. Dr. Murat Volkan DÜLGER Academic / Attorney volkan.dulger@dulger.av.tr
More publications on this topic
Dülger, Murat Volkan / Çetin, Selin / Aydınlı, Celal Duruhan, “Algorithmic Decision-Making and Data Protection” (Algoritmik Karar Verme ve Veri Koruması), 2020.
Dülger, Murat Volkan, “Artificial Intelligence Technologies and Data Protection Law” (Yapay Zeka Teknolojileri ve Veri Koruma Hukuku), Academia.edu, 29 September 2019.
Dülger, Murat Volkan, “Emotional and Sexual Relations Between AI Entities and Humans: Are Sex Robots Replacing People?” (Yapay Zekalı Varlıklar ve İnsanlar Arasında Duygusal / Cinsel Yakınlaşmalar: İnsanların Yerini Seks Robotları Mı Alıyor?), Hukuk ve Daha Fazlası Dergisi (Law and More), No. 6, June 2018, pp. 64–70.
Dülger, Murat Volkan, “The Reflection of AI Entities in the Legal World: How Should Their Legal Status Be Determined” (Yapay Zekalı Varlıkların Hukuk Dünyasına Yansıması: Bu Varlıkların Hukuki Statüleri Nasıl Belirlenmeli), Academia.edu, 9 March 2018.
Dülger, Murat Volkan, “Today's Artificial Intelligence Technology and the Reality of the ‘Robot Judge / Lawyer’” (Günümüz Yapay Zeka Teknolojisi ve 'Robot Yargıç/Avukat' Gerçeği), Hukuk ve Daha Fazlası Dergisi (Law and More), No. 5, January–March 2018, pp. 4–9.
Dülger, Murat Volkan, “The Use of Three Technologies in Money Laundering: Fintech, Crypto Assets and Artificial Intelligence” (Suç Gelirlerinin Aklanmasında Üç Teknolojinin Kullanılması: Fintek, Kripto Varlıklar ve Yapay Zeka), conference presentation, 2026.

