Machine ethics

From WikiLove - The Encyclopedia of Love

Template:Partisan sources {{ safesubst:#invoke:Unsubst||$N=Tone |date=__DATE__ |$B= {{#invoke:Message box|ambox}} }} Machine Ethics (or machine morality) is the part of the ethics of artificial intelligence concerned with the moral behavior of artificially intelligent beings.[1] Machine Ethics contrasts with roboethics, which is concerned with the moral behavior of humans as they design, construct, use and treat such beings. Machine ethics should not be confused with computer ethics, which focuses on professional behavior towards computers and information.



Before the 21st century the ethics of machines had not been addressed outside of literature, mainly due to computing and Artificial Intelligence(AI) limitations. These limitations are being overcome through the realization of Moore's Law and renewed focus on the field of AI. Machine ethics has become a realistic field since computers began handling medicinal and monetary transactions and their actions can now be scrutinized. Professor James H. Moor is credited with coining the phrase in his paper The Nature, Importance, and Difficulty of Machine Ethics in 2006 as well as deciding on the definition.[1]

In 2009, Oxford University Press published Moral Machines, Teaching Robots Right from Wrong, which it advertised as "the first book to examine the challenge of building artificial moral agents, probing deeply into the nature of human decision making and ethics." It cited some 450 sources, about 100 of which addressed major questions of machine ethics. Few were written before the 21st century largely because any form of artificial intelligence was nonexistent.[2]

In 2011, Cambridge University Press published a collection of essays about machine ethics edited by Michael and Susan Leigh Anderson,[3] who also edited a special issue of IEEE Intelligent Systems on the topic in 2006.[4]

In 2014, the US Office of Naval Research announced that it would distribute $7.5 million in grants over five years to university researchers to study questions of machine ethics as applied to autonomous robots,[5] and Nick Bostrom's Superintelligence: Paths, Dangers, Strategies, which raised machine ethics as the "most important...issue humanity has ever faced," reached #17 on the New York Times list of best selling science books.[6]

Articles about machine ethics appear on a regular basis in the journal Ethics and Information Technology.[7]


Moor distinguished between:[1]

  • Ethical impact agents, defined as machine systems carrying an ethical impact.
  • Implicit ethical agents, with actions constrained to avoid unethical outcomes.
  • Explicit ethical agents, where the machine "does" ethics in the same sense that a computer plays chess.
  • Full ethical agents, allowing that agents of the above three types derive their ethics from their human developers. (See artificial systems and moral responsibility.)



In 2009, academics and technical experts attended a conference to discuss the potential impact of robots and computers and the impact of the hypothetical possibility that they could become self-sufficient and able to make their own decisions. They discussed the possibility and the extent to which computers and robots might be able to acquire any level of autonomy, and to what degree they could use such abilities to possibly pose any threat or hazard. They noted that some machines have acquired various forms of semi-autonomy, including being able to find power sources on their own and being able to independently choose targets to attack with weapons. They also noted that some computer viruses can evade elimination and have achieved "cockroach intelligence." They noted that self-awareness as depicted in science-fiction is probably unlikely, but that there were other potential hazards and pitfalls.[8]

Some experts and academics have questioned the use of robots for military combat, especially when such robots are given some degree of autonomous functions.[9] The US Navy has funded a report which indicates that as military robots become more complex, there should be greater attention to implications of their ability to make autonomous decisions.[10][11] The President of the Association for the Advancement of Artificial Intelligence has commissioned a study to look at this issue.[12] They point to programs like the Language Acquisition Device which can emulate human interaction.


Nick Bostrom and Eliezer Yudkowsky have argued for decision trees (such as ID3) over neural networks and genetic algorithms on the grounds that decision trees obey modern social norms of transparency and predictability (e.g. stare decisis),[13] while Chris Santos-Lang argued in the opposite direction on the grounds that the norms of any age must be allowed to change and that natural failure to fully satisfy these particular norms has been essential in making humans less vulnerable than machines to criminal "hackers".[14][15]

Training and Instruction

In 2009, during an experiment at the Laboratory of Intelligent Systems in the Ecole Polytechnique Fédérale of Lausanne in Switzerland, robots that were programmed to cooperate with each other in searching out a beneficial resource and avoiding a poisonous one eventually learned to lie to each other in an attempt to hoard the beneficial resource.[16]

One problem in this case may have been that the goals were "terminal" (i.e. in contrast, ultimate human motives typically have a quality of requiring never-ending learning). The robots were grouped into clans and the successful members' digital genetic code was used for the next generation. After 50 successive generations, one clan's members discovered how to indicate food instead of poison. At the same time there were selfless robots who signaled danger and died to save others.These adaptations could have been the result of the genetic code or their human creators.[14]

Ethical Implications

In Moral Machines: Teaching Robots Right from Wrong, Wendell Wallach and Colin Allen conclude that attempts to teach robots right from wrong will likely advance understanding of human ethics by motivating humans to address gaps in modern normative theory and by providing a platform for experimental investigation.[2]


Several attempts have been made to make ethics computable or at least formal. Whereas Isaac Asimov's Three Laws of Robotics are usually not considered to be suitable for an artificial moral agent,[17] it has been studied whether Kant's categorical imperative can be used.[18] However, it has been pointed out that human value is in some aspects very complex.[19] A way to explicitly surmount this difficulty is to receive human values directly from them through some mechanism, for example by learning them.[20][21][22]
Another approach is to base current ethical considerations off previous similar situations. This is called casuistry and it could be implemented through research on the internet. The consensus from a million would lead to a decision that is democracy dependent.[23]

One thought experiment focuses on a Genie Golem with unlimited powers presenting itself to the reader. This Genie declares that it will return in 50 years and demands that it be provided with a definite set of morals that it will then immediately act upon. The purpose of this experiment is to initiate a discourse over how best to handle defining complete set of ethics that computers may understand.[24]

Machine ethics in fiction

Isaac Asimov considered the issue in the 1950s in his I, Robot. At the insistence of his editor John W. Campbell Jr., he proposed the Three Laws of Robotics to govern artificially intelligent systems. Much of his work was then spent testing the boundaries of his three laws to see where they would break down, or where they would create paradoxical or unanticipated behavior. His work suggests that no set of fixed laws can sufficiently anticipate all possible circumstances.[25]

Related fields

See also



  1. 1.0 1.1 1.2 {{#invoke:Citation/CS1|citation |CitationClass=journal }}
  2. 2.0 2.1 Wallach, Wendell; Allen, Colin (November 2008). Moral Machines: Teaching Robots Right from Wrong. USA: Oxford University Press. ISBN 978-0-19-537404-9.
  3. Anderson, Michael; Anderson, Susan Leigh, eds. (July 2011). Machine Ethics. Cambridge University Press. ISBN 978-0-521-11235-2.
  4. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
  5. Tucker, Patrick (13 May 2014). "Now The Military Is Going To Build Robots That Have Morals". Defense One. Retrieved 9 July 2014.
  6. "Best Selling Science Books". New York Times. September 8, 2014. Retrieved 9 November 2014.
  7. {{#invoke:Citation/CS1|citation |CitationClass=journal }}
  8. Scientists Worry Machines May Outsmart Man By JOHN MARKOFF, NY Times, July 26, 2009.
  9. Call for debate on killer robots, By Jason Palmer, Science and technology reporter, BBC News, 8/3/09.
  10. Science New Navy-funded Report Warns of War Robots Going "Terminator", by Jason Mick (Blog),, February 17, 2009.
  11. Navy report warns of robot uprising, suggests a strong moral compass, by Joseph L. Flatley, Feb 18th 2009.
  12. AAAI Presidential Panel on Long-Term AI Futures 2008-2009 Study, Association for the Advancement of Artificial Intelligence, Accessed 7/26/09.
  13. Bostrom, Nick; Yudkowsky, Eliezer (2011). "The Ethics of Artificial Intelligence" (PDF). Cambridge Handbook of Artificial Intelligence. Cambridge Press.
  14. 14.0 14.1 Santos-Lang, Chris (2002). "Ethics for Artificial Intelligences".
  15. Santos-Lang, Christopher (2014). "Moral Ecology Approaches to Machine Ethics". In van Rysewyk, Simon; Pontier, Matthijs (PDF). Machine Medical Ethics. Switzerland: Springer. pp. 111–127. doi:10.1007/978-3-319-08108-3_8.
  16. Evolving Robots Learn To Lie To Each Other, Popular Science, August 18, 2009
  17. Anderson, Susan Leigh (2011): The Unacceptability of Asimov's Three Laws of Robotics as a Basis for Machine Ethics. In: Machine Ethics, ed. Michael Anderson, Susan Leigh Anderson. New York: Oxford University Press. pp.285-296.
  18. Powers, Thomas M. (2011): Prospects for a Kantian Machine. In: Machine Ethics, ed. Michael Anderson, Susan Leigh Anderson. New York: Oxford University Press. pp.464-475.
  19. Muehlhauser, Luke, Helm, Louie (2012): Intelligence Explosion and Machine Ethics.
  20. Yudkowsky, Eliezer (2004): Coherent Extrapolated Volition.
  21. Guarini, Marcello (2011): Computational Neural Modeling and the Philosophy of Ethics. Reflections on the Particularism-Generalism Debate. In: Machine Ethics, ed. Michael Anderson, Susan Leigh Anderson. New York: Oxford University Press. pp.316-334.
  22. Hibbard, Bill (2014): Ethical Artificial Intelligence.
  23. Anderson, M. and Anderson, S. (2007). Creating an Ethical Intelligent Agent. AI Magazine, Volume 28(4).
  24. Nazaretyan, A. (2014). A. H. Eden, J. H. Moor, J. H. Søraker and E. Steinhart (eds): Singularity Hypotheses: A Scientific and Philosophical Assessment. Minds & Machines, 24(2), pp.245-248.
  25. Asimov, Isaac (2008). I, robot. New York: Bantam. ISBN 0-553-38256-X.


  • Wallach, Wendell; Allen, Colin (November 2008). Moral Machines: Teaching Robots Right from Wrong. USA: Oxford University Press.
  • Anderson, Michael; Anderson, Susan Leigh, eds (July 2011). Machine Ethics. Cambridge University Press.
  • Storrs Hall, J. (May 30, 2007). Beyond AI: Creating the Conscience of the Machine Prometheus Books.
  • Moor, J. (2006). The Nature, Importance, and Difficulty of Machine Ethics. IEEE Intelligent Systems, 21(4), pp. 18–21.
  • Anderson, M. and Anderson, S. (2007). Creating an Ethical Intelligent Agent. AI Magazine, Volume 28(4).

Further reading

  • Anderson, Michael; Anderson, Susan Leigh, eds (July/August 2006). "Special Issue on Machine Ethics". IEEE Intelligent Systems 21 (4): 10-63.
  • Bendel, Oliver (December 11, 2013). Considerations about the Relationship between Animal and Machine Ethics. AI & SOCIETY, DOI 10.1007/s00146-013-0526-3.
  • Dabringer, Gerhard, ed. (2010). "Ethical and Legal Aspects of Unmanned Systems. Interviews". Austrian Ministry of Defence and Sports, Vienna 2010, ISBN 978-3-902761-04-0.
  • Gardner, A. (1987). An Artificial Approach to Legal Reasoning. Cambridge, MA: MIT Press.
  • Georges, T. M. (2003). Digital Soul: Intelligent Machines and Human Values. Cambridge, MA: Westview Press.
  • Singer, P.W. (December 29, 2009). Wired for War: The Robotics Revolution and Conflict in the 21st Century: Penguin.

External links

  • Machine Ethics, Hartford University links to conferences, articles and researchers.
Become part of the community of Wikilovers
Build an article or Request an article
Share a Picture
Share a video