The Volokh Conspiracy

(Update:  Thanks, Glenn, for the Instalanche!  Please see the further post above, on what I think the US legal position ought to be – what the Obama administration should be stating as its view of the relevant international law, and how I think the WSJ’s op-ed today in defense of drone strikes is not actually as protective of them as it might be.  I should add, too, that the part that Glenn was nice enough to quote is not really any different from what PW Singer says repeatedly in Wired for War, and indeed, I somewhat imagine that some of the human rights lawyers with whom I’ve had this conversation actually took their position from Singer.  It’s not some deep secret.)

The National Journal has a two part cover story on Predator drone strikes – required reading for those following the targeted killing and Predator drone developments, and although it is behind a subscription wall, no question that this National Journal issue is making the rounds of Washington and the agencies.  If you follow this topic, you’ll want to make sure you get a copy.

Part 1:  ‘Wanted: Dead’:  With little public debate or notice, the Obama administration has significantly stepped up its targeted assassinations, by James Kitfield. Well-sourced, well-researched story on the ramping up of targeted killing by the administration.

Part 2:  Are Drone Strikes Murder?  A growing number of experts say the legal foundations for targeted drone killings are shaky at best, by Shane Harris. Harris has sought out a wide variety of legal views for this piece, and the result is the best journalist take on the legal issues involved that I’ve read.  In particular, Harris has understood several things no other journalist has (at least that I’ve seen), including the importance in this debate of the customary international law of self-defense, and the controversies over what it makes to undertake “direct participation in hostilities” so as to make yourself a possible target.  Harris interviews a range of sources with, I guess, me on one side, and Mary Ellen O’Connell and Nils Melzer on the other.  But also John Radsan, William Banks, Matthew Waxman, and more – and lots of NGO folks, too.  This piece gets the argument over the law better than any journalism I’ve seen, and Harris has spent lots of time interviewing experts in depth to understand what’s at stake.

Not only is use of Predator drone strikes expanding, indications are that the Air Force has been moving forward with new and more discriminating technology - the “micro-drone” appears to be under development, according to a source indispensable for the outsider keeping up with military robotics, Wired’s Danger Room (under the fold): Continue reading ‘National Journal on Targeted Killing and Predator Drone Strikes’ »

Happy new year to everyone!  H/T Instapundit, to this excellent story with many useful links by Kristina Grifantini, Tuesday, December 29, 2009, on various advances in robotics during 2009, from MIT’s Technology Review:

In the past year, researchers have developed new robots to tackle a variety of tasks: helping with medical rehabilitation, aiding military maneuvers, mimicking social skills, and grasping the unknown …  During the past 12 months, robots got better at grasping, smiling, and avoiding angry humans.

If this short report in Newsweek is true, then I think I would worry about legal liability more than before, if I were a CIA officer involved in Predator drone strikes.  I would have a much greater level of concern that the administration would not back me up in case of an indictment in a European court, for example.  Or that it would not take steps to ensure, once the Obama administration ends, that its officials would be protected from future prosecutions, by sending out an unambiguous message that no American administration, Democratic or Republican, will tolerate such moves against American officials.  The international community that largely regards drone strikes as

• (a) extrajudicial executions and murder by any other name;
• (b) American cowardice in using technological superiority to avoid having to take personal risks to confront its targets;
• (c) a reason why America’s enemies hide out among noncombatant human shields, with the result that the Americans “force” their enemies to violate the laws of war; and
• (d) an invitation for the United States to use violence as a tool of frequent convenience because it does not have its personnel at personal risk (rather than seeing these advances in technology as humanitarian steps forward over a quarter century to increase discrimination in targeting, and rather than seeing that not having its people at risk allows the targeting to proceed on a more methodical basis, rather than in-out with greater, rather than lesser, pressure to act in the moment),

has so far refrained from doing to US officials what it is endeavoring to do to Israeli officials because of a belief, so far as I can tell, that Obama personally backs this, as he did in his campaign up until now.  His personal authority, rather than views of the United States as such, is the key source of inhibition by NGOs, UN representatives, and others.  The political calculation for the international community in this kind of debate is simple: attacking the United States increases one’s global legitimacy, while attacking Obama, at least at this point, does not.

Signals by the administration in the other direction will likely embolden legal action against American officials once this administration is over.  Signals from the international community, if not vigorously contested and rejected by the Obama administration, that the international community will pursue such actions down the road will have pretty much the same effect today: to disincentivate American intelligence officials from doing anything that they are not 100% sure will be legally protected now and in the future.

It is crucial, in my view, that the administration put on the record, or be pushed into putting on the record, a plain and broad statement that its drone policy is legal, not on narrow grounds of an armed conflict in Afghanistan spilling over into Pakistan, but on broader grounds of self-defense.  It would be consistent with what the President said at West Point, after all, in referring to action in Yemen and Somalia or other places in order to deny Al Qaeda safe haven wherever it might go.  But the administration needs to say say, plainly and broadly, as a declaration of the American view of the international law of self-defense.

Says Newsweek’s Mark Hosenball:

One person standing in the way of expanded missile strikes: President Obama. Five administration officials tell NEWSWEEK that the president has sided with political and diplomatic advisers who argue that widening the scope of the drone attacks would be risky and unwise. Obama is concerned that firing missiles into urban areas like Quetta, where intelligence reports suggest that Taliban leader Mullah Mohammed Omar and other high-level militants have sometimes taken shelter, would greatly increase the risk of civilian casualties. It would also draw protests from Pakistani politicians and military leaders, who have been largely quiet about the drone attacks as long as they’ve been confined to the country’s out-of-sight border region. The White House has been encouraged by Pakistan’s own recent military efforts to root out militants along the Afghan border, and it does not want to jeopardize that cooperation.

Of course this is Newsweek; it is quasi-reporting and quasi-opinion, so it is hard to know what to make of this.  Hosenball might be spinning things; he might be getting spun; there are many possibilities.  But if this were true, and I were a covert operations official at the CIA, I would be worried.

(PS.  Update on the British arrest warrant against a senior Israeli official:)

Israel reacted angrily Tuesday to a British arrest warrant for former Foreign Minister Tzipi Livni on war-crimes allegations, with the government threatening to sideline the U.K. in Mideast peace talks.

A Westminster, London, magistrate court on Saturday issued the warrant, alleging crimes related to Israel’s military operations in the Gaza Strip in late 2008 and early 2009. Ms. Livni, who is now opposition leader, was foreign minister at the time and one of three government officials — with then-Prime Minister Ehud Olmert and Defense Minister Ehud Barak — to oversee the offensive.

The warrant was issued ahead of a U.K. convention of the Jewish National Fund, to which Ms. Livni had been invited, but had declined to attend. The warrant was revoked by the court on Monday after it was clear she wasn’t in the country.

Of course, it does not require an actual prosecution in order to create big effects; the uncertainties introduced about the future are enough to shift behavior today.  That is also true of CIA officials in this country – not knowing what will, or will not be, sufficient to provoke an arrest warrant abroad by some magistrate acting on vague and discretionary authority, such as Baltasar Garzon in Spain, and moreover acting in a climate of local public opinion against the foreign country and its agents, such as the United States or Israel.

The British legal system allows private individuals and organizations to request arrest warrants from local courts under the principle of “universal jurisdiction.” The judicial concept allows domestic courts around the world to try cases of war crimes and crimes against humanity, even if the infraction occurred abroad and the suspect isn’t a citizen.

In recent months, U.K. lawyers representing Palestinian groups sought an arrest warrant for Mr. Barak, but it was denied on grounds of diplomatic immunity.

Israel called for “immediate” action from the U.K. to block plaintiffs from using its legal system to put Israeli leaders on trial for actions in the Palestinian territories. Ms. Livni is the fourth senior Israeli official since 2004 that pro-Palestinian activists have sought to detain using British courts.

As this British situation shows, it is a process by which central government can shrug, if it wants to, say it’s the independent judiciary, we can do nothing, while allowing the process to be captured by activists and advocates further down the system.  Protests by Israel mean little, protests by the United States (if it is inclined to make them, which is far from guaranteed) mean little more.  And, to judge by the behavior of Spain, what matters is the protest, public or private, of a rising power with teeth and the expectation that it will be heeded – China’s private objections to Spain’s expansive universal jurisdiction provisions and the indirect threat of withholding economic benefits, I’m told by decently connected friends in Spain (but cannot verify), was important in getting Spain’s legislature to consider scaling back its ambitions on criminal universal jurisdiction.

It is also important to recognize that the United States has its own civil law version of such universal jurisdiction, the Alien Tort Statute – and one of these days, it too will face a challenge from China on behalf of one of its corporations getting sued in the US for its actions in Africa or elsewhere, and it will be exceedingly interesting to see how the US reacts.

One of my favorite issues of the New York Times Magazine is its “year in ideas” issue, which comes annually in December.  It has a short item this year related to battlefield robotics and law and ethics, by Dara Kerr, “Guilty Robots.” (If you go over the Opinio Juris version of this post, you can pick up the “robots” tag to get all the blogging there on international law of war and ethics and battlefield robots.)

[I]magine robots that obey injunctions like Immanuel Kant’s categorical imperative — acting rationally and with a sense of moral duty. This July, the roboticist Ronald Arkin of Georgia Tech finished a three-year project with the U.S. Army designing prototype software for autonomous ethical robots. He maintains that in limited situations, like countersniper operations or storming buildings, the software will actually allow robots to outperform humans from an ethical perspective.

“I believe these systems will have more information available to them than any human soldier could possibly process and manage at a given point in time and thus be able to make better informed decisions,” he says.

The software consists of what Arkin calls “ethical architecture,” which is based on international laws of war and rules of engagement.

The “guilty” part comes from a feature of Professor Arkin’s ethical architecture, in which certain parameters cause the robot to become more “worried” about the rising calculations of collateral damage and other such factors.

After considering several moral emotions like remorse, compassion and shame, Arkin decided to focus on modeling guilt because it can be used to condemn specific behavior and generate constructive change. While fighting, his robots assess battlefield damage and then use algorithms to calculate the appropriate level of guilt. If the damage includes noncombatant casualties or harm to civilian property, for instance, their guilt level increases. As the level grows, the robots may choose weapons with less risk of collateral damage or may refuse to fight altogether.

I am agnostic as to whether at some point in the future, robots might prove to be ethically superior to humans in making decisions about firing weapons on the battlefield.  When I say agnostic, I mean genuinely agnostic – it seems to me an open question of where technology goes, and in, say, a hundred years, who can say?  For thing, I do fully imagine that roboticized medicine, surgery and operations, will very possibly have reached the point where it might well be presumptive malpractice for the human doctor to override the machine.  It is not impossible for me to imagine – far from it – a time in which it would be a presumptive war crime for the human soldier to override the ethical decisions of the machine.

But maybe not.  Although I am strongly in favor of the kinds of research programs that Professor Arkin is undertaking, I think the ethical and legal  issues, whether the categorical rules or the proportionality rules, of warfare involve questions that humans have not managed to answer at the conceptual level.  Proportionality and what it means when seeking to weigh up radically incommensurable goods – military necessity and harm to civilians, for example – to start with in the law and ethics of war.  One reason I am excited by Professor Arkin’s attempts to perform these functions in machine terms, however, is that the detailed, step by step, project forces us to think through difficult conceptual issues regarding human ethics at the granular level that we might otherwise skip over with some quick assumptions.  Programming does not allow one to do that quite so easily.

And it is open to Professor Arkin to reply to the concern that humans don’t have a fully articulated framework, even at the basic conceptual level, for the ethics of warfare, so how then is a machine going to do it?  ”Well, in order to develop a machine, I don’t actually have to address those questions or solve those problems.  The robot doesn’t have to have more ethical answers than you humans – it just has to be able to do as well, even with the gaps and holes.” I’m not sure that answer (which I’m putting into Professor Arkin’s mouth entirely hypothetically, let me emphasize) would be sufficient – partly because I suspect that intuitions applied casuistically by human beings often encode and respond to facts that affect our ethical senses in ways that would not really be articulable, by human or machine.  And partly because we probably do think that in various ways, the machine has to be better than the human.

Many readers will by now be familiar with Peter W. Singer’s widely noticed Wired for War. But I would suggest following it up with Professor Arkin’s own new book, Governing Lethal Behavior in Autonomous Robots (particularly now that Amazon has dropped the price from $60 to $40).  (I guess I should also add that this discussion is about battlefield robotics in the sense of “autonomous” firing systems – not the current robotics question of human controlled, but remote platform unmanned combat vehicles, Predators and drones, and targeted killing.)

(Update:  Thanks Instapundit!  Let me add two things, looking to the comments.  First, I agree that “regret” is the correct term here, not “guilt.”  Guilt is the term used in the NYT article – but as moral emotions go, it is one of the most difficult conceptually to frame, and I say this as a student of one of the great philosophers on this topic, UCLA’s Herbert Morris.  Second, I agree that Professor Arkin does indeed mean “better” than human in a very limited, deliberately limited frame; to get a sense of the parameters he means, look at his excellent book.  Finally, for those concerned that without consciousness or intentionality on the part of a robot, no matter how sophisticated the programming, it isn’t really “morality,” I am happy to call it some form of moral simulacrum, because the issue, for these purposes, is how close behaviorally the robot can come via its programming.)

I mentioned earlier a panel discussion I participated in on robots, law and society at Stanford Law School a couple of weeks ago.  Adam Gorlick at Physorg.com has a good article summing up the discussion.

Ryan Calo, of Stanford Law School’s Center for Internet and Society, raises some of the fundamental liability issues and the implications for research and development – and I fully share his concerns, and more.  (You can read an early draft of one of his academic law papers on privacy at SSRN.)  What happens, for example, if the robot fails to call 911 as programmed?

[W]ho will be to blame if a robot-controlled weapon kills a civilian? Who can be sued if one of those new cars takes an unexpected turn into a crowd of pedestrians? And who is liable if the robot you programmed to bathe your elderly mother drowns her in the tub?

As mechanical engineers and computer scientists at Stanford develop technology that will transform the stuff of science fiction into everyday machinery, scholars at the Law School are thinking about the legal challenges that will arise.

“I worry that in the absence of some good, up-front thought about the question of liability, we’ll have some high-profile cases that will turn the public against robots or chill innovation and make it less likely for engineers to go into the field and less likely for capital to flow in the area,” said M. Ryan Calo, a residential fellow at the Law School’s Center for Internet and Society.

And the consequence of a flood of lawsuits, he said, is that the United States will fall behind other countries – like Japan and South Korea – that are also at the forefront of personal robot technology, a field that some analysts expect to exceed $5 billion in annual sales by 2015.

“We’re going to need to think about how to immunize manufacturers from lawsuits in appropriate circumstances,” Calo said, adding that defense contractors are usually shielded from liability when the robots and machines they make for the military accidentally injure a soldier.

One of the features of this discussion, as well as a discussion at Harvard Law School last week, was to ask what makes robots different as a matter of technology.  A decent rule of thumb is that robots feature:

• mobility or or locomotion or at least some mechanism for physically interacting between it and the world, at the ‘gross motor level’ (yes, one can talk of computers interacting with the world by running the phone system or something, but robot technologies are special in part because of the machine itself physically taking motor actions);
• sensors that allow the robot to take in signals and streams of input from the outside world; and
• computational power (specifically, computational power that allows for learning of some kind to adapt the other features of physical action and mobility and sensors).

I don’t want to get doctrinal or theological here about the Essential Nature of Robots; these are just rules of thumbs.  But they figure in how one thinks of robots as distinguished loosely from other technologies – and another thing noted in one of these discussions, and not a surprise, is that a larger proportion of the folks who work on them tend to come from mechanical engineering than computer science than is the case in computers or internet technologies.

(Update:  One of the comments suggests that Asimov already addressed these questions fifty years ago, albeit speculatively.  It’s an interesting sideline question.  In my possibly fallible memory of the Robot stories, Asimov addressed many questions on the assumption of the three laws; the stories turned on interpretations of those laws.  I often thought the most interesting aspect of the Robot stories was the construction of the different societies, based around robotics technology – Earth, crowded with humans and robot-phobic, at the one extreme, and Aurora, void of humans and filled with robots at the other extreme.  It wasn’t the robots that were interesting – it was the comparison of the two societies.  But I also think that the assumptions that Asimov built into his robot stories in order to give – as with all good fantasy, speculative fiction, fairy tales, ghost stories, horror – a structure with its own internal set of rules to keep us interested in the narrative, also meant that his stories did not address many of the things referenced in the post above.  What do you think?  Did Asimov cover the bases, and the rest is simply inflection; or am I right in thinking that there are vast areas quite untouched?)

One small nugget I took away from the (absolutely terrific) Stanford Law School robotics panel last week was a much better appreciation of how robotics will interact with advanced societies aging – elder-care, health care for the old and infirm, and so on.  Japan leads the way.

Paul Saffo (Stanford professor, futurist, and technology journalist, and very smart guy) remarked that the last ten years had seen an important technological shift, crucial to robotics, in the development of cheap sensor devices.  Sensor devices that could harness the computational power of the chip and make it possible to interface with the real world and, combined with improvements in elements of motion and locomotion, gives the world genuine robots.  It is movement, sensing, and computational power in combination that makes it possible for robots to do things, and do things for us.

That leads to the age of robotics, and – depending in part on what happens to R&D budgets in health care – the care of the elderly is one natural area of application, as well as a source of revenue to fund the industry.  More, faster please, as Glenn Reynolds might say.  Saffo also remarked that in a certain way, old people coming to depend on robots to help and do things for them as the fulfilment of “I have always depended on the kindness of strangers” – robotic strangers, in this case.

I added, and think it more important in setting out future technology trends here than one might initially figure, that a driver of robotic care for the elderly will be that the elderly themselves prefer robotic strangers caring for them, rather than human strangers.  Particularly in all the intimate, intrusive, personal things like bathing and toiletting – I at least would vastly prefer to interact with a machine rather than a home care person.  Robots in that sense help me avoid having to depend upon the kindness of strangers.

This is outside of my usual area of robotic remit – robots and the laws and ethics of war.  But I am rapidly moving to backfill into these other areas as it becomes clear that these questions of technology, but also of law, are interrelated and often versions of the same thing.  The robotic decision whether to fire a weapon or not, if technology ever comes to that point, is importantly interconnected with the question an eldercare robot might have to ask regarding whether to call 911.

(There are several topics raised by the Stanford discussion on robotics and I’ll try to get to several of them over the next few posts.  But I wanted to thank Ryan Calo and all the folks who put the discussion together – it was a great set of discussions for me and I hope for everyone who attended.  I realized, sitting and listening, that there are not that many places in the US where you could hold that kind of discussion, with an audience including engineers and technologists and scientists sitting in the office who actually work in the field, not just in academic departments, but in commercial firms and ventures, trying to make it real.)

I’ve been traveling recently, and so have been away from posting.  One of the enforced virtues of traveling – one of the few virtues of traveling for me these days – is the plane flight with no internet.  And if the big guy in front of me reclines his seat, as he always does, I can’t even get to my computer.  So I read  on flights.  I should have some reading gadget, Kindle or whatever, but I’m not that far along yet, and for that matter I should get an economy class friendly little word-processor to use on flights, but I’m cheap.  Here’s a selection across the varied reading on my flights.  No particular theme or order, I’m afraid (on account of the mixed-up topics here, I think I won’t open to comments; too jumbled to be productive). Continue reading ‘Reading While Traveling, Hard Copy and No Internet’ »

If you are going to be around Palo Alto next Thursday evening, you might consider attending a panel discussion on robotics and law at Stanford Law School.  I’ll be on a panel alongside some very interesting and knowledgeable folks taking up varied aspects of robotics (my particular interest is robotics and war, but the panel will be considering many areas of robotics).  The particulars are below the fold.

(Update:)  Here’s the assigned topic for comments, following up on Laura’s opening comment … should the panel discuss the Three Laws?  Are they a useful ethical/legal frame for dealing with robots in various aspects of human life?  Did Asimov lead us all astray by proposing them?  Should we instead avoid discussing them altogether?  What would you propose would be a better set of principles/laws/guidelines for robot-human interactions?

(I’ll also be giving a lunch talk/discussion that same day sponsored by various student organizations at SLS specifically on robotics and armed conflict. And thanks Glenn for the Instalanche!)

Continue reading ‘Law and Robotics Panel at Stanford Law School’ »