Amy asked:

In light of the looming possibilities of artificial intelligence and nanotechnology, just what exactly does
it mean to be human? If we could create intelligent "beings," would they be tools with our intentions,
or is it possible that through emergent consciousness that they may develop their own teleological
goals? Would these "beings" warrant moral consideration? Who or what does warrant moral
consideration?

============

I just got back from a conference where this question was posed and answered by Ray Kurzweil
( http://www.kurzweiledu.com/index.html), in a manner I find reasonably convincing. He argued that
any entity capable of suffering deserves moral consideration. So this would include what animals we
could determine actually suffer (rather than behave as if they are suffering — an objection of
Descartes which might conceivably apply, say, to insects), and machines we build or cause to be built
at some time which also suffer, as best as we can determine.

But one could object that any conscious being deserves moral consideration. The questions then
become, 1) can an entity be conscious and not be able to suffer, 2) can an entity suffer and not be
conscious? As far as 2) goes, one would have to say no, I would think... but then we get into
considerations of degrees of consciousness. Sartre and others have described first- and
second-order consciousness, and one might carry that further and hypothesize degrees within those
categories. Then we have a much harder question, i.e., how far down can we go and still suffer? No
one knows the answer to that.

The next question is, what is "moral consideration"? Assume that a dog is conscious, but not as
conscious as a human. Does it deserve less moral consideration because of that? The operative
answer, as we see it instantiated, is yes. But that assumes we can say what "less" is, and that we
have means of knowing that dogs are less conscious than we are, and that indeed our logic making
that connection is correct.

Well, I'm going to leave it here. Books have been written on this type of thing, not to mention tons of
sci-fi. On the latter, you might check out P.J. Farmer's stuff; he's fascinated by morality as it relates to
aliens.

Steven Ravett Brown

I want to pass on the first question, as it is a bit too big for me! I'll tackle the questions on beings with
Artificial Intelligence.

I would go for the second of your possibilities. It seems to me that consciousness must be an
emergent property, underpinned by a certain complexity of 'brain' development. While this has to be
an empirical question, so we will have to wait for that answer, I don't see any reason why such
complexity is only achievable in carbon based life forms. We are probably a long way from achieving
it in our inventions yet, but I would hazard that we will reach it sometime.

Such beings would warrant moral consideration, it seems to me. Any being that is conscious of its
own actions and the effects (for better or worse) of those actions on other similarly conscious beings
would count for moral consideration. Such a being can have a reflective image of itself as a moral
actor, and hence can develop a moral character.

I suspect though, that there is another requirement for the development of both sufficient intelligence
and of a moral character. That is that these beings would have to grow up in a community of
intelligent beings — maybe beings just like itself, but equally in a human community. I won't go into all
my reasons here, but I believe that the same holds for humans.

Tim Sprod