. >> stuart russell had a question specifically addressed to you. stuart russell is one of the computer scientist that we all respect greatly. dan, you seem to divide a.i. symptoms into these tools with no middle ground. as a program with representing goals in the real world and such agents could be competent in the little world and yet nonconscious. do you believe that consciousness will necessarily create them as we make the programs more confident in general? can you tell us how not to make conscious a.i. systems? >> good question. i'm glad i answered. indeed we can have very very intelligent systems which are not conscious in any interesting way. but they will see conscious in many ways but they will not have important features that we have. it's very much a matter of whether they are capable of taking their own interstates as objects of scrutiny in doing that recursively and indefinitely. and that's a very special feature which i think no nonhuman animal has that capacity and that's a big difference between human consciousness and animal sentim