I totally agree -- I think it really rattled him that she could con him *that* well, not to mention that he could actually fall for her so deeply and it be so untrue (well, initially, at least), etc.

And his line about "She doesn't look at me like I'm a criminal" just about broke my heart!! *sniffles*