The limits we put on others