Your feelings are human feelings and God does not want us to deny our humanness, He created us body and soul. The guilt in this behavior is not you but the persons who view you in a negative manner without knowing you. That is their fault, or their sin by being uncharitable. You can be the source of their good intentions and kindness but they choose not to accept it. Remember what you are experiencing are human feelings and that is allowed what is not allowed is to treat others as they treat you. Rise above the din of the crowd.