If medication actually gave you some relief I am not sure why you would hesitate. I had anxiety once that was "induced" by dense trees, high humidity. It greatly subsided the second I got out of the trees, but it did not disappear entirely for months. I rarely ever take medication and a lot of it doesn't agree w me, so naturally I was afraid of even going that route when I had anxiety...it seemed everything was "scarier" when I had it. As many have said..exercise - it is a "natural drug" and soooo effective. I like to walk/hike - so I did for many miles a day - sometimes twice a day. It was a life saver! I found I couldn't sit for long w/o it getting worse, and I never consumed caffeine during that time b/c it would seemingly amplify it 10 time over. Everyone is different, but I took a lot of vitamin B complex and Mg which actually helped.
I did not have the misfortune of having to "get into" it with someone during my episode with it - but if I had of been confronted with an unpleasant exchange, I think I would have just - not participated. :) That is pretty easy to do, you just say, "I can't discuss this with you right now, I will let you know when I can, though - good bye".