Intimidating?

I have been told many times by several people that ( especially by men) they find me very intimidating. Apparently, I am always very sure of myself and sound really confident and fearless. Yes, I am a confident person but I fail to understand why I am intimidating? I would be lying if I say it does'nt bother me. It does.

While my mom thinks its good that people are scared of me and that men always find smart women intimidating! :P...I genuinely don't want to come across as if I am unapproachable or intimidating, especially in the workplace. It happens that people form a perception and it tends to stay as I don't really get an opportunity to prove otherwise, unless the other person gets to know me personally, which can be rare. I must admit I am not one of those shy, demure kind of women nor someone who hesitates to speak her mind. Is that arrogant behavior? I don't think so! I do not mince words. PERIOD. At the same time, I am not rude, am polite but am not sugary sweet.

As a result of all this behavioral speculation, I have tried to tone down my personality, so to speak. I try to think before i speak and ensure that I don't sound 'arrogant' or 'intimidating'. In this painstaking ( for me, it is)process of modifying my personality, I forget what I really want to say and end up focussing on how I come across...which is not a great help either! All my thoughts are lost in the presentation!
What do I do? I can't say 'I am like this wonly! Take it or leave it! In the real world, it does'nt work that way. Any suggestions?