You are not alone on that one.
I think a lot of us wish that being an AB or a Little was more socially acceptable. It is an inconvenience to have to hide this side of myself from others until I get to know them and learn if they can be trusted and even then, as an AB or a Little you are always at risk of people not getting this and taking offence to it.
I agree with you, being an AB or Little doesn't hurt anything or anyone. The vast majority of us are very courteous and private with this interest and it can be a source of much happiness, fun and healing for many of us. The fact that others can deem being an AB or a Little as wrong ticks me off, I wish more people could just embrace a "don't knock it until you try it" mentality or take it upon themselves to become more educated on the subject.
Sadly, our society thinks that anything outside the realm of "normal" human behaviour (define that one for me) needs to be shunned or avoided, this can make it difficult for a lot of AB's and Littles to find understanding people to surround themselves with. Fortunately, I've been very selective with who I've told, as a result all of my friends accept and or understand this side of myself and treat me no differently for it and I've been lucky enough to have had very accepting girlfriends and partners who have actually played Mommy to me, so never underestimate that there are people out there who can understand us.
Also, if you are looking for friends to talk about this with, you'll be meeting a lot of them here on ADISC. If you have any close friends you think could understand, maybe work up some courage and confide in them when the time feels right, it's always a huge weight off your shoulders and a huge step towards self-acceptance when you tell someone and they accept it.
One more thing, Welcome to the site Sweethoney