From Wikapedia:
There is no recognized
etiology for infantilism and there is little research done on the subject. It has been linked to
masochism and a variety of other
paraphilias. Although it is commonly confused with
pedophilia, the two conditions are distinct and infantilists do not seek children as sexual partners.
[12][13] A variety of causes have been proposed, including altered
lovemaps,
imprinting gone wrong and
errors in erotic targets, though there is no consensus. A variety of organizations exist to discuss infantilism or meet with other practitioners throughout the world.
I would also add attachment theory. Having been on this site for 13 years, I have observed many different reasons given by members for their attraction to diapers and other baby objects. Because of all the different personal histories, I don't think Infantalism is caused by childhood trauma, bullying or a loving family. I lean toward the theory that we created some sort of sexual attachment to diapers at a very early age, seeing our siblings or neighbor children in diapers and experiencing some sort of arousal.
That said, I was adopted at the age of two and I believe I may have spent some time in an adoptive institution. I know I was not potty trained when my parents adopted me as I can remember my mom changing me as well as her telling me when I was 4 years old that she still had my diapers. I suspect that there will be theories as to why we are attracted to diapers and that will be as far as it gets with us never finding anything that is definitive.