I keep avouding men. Is that wrong?

You will not heal/grow by avoiding people, reading books or seeing therapists as much as you may by building the relationship you have always wanted and never had, lasting love. Build your self-esteem by dating and walking away from the relationships you do not want, and having an open heart to the right one.