I'm pretty good at finding words to fill in that blank: intimidating, time-consuming, scary, not-my-personality, not-my-responsibility, etc.
In addition to nervousness or passivity, I think it's often the fear of the unknown that stops us from asking others about their spiritual life and sharing with them an opportunity to have a relationship with Jesus. Like a good boy scout we feel we should to be ready for every circumstance, know every answer, and have not even a hint of fear when sharing the truth of Jesus with others.
When Jesus called the first two disciples (uneducated fishermen who were learning the family business), he didn't tell them "I want you to start holding open air rallies, teaching courses on apologetics, doing altar-calls, developing church-planting ministries, teaching seminary-level courses on eschatology, etc." He simply asked them to follow him and walk in his path. The equation was simple:
1. You follow me,
2. I will make you a fisher of men.
These two fishermen stopped what they were doing and immediately followed him... No excuses. Jesus didn't demand that they had their act completely together. He didn't demand that they had a game plan for reaching the world, He simply demanded that they be willing to follow him.
Would you stop worrying about what you will make of yourself, would you stop worrying about what others will think of you, and would you even stop worrying about what Jesus will make of you, and just start following Him?