I just don't think I would question that about a man at this point. I would want to know that he has lived a life according to God's Word, that he abhors immorality, and that he will hold to the highest ideal, even in a compromising situation. It doesn't even matter to me whether it was before or after becoming a Christian. If God loves someone who has repented of their sin, and it makes no difference to him what they've done or what they were or what they are now, then why should it matter to us?
Even so we should all save ourselves not just for our soul mate, but for God's glory and purpose.
Women (usually) don't struggle after seeing a TV ad/billboard/magazine page showing a man in his underwear with body curves revealed. It doesn't seem to hook, grip, and mesmerize women as it does men. I often find (and I believe true from the men that I've known over the years) that for us guys, sheer willpower is not sufficient to keep our minds pure. Point out anything in me that offends you, and lead me along the path of everlasting life. Point out anything in me that offends you, and lead me along the path of everlasting life. I find that it can be more difficult for a non-virgin to maintain the boundaries with which I am comfortable.
Even men's accountability groups doesn't always overcome the inherent propensity of us guys to lust. However, as far as holding it against them, I believe that once they become a Christian if God does not hold the sin against him then neither can I. I can't judge because I've screwed up royally myself.
If he is a virgin in God's eyes because the slate is wiped clean then he is a virgin in mine too. Virginity is the gift God gave to each of us to give away to the one we are meant to love with all our being.