First of all, the fact that you're concerned about this tells me you're not as hard-hearted as you think. If you had a really hard heart, you simply wouldn't care at all about it. So take heart in that.
Second, let me tell you personally that you're not the only one that's had this happen to you. You're in good company. I've had this struggle at times myself, and I had a VERY hard time confiding in people about it, partly because I was running around with a predominantly Arminian/Pentecostal crowd at the time, and I feared that they'd accuse me of not being a Christian, or that they'd just say "do more, try harder," or have me come to the altar again. Then I started reading about people like Luther, who went through what you described. He talked about feeling a real spiritual assault. That really made me feel good, because it meant that I wasn't alone in this. That was the other thing I loved about Calvinism; Calvinists were (and are) very frank about their sins and shortcomings, and it made me realize that this struggle wasn't something I was alone in. So take heart, brother: you're standing with Paul, Peter, Augustine, Calvin, Luther, and many other great saints in having hard times with sin.
We tend to take one of two extremes as Christians. Either we make sanctification a passive matter, believing that God works it and we just "go along," or we go to the opposite Arminian/semiPelagian view of thinking that we have to do it all while God sits back as a passive spectator and wrings His hands hoping we make the right decisions.
The truth is that God expects us to make a discipline of sanctification, but at the same time it's not something done apart from Him. Philippians 2:12-13 is a good illustration of this. We have Paul telling us to work out (note that it's not "work for") our salvation with fear and trembling, then follows it up with a reminder that it is God that works in you. God is with us in our sanctification, no matter how holy or unholy we feel, no matter how much or how little we accomplished. If you are trusting in Jesus Christ and His work on the cross for your salvation, be assured that God is with you.
Now, on a more practical level, consider this: are there external things that make it easier for your mind to wander? For me I had to start turning off the TV and make a deliberate effort to stay away from areas on the Internet. At times, it also meant not socializing with some people, so as to avoid temptations to be loose with my humor or crude with my words. If you have areas like this, you may want to consider "cutting them off" as Jesus talked about.
Something else to remember: the more we are in the Scriptures and in prayer, the harder it is for our mind to wander. There is a good discipline aspect to regular Bible reading and prayer, and I can tell a difference in myself when I'm not as faithful about it. Mind you, it's not the discipline in and of itself that is what matures me, but it is through that discipline that I commune with God and enjoy fellowship with him. And in those times, God has brought to mind sins that I need to confess and repent of. It's not always fun, but in the end it makes us better.
And finally, don't hesitate to ask for prayer from other believers! We are here to lift each other up to God, especially when things like this beset us. Remember that you are not alone: God is with you, and so are His children, YOUR brothers and sisters, washed in the same blood you are, walking in fellowship with God just like you are, and who will spend eternity in the presence of God, just like you will be.
So you'd better learn to get along with us
Seriously, Shawn... I will pray for you, and I'm sure that others here will do the same. And run to God with all your might. Run to Him whether you feel like it or not. Run to Him regardless of your thoughts, your emotions, your lack of emotions-whatever. But run to Him. He waits for His children at His throne of grace.
In the grace of God
J. Dean
