We also have a tendency to think in terms of concrete acts, and to put them in this category or that one. So, we ask "which act was Adam's sin," or the first moment of sin; rather than (as I think apt) thinking in terms of a process. Short-circuit the process on the way to taking and eating, and that sin has been vanquished, even it if was a near thing.
But looking back, one can see the earlier beginnings of taking down the wall against sin, earlier failures to resist in a robust and full fashion the attacks on faith and obedience. All together, then, we see the cumulative work of sin and temptation discovered in the actual eating.
Suppose, though, that Adam and Eve found the necessary power to resist the temptation. It had begun, then it wore them down. In our present condition, even to be tempted is pretty much to sin to a lesser degree. But Adam and Eve were not yet fallen, though they were in process of falling. We don't know, however, if there was some way to halt that fall in the early stages. I'm of the opinion that to find the way of escape would be to know weakness, but not (being still upright) to have fallen.
Perhaps expelling the serpent was a thing to be recognized, to be learned. Supposing Adam instantly should know the serpent for the threat he was is a bit of judgment of him we may not be entitled to make. But suppose he finally saw the threat, and acted accordingly. Would he be guilty of a sin for not acting previously? That seems too fine an assessment.
Man does not have perfect knowledge to begin with, being a creature. But later on, he lacked the will to act properly; his spiritual faculties were beclouded, though he is certainly to be faulted at the end for every part of the failure, where it is shown he should have done otherwise due to his power or knowledge.