A person did some bad deed (paapa karma or vikarma). The person went to hell and was punished there. The punishment reduced the intensity of quality (guna or samskara) which is the cause of that deed. But the quality is not completely removed and a small trace of it remains. Punishment cannot remove the quality which is a form of wrong knowledge. The wrong knowledge can be completely removed only by right knowledge and not by external physical punishment. Of course the punishment reduces the intensity of wrong knowledge to a great extent. A thief can leave his quality of stealing only on realization obtained by the right knowledge. Any extent of torture cannot remove this quality completely. Therefore, even when the thief comes out of the police station, a trace of the quality of stealing remains.
But if he meets a Satguru and receives right knowledge, the quality can be removed completely. Valmiki, a robber, became a sage by the preaching of Narada but not by the third degree treatment in a police station. The samskara or quality is knowledge and can be destroyed by another type of knowledge only, as a diamond is cut by another diamond. Except the knowledge, no other physical works like chanting the name, doing worship, singing songs or any other physical exercise can remove the wrong knowledge which is the cause of sins.
Now what is this wrong knowledge? Is it not karma? As I told you the knowledge is also karma because any part of the creation is karma only. Quality or samskara is a process of feeling or thinking of awareness which is karma only. But the karma of awareness can be removed by another karma of awareness only and not by the karma of matter or inert energy. Beating and burning are the works done by inert matter and inert energy. They can affect the body and since the body is connected to mind, it can create fear and reduce the intensity of the wrong knowledge but cannot remove it from the root. Only the logical analysis of right knowledge can remove this samskara or quality from the root.
When the quality is removed, it�s corresponding deed also disappears. Therefore the right knowledge removes the cause of the karma and thus burn the quality and its deed from the root (Jnanagnih Sarva karmani-the Gita). When the sinner comes to the earth from the hell he is with the trace of quality. Since the quality is also basically karma, the trace of quality is called as karma sesha (remains of karma). This trace of quality grows in a congenial atmosphere into a big tree and the deed is again repeated. The punishment of this deed may be in the hell or sometimes may be immediate. Sometimes the thief may be caught by the house owner on the spot and may be beaten immediately. Sometimes the thief may be caught in the police station and may be beaten there. Even if he escapes these two places he will certainly be beaten in the hell.
Therefore the deed of the previous birth punished in the hell comes down along with the soul as karma sesha (quality or samskara) which is responsible for the corresponding deed on this earth and which may yield its fruit here itself. In such case his previous quality called as previous karma is certainly responsible for the misery here. Here you have to understand one point. The karma of the previous birth is not giving its result directly in this birth. The samskara which was responsible to the karma of the previous birth continues into this birth as a trace, which grows in a congenial atmosphere and generates its corresponding karma once again, which may yield sometimes its fruit here itself. Thus the karma of the present birth is only responsible directly for your misery here and the present karma has its root in the samskara of previous birth. This samskara can be called as karma and thus in that sense it is said that the karma of the previous birth is responsible for the present misery.