First try - MOV video with 4 different tracking points and normal head movements, for some reason I couldn't export the tracking information into Cinema 4D object tracking, option "Run 3D solver" unavailable. I checked online and seems like the problem was the path.
Fixed all the red paths and wrong tracking frame by frame...same situation
Second try - Recorded another video with slow head movements and only two very visible tracking points, unfortunately, same situation, but tracking quality was much better.
Checked online for solutions again, most tutorials people had started using images instead of movie files for their footage.
Even though tracking worked perfectly, the object tracking had the same situation.