Stereo capture with v4l2

Stereo capture with v4l2

I am trying to capture images from 2 cameras using the driver v4l2. I want to capture them at the same time so I use threads with conditions to start the capture at the same time. The problem is that the capture is always slower for the second device (I tried with different cameras and inverting them).
I modified a little bit this example for the capture.
I use this camera (both camera), but I also tried with one camera from minoru and a common webcam (in 640*480 at 15 fps)
Here is the output of my program:
Signal
/dev/video1 debut: 00:00:00.637.043
/dev/video0 debut: 00:00:00.637.157
/dev/video1 fin: 00:00:00.637.645
/dev/video0 fin: 00:00:00.700.939
Signal
/dev/video1 debut: 00:00:00.701.104
/dev/video0 debut: 00:00:00.701.163
/dev/video1 fin: 00:00:00.704.868
/dev/video0 fin: 00:00:00.768.935
Signal
/dev/video1 debut: 00:00:00.769.024
/dev/video0 debut: 00:00:00.769.138
/dev/video1 fin: 00:00:00.769.627
/dev/video0 fin: 00:00:00.836.922
So I don't understand why the first capture is really fast and the second one is not. Any idea or recommandations ? Could this come from the v4l2 drivers ?