Okay I found the problem.
Setting dinfo.do_fancy_upsampling = FALSE in the couple places they initiailize dinfo in jpegutils.c now has the colors coming out correct.
Looking into libjpeg after jpeg_create_decompress and then jpeg_read_header the do_fancy_upsampling is set to its default of TRUE.
So either libjpeg does not handle it being set to TRUE correctly or the motion program should have set it to FALSE.
Now to move on to the ffmpeg support problem.
They actually have inline assembly in the code.
Okay that was easy. All I had to do was take out the inline assembly code. It looks to be all working now.
More features then the packaged version.