After almost two weeks (yes, two dam weeks!!!) of debuging, I finally found out the reason why the SSC interface connected to my 16bit ADC doesn't work correctly at higher speed.
I connected SSC RK pin to output clock to ADC and RF pin output waveform to ADC to start periodic data conversion, and RD pin receive ADC serial data output. If I set CMR register to 1, which means SSC works at F_PBA/2, then the SSC RHR register always give me the 16bit reading that is right shifted 1 bit, which means SSC has missed 1 bit from ADC output, it's always like this no matter if the CKI bit in RCMR register is set or clear, and no matter what F_PBA frequency is.
But if I set CMR to 2, which is F_PBA/4, then the SSC reading is always correct.
So now I can run the F_PBA at quite fast speed to have higher ADC sampling rate.
hope this could help someone who got the same problem.