Tektronix sold our video products division to Telestream. Please contact them with questions on video products. http://www.telestream.net/
The DPOJET help manual states:
"The Phase Noise measurement performs a jitter measurement, converts the result into the frequency domain, and reports
the rms jitter integrated between two specific frequencies selected by the user."
So a TIE jitter measurement is performed and an FFT is conducted of that.
The integration limits that are used depends on what the user wants to see, which is generally based on the specification for the clock.