High-Speed Serial Channel Simulation

Simply put, it's a simulation that connects the entire transmission path between transmitter and receiver—IC package → PCB trace → connector → cable → connector → PCB → IC—using S-parameters, and predicts the eye diagram including equalizers.

Yes. For example, PCIe 6.0 uses 64 GT/s PAM4 signals, with a Nyquist frequency of 16 GHz. The channel insertion loss can sometimes exceed 30 dB. Designing compensation for this with CTLE+DFE is impossible without simulation. We "cascade connect" the S-parameters of each part and solve the entire system—transmit equalizer → channel → receive equalizer—consistently.

To be precise, the voltage amplitude ratio is $10^{-30/20} \approx 1/31.6$. The equalizer restores the signal from a state where it's almost gone at the Nyquist frequency. That's precisely why channel simulation, which "verifies the design including equalizers," becomes essential.

Each component is modeled as a 2-port network (S-parameters) and connected in series. For example, a server backplane would have a configuration like this:

You can't just multiply S-parameters directly. If you convert them to T-parameters (Transmission matrices), you can cascade them via matrix multiplication. For a 2-port network:

The conversion formula from S-parameters to T-parameters is:

Yes, tools like ADS or Sigrity read Touchstone files (.s2p/.s4p) and automatically calculate the cascade connection. However, there's a caveat—when the frequency points of each S-parameter don't align, interpolation is needed, and the accuracy of this interpolation can affect the results. Extrapolation at the high-frequency end requires particular caution.

The three most important ones for channel simulation are:

• $S_{21}$ (Insertion Loss): How much signal passes through. Attenuation increases with frequency. Directly indicates the "difficulty" of the channel.
• $S_{11}$ (Return Loss): How much is reflected at the input port. An indicator of impedance mismatch. $|S_{11}| < -15\text{dB}$ is a typical target.
• $S_{\text{NEXT/FEXT}}$ (Crosstalk): Interference from adjacent channels. Evaluated using multi-port S-parameters.

Representing the entire channel after cascade connection as a single transfer function $H_{\text{ch}}(f)$ gives:

The end-to-end transfer function including equalizers is:

DFE is a nonlinear process that cannot be expressed in the frequency domain, so it's not included here. DFE is calculated separately in the time domain.

The fundamental difference is that CTLE is a linear filter while DFE is a nonlinear feedback.

CTLE (Continuous Time Linear Equalizer) is a high-pass filter that "boosts" the high-frequency components attenuated by the channel. Its transfer function looks like this:

$f_z$ is the zero frequency (boost start point), $f_p$ is the pole frequency (boost upper limit). The high-frequency gain increases in the frequency band between the zero and the pole. In practice, multiple CTLE stages are often used, providing around 10–15 dB of boost.

Exactly, that's the weakness of CTLE. CTLE involves noise enhancement. That's where DFE (Decision Feedback Equalizer) comes in. DFE uses past decided bits to cancel post-cursor ISI (Inter-Symbol Interference):

$\hat{d}[n-k]$ is the past decided bit value, $c_k$ is the DFE tap coefficient. Importantly, DFE does not amplify noise. Because it uses decision results ($\pm 1$), it can accurately cancel only the signal component. In PCIe 6.0, the number of DFE taps can be 10 to 24.

Good question. That's the "error propagation" problem, a fundamental weakness of DFE. If a decision error occurs, subsequent ISI cancellation also becomes incorrect, potentially causing a burst of consecutive errors. In actual SerDes, FEC (Forward Error Correction) covers DFE's error propagation. In channel simulation, whether this error propagation is correctly modeled significantly changes the accuracy of BER prediction.

In practice, Tx-FFE (Feed-Forward Equalizer) is also used. This is a method that "pre-distorts" the signal on the transmitter side, implemented as an FIR filter:

Pre-cursor taps ($a_{-1}$, etc.) and post-cursor taps ($a_1, a_2$, etc.) preemptively cancel ISI. PCIe 5.0 specifies 3 taps, and PCIe 6.0 specifies up to 10 Tx-FFE taps.

A regular eye diagram is created by overlaying waveforms from an actual bit stream. This is called "bit-by-bit simulation." It's intuitive, but to verify a BER of $10^{-15}$, you'd need to stream over $10^{17}$ bits, which would take over 40 minutes even at 64 GT/s.

Statistical eye analysis is a method that calculates probability distributions from the pulse response and directly obtains BER contour lines.

Differentiate the step response $s(t)$ to get the pulse response $p(t)$, then extract the pulse response values at sampling times. You get the main cursor $p_0$ and the values before and after it: $p_{-k}$ (pre-cursor ISI) and $p_k$ (post-cursor ISI).

Each ISI term is weighted by a random bit ($\pm 1$), so the total ISI sum forms a discrete probability distribution. For NRZ with N ISI terms, there are $2^N$ possible combinations. Convolving this probability distribution with the Gaussian noise distribution gives the BER directly at each voltage and time:

$P_i$ is the occurrence probability of ISI combination $i$, $V_i(t)$ is the received voltage for that combination, $\sigma_n$ is the noise standard deviation. The Q-function is the tail probability of the normal distribution. This calculation finishes in seconds, so even ultra-low BER levels like $10^{-15}$ can be evaluated instantly.

The statistical eye is very accurate within the scope of linear channels + linear equalizers. However, DFE error propagation and nonlinear effects like slicer thresholds in PAM4 become approximations. In practice, it's common to explore a large design space using the statistical eye and then perform precise verification of final candidates with bit-by-bit simulation.

COM is a unified metric for channel quality defined in IEEE 802.3. It's the equalized eye opening normalized by noise, jitter, and crosstalk, expressed in dB:

$A_{\text{signal}}$ is half the eye opening after DFE+CTLE equalization, $A_{\text{noise}}$ is the effective value of all noise, jitter, residual ISI, and crosstalk. COM > 3dB is a common pass criterion, meaning the target BER (e.g., $10^{-15}$) is met with sufficient margin.

COM calculation requires quite a lot of inputs. Even the main ones include:

That's precisely why automation is crucial. The IEEE 802.3 committee publishes official MATLAB scripts (COM calculation code), which have become the de facto industry standard. Vendor tools also implement this algorithm. The key point is that in COM calculation, the Tx-FFE and CTLE parameters are automatically optimized to find the best COM value.