>
>
> We recently designed an 8-channel complex waveform generator. Each output
> stage is composed of a DAC, a lowpass filter, an output amplifier, a test
> relay, and an output connector. It's this one:
>
> http://www.highlandtechnology.com/DSS/V346DS.html
>
> You can see the gold output connectors, and the relays are hiding just
> behind the front panel.
>
> The harmonic distortion seemed a bit high, in the -40 dBc range at 32 MHz
> and max level output. We were poking around with a spectrum analyzer and
> happened to do a 0-3 GHz sweep and lo, a big line at about 1 GHz.
> Something's oscillating!
>
> Cut to the bottom line: the eight output amps, 1.5 GHz current-mode
> opamps, are individually stable, but oscillate together. Futzing with some
> amps may affect the outputs of others, several channels away. And the
> ensemble oscillations have multiple stable modes, including the occasional
> "off."
>
> What's happening is that the front panel is electromagnetically resonating
> in a fundamental violin-string mode (peak swing in the middle) at about 1
> GHz, and couples pretty well into all the output stages; no doubt the
> relays are helping. A few well-placed capacitors fix the problem. It took
> a while to figure this out.
>
> So the observation is: when something goes wrong, there are a number of
> likely causes. Here, they were channel-channel trace couplings, Vcc
> coupling, amplifier loop stability, pad-plane parasitic capacitance, plain
> rotten opamps, stuff like that. But a complex system has many possible,
> convoluted causalities other than the obvious ones. Suppose there are a
> billion possible interactions, not unreasonable for a system with hundreds
> of themselves-complex parts, all close and well-coupled and interacting at
> frequencies like this. Suppose most of those failure modes [1] are wildly
> improbable, like one chance in a billion of ever happening.
>
> 1e9 * 1e-9 = 1
>
> The final solution was wildly improbable. If suggested as a cause, one
> would be tempted to say "no, that's just too bizarre." It was probable
> that the actual problem *was* wildly improbable.
>
> This sort of thing happens all the time in our business, in hardware and
> software. Insanely unlikely insanely complex things happen, because there
> are potentially so many of them. That makes it fun to track them down.
>
> John
>
>
> [1] "failure mode" being a subjective thing. I think a 1 GHz oscillation
> is a failure because I don't want one. For all I know, the circuit may be
> proud of itself for pulling this off.