Optimal Design of Delta-Sigma Modulators

Control theory sometimes gives a powerful tool for solving problems in signal processing. LMI (Linear Matrix Inequality) is one of them. Today, I'd like to introduce my paper (preprint PDF) on LMI application to delta-sigma modulation:

M. Nagahara and Y. Yamamoto, Frequency Domain Min-Max Optimization of Noise-Shaping Delta-Sigma Modulators, IEEE Trans. on Signal Processing, Vol. 60, No. 6, pp. 2828-2839, 2012.

What is delta-sigma modulation? The Wikipedia entry reads:

Delta-sigma (ΔΣ; or sigma-delta, ΣΔ) modulation is a method for encoding analog signals into digital signals or higher-resolution digital signals into lower-resolution digital signals. The conversion is done using error feedback, where the difference between the two signals is measured and used to improve the conversion. The low-resolution signal typically changes more quickly than the high-resolution signal and it can be filtered to recover the high-resolution signal with little or no loss of fidelity. 

To filter out the quantization noise from low-resolution quickly-changing signals, it is essential to push away the quantization noise out of the frequency band of the original signal. In other words, it should be sufficiently attenuated the noise transfer function (NTF) from the quantization noise to the modulator output (before the lowpass filter).

Although a delta-sigma modulator ia a highly nonlinear system, designing is a linear problem like filter design; shape the frequency response of NTF.

Conventionally, NTF is shaped such that the squared norm of the magnitude (i.e., H² norm) of NTF on the frequency band of interest. As I noted in my past entry that an H²-based system shows very nice performance for almost all frequencies but is fragile for a frequency, say f₀ [Hz] (see the picture below).

To avoid this, we introduced the H^∞ norm to uniformly attenuate the magnitude on a frequency band. A difficulty is that this problem is not a standard H^∞ problem since it does not optimize over the whole frequency band [0,π) but on a subset [0,Ω), Ω<π. You cannot use the standard hinfsyn
command in MATLAB. How to solve it?

The solver is found in control theory, generalized KYP (GKYP) lemma. This solves our problem via LMI (linear matrix inequality).

The picture below shows two plots: the Bode magnitude plot of NTF designed by GKYP (red line) and by H²-based zero optimization (blue line). The magnitude by GKYP is uniformly attenuated over the low frequency range, while that by H² shows peaks in this band. The difference between the two maximal magnitudes at the frequency ω = π/32 is approximately 11.2 (dB), and the difference at low frequencies is about 12.4 (dB).

The paper is available here.
MATLAB codes are available here.
GKYP for signal processing is also discussed in this article.

A related blog entry is here.

Causality in signal processing

Often, signal processing theorems give a non-causal filter. For example, see the following paper:

M. Unser, A. Aldroubi, and M. Eden,
B-spline Signal Processing: Part II---Efficient Design and Applications,
Signal Processing, IEEE Transactions on , vol.41, no.2, pp.834-848, Feb 1993

A non-causal filter as above is no problem in image processing. However, in a real time system, in particular, in a feedback loop, non-causality should cause a problem of implementation. The simplest way to avoid this is truncation of the non-causal part of the filter impulse response, say, {f(n): n<0}. This absolutely degrades the filter performance, and the second simplest is to truncate {f(n): n<-N} (often with a window) and shift the residual by N.

Recently, much more sophisticated methods are proposed, via constraint least squares, which is related to H² optimization:

M. Unser and M. Eden,
FIR approximations of inverse filters and perfect reconstruction filter banks,
Signal Processing, Vol. 36, No. 2, pp. 163-174, 1994.

and via H^∞ optimization:

M. Nagahara and Y. Yamamoto,
H∞ optimal approximation for causal spline interpolation,
Signal Processing, Vol. 91, No. 2, pp. 176-184, 2011.

As I mentioned in a previous entry that H^∞ optimization leads to robustness against signal uncertainty. If you like robustness (or if you don't like risk-taking behavior), then you must choose H^∞! You can use MATLAB codes for the H^∞ design here.

H2 versus H∞

In signal processing, H² and H^∞ norms are most important measures for system performance. In this entry, I try to explain the difference between them in view of filter design.

For a stable and causal system F(z), the H² norm is defined by

This is the average (or root-mean-square) value of the magnitude of the frequency response of F(z).

On the other hand, the H^∞ norm is defined by

This is the maximum value of the magnitude of the frequency response of F(z).

Now, let us consider the following inverse-filtering problem:

Given a filter H(z), find another filter K(z) such that H(z)K(z) ≈ 1.

For this problem, we can use the above two norms.  By the H² norm, the problem is formulated as

(H2) Find stable and causal K(z) that minimizes || H(z)K(z)-1 ||₂

By the H^∞ norm, the problem is also formulated as

(Hinf) Find stable and causal K(z) that minimizes || H(z)K(z)-1 ||_∞

The difference is that (H2) tries to minimize the average magnitude of the error system H(z)K(z)-1 while (Hinf) tries to minimize the maximum magnitude.  The following picture shows an example of the two designs.

The blue line is the magnitude of the frequency response of the error H(z)K(z)-1 by the H² design. The error is very small at almost all frequencies but is very large around a frequency, say f₀. On the other hand, the red line is the error magnitude by H^∞ design, which is uniformly small since the H^∞ optimization is a minmax one that tries to make the response as flat as possible. The average value by H² is smaller than by H^∞ and the maximum value by H^∞ is smaller than by H².

By this picture, the difference is clear.

H²-designed filter shows very nice performance for almost all frequencies but is fragile for the frequency f₀. Hence, H² is the better if it is quite certain that the input signals do not contain frequency components around f₀.  On the other hand, H^∞-designed filter guarantees a certain error level for all frequencies. In other words, H^∞ optimization leads to robustness against uncertainty in the frequency components of input signals.

For details, please check this article, and matlab codes for the article.