Beamformer-based Sound Localization

To maximize the output energy of a delay-and-sum beamformer in order to find the maximum value of the output of a beamformer steered in all possible directions. Using the Reliability Weighted Phase Transform (RWPHAT) method, The output energy of M-microphone delay-and-sum beamformer is

${\displaystyle E=K+2\sum _{{m}_{1}=1}^{M-1}\sum _{{m}_{2}=0}^{{m}_{1}-1}{{R}^{\text{RWPHAT}}}_{i,j}\left({\tau }_{{m}_{1}}-{\tau }_{{m}_{2}}\right)}$

Where E indicates the energy, and K is a constant, ${\displaystyle {{R}^{\text{RWPHAT}}}_{i,j}\left({\tau }_{{m}_{1}}-{\tau }_{{m}_{2}}\right)}$ is the microphone pairs cross-correlation defined by Reliability Weighted Phase Transform:

${\displaystyle {{R}^{\text{RWPHAT}}}_{i,j}\left(\tau \right)=\sum _{k=0}^{L-1}{\frac {{\zeta }_{i}\left(k\right){X}_{i}\left(k\right){\zeta }_{j}\left(k\right){{X}_{j}}^{*}\left(k\right)}{\left|{X}_{i}\left(k\right)\right|\left|{X}_{j}\left(k\right)\right|}}{e}^{j2\pi k\tau /L}}$

the weighted factor ${\displaystyle {{\zeta }^{n}}_{i}\left(k\right)}$reflect the reliability of each frequency component, and defined as the Wiener Filter gain ${\displaystyle {{\zeta }^{n}}_{i}\left(k\right)={\frac {{{\xi }^{n}}_{i}\left(k\right)}{{{\xi }^{n}}_{i}\left(k\right)+1}}}$, where ${\displaystyle {{\xi }^{n}}_{i}\left(k\right)}$ is an estimate of a prior SNR at ${\displaystyle i^{th}}$ microphone, at time frame ${\displaystyle n}$, for frequency ${\displaystyle k}$, computed using the decision-directed approach.^[8]

The ${\displaystyle x_{m_{n}}}$ is the signal from ${\displaystyle {m}^{th}}$ microphone and ${\displaystyle {\tau }_{{m}_{n}}}$ is the delay of arrival for that microphone. The more specific procedure of this method is proposed by Valin and Michaud^[9]

The advantage of this method is that it detects the direction of the sound and derives the distance of sound sources. The main drawback of the beamforming approach is the imperfect nature of sound localization accuracy and capability, versus the neural network approach, which uses moving speakers.

Acoustic Vector Array

Picture of a 3D Acoustic Vector Sensor (air)

• Contains three orthogonally placed acoustic particle velocity sensors (shown as X, Y and Z array) and one omnidirectional acoustic microphone (O).

• Commonly used both in air ^[10] and underwater.

• Can be used in combination with the Offline Calibration Process^[11] to measure and interpolate the impulse response of X, Y, Z and O arrays, to obtain their steering vector.

A sound signal is first windowed using a rectangular window, then each resulting segment signal is created as a frame. 4 parallel frames are detected from XYZO array and used for DOA estimation. The 4 frames are split into small blocks with equal size, then the Hamming window and FFT are used to convert each block from a time domain to a frequency domain. Then the output of this system is represented by a horizontal angle and a vertical angle of the sound sources which is found by the peak in the combined 3D spatial spectrum.

The advantages of this array, compared with past microphone array, are that this device has a high performance even if the aperture is small, and it can localize multiple low frequency and high frequency wide band sound sources simultaneously. Applying an O array can make more available acoustic information, such as amplitude and time difference. Most importantly, XYZO array has a better performance with a tiny size.

The AVS is one kind of collocated multiple microphone array, it makes use of a multiple microphone array approach for estimating the sound directions by multiple arrays and then finds the locations by using reflection information such as where the direction is detected where different arrays cross.

Motivation of the Advanced Microphone array

Sound reflections always occur in an actual environment and microphone arrays^[12] cannot avoid observing those reflections. This multiple array approach was tested using fixed arrays in the ceiling; the performance of the moving scenario still need to be tested.

Learning how to apply Multiple Microphone Array

Angle uncertainty (AU) will occur when estimating direction, and position uncertainty (PU) will also aggravate with increasing distance between the array and the source. We know that:

${\displaystyle PU\left(r\right)={\frac {\pm AU}{360}}\times 2\pi \times r}$

Where r is the distance between array center to source, and AU is angle uncertainly. Measurement is used for judging whether two directions cross at some location or not. Minimum distance between two lines:

${\displaystyle dist\left(dir_{1},dir_{2}\right)={\frac {\left({\overrightarrow {v_{1}}}\times {\overrightarrow {v_{2}}}\right)\times {\overrightarrow {p_{1}p_{2}}}}{\left|{\overrightarrow {v_{1}}}\times {\overrightarrow {v_{2}}}\right|}}}$

where${\displaystyle dir_{1}}$and ${\displaystyle dir_{2}}$ are two directions, ${\displaystyle v_{i}}$are vectors parallel to detected direction, and ${\displaystyle p_{i}}$are the position of arrays.

If

${\displaystyle dist(dir_{1},dir_{2})<abs(PU_{1}(r_{1}))+abs(PU_{2}(r_{2}))}$

Two lines are judged as crossing. When two lines are crossing, we can compute the sound source location using the following:

${\displaystyle \mathrm {POS} _{source}={\frac {\left(\mathrm {POS} _{1}\times w_{1}+\mathrm {POS} _{2}\times w_{2}\right)}{w_{1}+w_{2}}}}$

${\displaystyle \mathrm {POS} _{source}}$is the estimation of sound source position, ${\displaystyle \mathrm {POS} _{n}}$ is the position where each direction intersect the line with minimum distance, and ${\displaystyle w_{n}}$ is the weighted factors. As the weighting factor ${\displaystyle w_{n}}$, we determined use ${\displaystyle PU}$ or ${\displaystyle r}$ from the array to the line with minimum distance.