Tracking Spatial Sound

On this page I am posting some video and picture examples coming from my current research into tracking spatial sound using ambisonic technology. I am preparing a paper right now for SMC 2016 where some of these figures are taken from, and the videos are supplementary.

The figure below shows the impact of the ambisonic order (i.e. number of microphones on a sphere) on the spatial resolution.

ambiRenderMultiple

This last image shows an ambisonic order of 100. This would mean (100 +1)^2 = 10201 microphones on a sphere. This single image took almost 3 hours to compute on my mid 2010, 17-inch, MacBook Pro:

ambi_100Order-001

This is the path of a sine wave that was encoded into 1st, 3rd, and 7th order ambisonic systems, which I am trying to track:
Trajectory

 

1st Order rendering of the spherical data onto a 2D plane (x = azimuth; y = elevation):

1st Order rendering with tracked centroid overlay:

 

1st Order rendering showing detected blobs with centroid overlay:

 

3rd Order rendering of the spherical data onto a 2D plane (x = azimuth; y = elevation):

 

3rd Order rendering with tracked centroid overlay:

 

3rd Order rendering showing detected blobs with centroid overlay:

 

7th Order rendering of the spherical data onto a 2D plane (x = azimuth;  y = elevation):

 

7th Order rendering with tracked centroid overlay:

 

7th Order rendering showing detected blobs with centroid overlay:

 

The following images are the results of the tracker (a Kalman filter) trying to estimate the spatial trajectories for each ambisonic order. As expected, the hight the order, the better the final tracking result:

1st Order:

The orientation of the sphere with the tracking is not the same as the sphere with the original path shown above. I will upload updated images once I fix this issue.

trackingDataOut_1Order-00000

3rd Order:

trackingDataOut_3Order-00000

7th Order:

trackingDataOut_7Order-00000

 

Finally, some  excerpts of a real-world example of a 1st order recording of Luigi Nono’s A Pierre, Dell’azzurro silenzio, inquietum. 

50-60 seconds:

with centroid overlay:

showing detected blobs with centroids:

 

This picture shows the locations of the blob centroids (50-60 secs):

blobCentroid_Nono_50-60sec

And here is the result of using the Kalman filter on the data. The orientation of the sphere with the tracking is not the same as the sphere with the centroid overlay. I will upload updated images once I fix this issue.

trackingDataOut_NONO50-60

120-130 seconds:

showing detected blobs with centroids:

showing detected blobs with centroids:

This picture shows the locations of the blob centroids (120-130 secs):

blobCentroid_Nono_120-130sec

And here is the result of using the Kalman filter on the data. The orientation of the sphere with the tracking is not the same as the sphere with the centroid overlay. I will upload updated images once I fix this issue.

trackingDataOut_NONO120-130

300-310 seconds:

showing detected blobs with centroids:

showing detected blobs with centroids:

This picture shows the locations of the blob centroids (300-310 secs):

blobCentroid_Nono_300-310sec

And here is the result of using the Kalman filter on the data. The orientation of the sphere with the tracking is not the same as the sphere with the centroid overlay. I will upload updated images once I fix this issue.

trackingDataOut_NONO300-310