Ripple Sounds
Introduction
Ripple sounds are temporal and spectral modulated sounds (or noise). They mimic some aspects of vowels. They are often used in experiments for cochlear implant users.
There are basically two methods for creating ripple sounds, which are described in detail below:
- the FFT-iFFT method
- the band-filter method
- the Harmonic Complex method
FFT-iFFT method
The FFT-iFFT method for creating rippled sounds begins by generating two copies of a broad band signal, often noise. Each copy is temporally modulated in the time domain with a sine and a cosine of the same frequency, creating two orthogonal components. Next, the FFT is applied to both modulated signals to obtain their spectra. In the spectral domain, these spectra are modulated by multiplying them sine and cosine functions. The modulated spectra are then transformed back to the time domain using the inverse FFT. The two time-domain signals are either summed or subtracted, which results in ripples that move upward or downward in frequency. Finally, these ripples are scaled by a modulation depth factor (ranging from 0 to 1) and added to the original sound, controlling the intensity of the ripples in the resulting audio.
Below is an example of an implementation in Matlab. It is based on a broadband signal consisting of pink noise. The input parameters are:
| Term | Description |
|---|---|
| t | time domain array in seconds |
| octaves | frequency domain array in octaves above 100 Hz (half as many points as t) |
| ripples_per_sec | the ripple velocity in the time domain |
| phi | a phase that can be added to the time modulation |
| ripples_per_octave | the ripple density in the frequency domain |
| rippleType | determines if the ripple is ascending or descending |
| modulationDepth | half the amplitude of the modulation |
The variables that are sin modulated are denoted by the prefix 'sin_', the cos modulated by the prefix 'cos_'.
The variables in the time domain are denoted by the suffix '_td' and variables in the frequency domain by the suffix '_fd'.
% Calculate the number of samples and time vector
n = length(signal);
half_n = floor(n / 2);
t = (1:n)/sampleRate;
f = sampleRate * (1:half_n) / n + eps;
octaves = log2(f/100); % octave above 100 Hz
% Generate an array with pink noise
broadBandSignal_td = pinknoise(n);
% Create modulation for time domain
sin_modulation_td = sin(2 * pi * ripples_per_sec * t + phi);
cos_modulation_td = cos(2 * pi * ripples_per_sec * t + phi);
%apply time modulation to noise,
sin_modulatedSignal_td = sin_modulation_td .* broadBandSignal_td;
cos_modulatedSignal_td = cos_modulation_td .* broadBandSignal_td;
% perform fft
sin_modulatedSignal_fd = fft(sin_modulatedSignal_td);
cos_modulatedSignal_fd = fft(cos_modulatedSignal_td);
% Create modulation for frequency domain
sin_modulation_fd = sin(2 * pi * ripples_per_octave * octaves);
cos_modulation_fd = cos(2 * pi * ripples_per_octave * octaves);
% Extend the modulation functions for ifft compatibility by adding the mirrored (fliplr) signal
extended_sin_modulation_fd = [sin_modulation_fd, fliplr(sin_modulation_fd)];
extended_cos_modulation_fd = [cos_modulation_fd, fliplr(cos_modulation_fd)];
% Apply frequency modulation
sin_rippledSignal_fd = extended_sin_modulation_fd.* sin_modulatedSignal_fd;
cos_rippledSignal_fd = extended_cos_modulation_fd.* cos_modulatedSignal_fd;
% perform ifft
sin_rippledSignal_td = ifft(sin_rippledSignal_fd, 'symmetric');
cos_rippledSignal_td = ifft(cos_rippledSignal_fd, 'symmetric');
% calculate the rippled signal
switch rippleType
case 'ascending'
rippleSignal_td = sin_rippledSignal_td + cos_rippledSignal_td;
case 'descending'
rippleSignal_td = - sin_rippledSignal_td + cos_rippledSignal_td;
end
% Calculate the final rippled stimulus in the time domain
stimulus = broadBandSignal_td + modulationDepth * rippleSignal_td;
N.B. It is important that you call the function 'pinknoise' only once and stored the result in broadBandSignal_time. If you instead call the function pinknoise again in the last line it gives a wrong result, because each call creates a new random array and this will destroy the necessary interference in the last line.
Band filter method
%todo
Harmonic Complex method
The method uses a collection of sin wave carriers with different frequencies (called a Harmonic Complex) with random phases. Unmodulated the the carriers add up to noise. The carriers are modulated in time and frequency, which creates a rippled sound.
In the program it uses 256 logarithmically spaced frequencies for the Harmonic Complex.
This method is mathematically described on page 129 of the thesis of Lidwien Veugen.
% Calculate the number of samples and time vector
n_td = round(duration * sampleRate); % number of points in time domain
n_fd = 256; % Number of points in frequency domain
t = (1:n_td)/sampleRate; % Time vector
f_low = 100; % Lower frequency bound
f_high = 8200; % Upper frequency bound
f = logspace(log10(f_low), log10(f_high), n_fd); % Frequency vector
phi = (rand(1, n_fd) * 2 * pi)'; % Random phase for all frequencies
% Create a meshgrid for frequency and time to properly handle the dimensions
[T, F] = meshgrid(t, f); % T is time, F is frequency
octaves = log2(F/100); % Octaves above 100 Hz
% Create carriers waves with random phases
carriers = sin(2 * pi * F .* T + phi); % Use meshgrid to match dimensions
T_phase = 2 * pi * ripples_per_sec * T;
F_phase = 2 * pi * ripples_per_octave * octaves;
% Create modulation functions for frequency domain
% Determine ripple type
switch rippleType
case 'ascending'
modulation = 1 + modulationDepth * sin(T_phase - F_phase + phase);
case 'descending'
modulation = 1 + modulationDepth * sin(T_phase + F_phase + phase);
otherwise
error('Invalid rippleType. Choose either ''ascending'' or ''descending''.');
end
% Apply modulation to carriers
modulatedCarriers = modulation .* carriers;
% Sum modulated carriers across frequencies (rows)
rippledStimulus = sum(modulatedCarriers, 1);
Ripple functions
In the biofysica toolbox in Gitlab\biofysica\experiment\sound you can find:
- RL_generateRippleFFTtype()
- genripple()
RL_generateRippleFFTtype()
The function RL_generateRippleFFTtype() can create FFT type ripples. The number of frequency bands used by FFT is the samplerate divided by 2 in linear spacing.
genripple()
The function genripple() can make ripple in the FFT method as well as the Band method.
createRippleSpectrogram()
The funcion createRippleSpectrogram() creates the modulation signals for different frequency bands with logarithmic spacing. It can be used by dividing a signal into bands, multiply all the bands by the rippleSpectrogram bands and create the resulting rippled signal by adding all the resulting bands.
Ripple program
In the biofysica toolbox in Gitlab\biofysica\utilities\RL_ripple_stimulus_generator you can find the program RL_ripple_stimulus_generator.m. The program is based on the an implementation of the Strategy Design Patern. The program can create a signal consisting of a left and right part. The left or right parts can be noise or any ripple sound. The rippled sounds can be ascending or descending and can be restricted to a frequency band.
References
% todo