Ripple Sounds

Introduction

Ripple sounds are temporal and spectral modulated sounds (or noise). They mimic some aspects of vowels. They are often used in experiments for cochlear implant users.

There are basically two methods for creating ripple sounds, which are described in detail below:

• the FFT-iFFT method
• the band-filter method
• the Harmonic Complex method

FFT-iFFT method

The FFT-iFFT method for creating rippled sounds begins by generating two copies of a broad band signal, often noise. Each copy is temporally modulated in the time domain with a sine and a cosine of the same frequency, creating two orthogonal components. Next, the FFT is applied to both modulated signals to obtain their spectra. In the spectral domain, these spectra are modulated by multiplying them sine and cosine functions. The modulated spectra are then transformed back to the time domain using the inverse FFT. The two time-domain signals are either summed or subtracted, which results in ripples that move upward or downward in frequency. Finally, these ripples are scaled by a modulation depth factor (ranging from 0 to 1) and added to the original sound, controlling the intensity of the ripples in the resulting audio.

Below is an example of an implementation in Matlab. It is based on a broadband signal consisting of pink noise. The input parameters are:

Term	Description
t	time domain array in seconds
octaves	frequency domain array in octaves above 100 Hz (half as many points as t)
ripples_per_sec	the ripple velocity in the time domain
phi	a phase that can be added to the time modulation
ripples_per_octave	the ripple density in the frequency domain
rippleType	determines if the ripple is ascending or descending
modulationDepth	half the amplitude of the modulation

The variables that are sin modulated are denoted by the prefix 'sin_', the cos modulated by the prefix 'cos_'.
The variables in the time domain are denoted by the suffix '_td' and variables in the frequency domain by the suffix '_fd'.

    % Calculate the number of samples and time vector    
    n             = length(signal);  
    half_n        = floor(n / 2); 
    t             = (1:n)/sampleRate;
    f             = sampleRate * (1:half_n) / n + eps; 
    octaves       = log2(f/100); % octave above 100 Hz

    % Generate an array with pink noise
    broadBandSignal_td = pinknoise(n);

    % Create modulation for time domain
    sin_modulation_td = sin(2 * pi * ripples_per_sec * t + phi);
    cos_modulation_td = cos(2 * pi * ripples_per_sec * t + phi);        

    %apply time modulation to noise, 
    sin_modulatedSignal_td = sin_modulation_td .* broadBandSignal_td;
    cos_modulatedSignal_td = cos_modulation_td .* broadBandSignal_td;

    % perform fft 
    sin_modulatedSignal_fd = fft(sin_modulatedSignal_td);
    cos_modulatedSignal_fd = fft(cos_modulatedSignal_td);

    % Create modulation for frequency domain    
    sin_modulation_fd = sin(2 * pi * ripples_per_octave * octaves);
    cos_modulation_fd = cos(2 * pi * ripples_per_octave * octaves);
    
    % Extend the modulation functions for ifft compatibility by adding the mirrored (fliplr) signal
    extended_sin_modulation_fd = [sin_modulation_fd, fliplr(sin_modulation_fd)]; 
    extended_cos_modulation_fd = [cos_modulation_fd, fliplr(cos_modulation_fd)]; 

    % Apply frequency modulation
    sin_rippledSignal_fd = extended_sin_modulation_fd.* sin_modulatedSignal_fd;
    cos_rippledSignal_fd = extended_cos_modulation_fd.* cos_modulatedSignal_fd;
      
    % perform ifft
    sin_rippledSignal_td =  ifft(sin_rippledSignal_fd, 'symmetric');
    cos_rippledSignal_td =  ifft(cos_rippledSignal_fd, 'symmetric');

    % calculate the rippled signal
    switch rippleType
        case 'ascending'
            rippleSignal_td =   sin_rippledSignal_td + cos_rippledSignal_td;  
        case 'descending'
            rippleSignal_td = - sin_rippledSignal_td + cos_rippledSignal_td;  
    end    

    % Calculate the final rippled stimulus in the time domain          
    stimulus = broadBandSignal_td + modulationDepth * rippleSignal_td;    

N.B. It is important that you call the function 'pinknoise' only once and stored the result in broadBandSignal_time. If you instead call the function pinknoise again in the last line it gives a wrong result, because each call creates a new random array and this will destroy the necessary interference in the last line.

Band filter method

%todo

Harmonic Complex method

The method uses a collection of sin wave carriers with different frequencies (called a Harmonic Complex) with random phases. Unmodulated the the carriers add up to noise. The carriers are modulated in time and frequency, which creates a rippled sound.

In the program it uses 256 logarithmically spaced frequencies for the Harmonic Complex.

This method is mathematically described on page 129 of the thesis of Lidwien Veugen.

Here is a implementation in Matlab code:

    % Calculate the number of samples and time vector    
    n_td          = round(duration * sampleRate);   % number of points in time domain
    n_fd          = 256;                            % Number of points in frequency domain
    t             = (1:n_td)/sampleRate;            % Time vector
    f_low         = 100;                            % Lower frequency bound
    f_high        = 8200;                           % Upper frequency bound
    f             = logspace(log10(f_low), log10(f_high), n_fd); % Frequency vector
    phi           = (rand(1, n_fd) * 2 * pi)';         % Random phase for all frequencies

    % Create a meshgrid for frequency and time to properly handle the dimensions
    [T, F] = meshgrid(t, f); % T is time, F is frequency

    octaves       = log2(F/100);                    % Octaves above 100 Hz

    % Create carriers waves with random phases
    carriers = sin(2 * pi * F .* T + phi);

    T_phase = 2 * pi * ripples_per_sec    * T;
    F_phase = 2 * pi * ripples_per_octave * octaves;
    
    % Create modulation functions for frequency domain
    switch rippleType
        case 'ascending'
           modulationMaxtrix =  sin(T_phase - F_phase + phase);
        case 'descending'
           modulationMaxtrix =  sin(T_phase + F_phase + phase);
    end
    
    modulation = 1 + modulationDepth * modulationMaxtrix;

    % Apply modulation to carriers
    modulatedCarriers = modulation .* carriers;

    % Sum modulated carriers across frequencies (rows)
    stimulus = sum(modulatedCarriers, 1); 

Ripple functions

In the biofysica toolbox in Gitlab\biofysica\experiment\sound you can find:

• RL_generateRippleFFTtype()
• genripple()

RL_generateRippleFFTtype()

The function RL_generateRippleFFTtype() can create FFT type ripples. The number of frequency bands used by FFT is the samplerate divided by 2 in linear spacing.

genripple()

The function genripple() can make ripple in the FFT method as well as the Band method.

createRippleSpectrogram()

The funcion createRippleSpectrogram() creates the modulation signals for different frequency bands with logarithmic spacing. It can be used by dividing a signal into bands, multiply all the bands by the rippleSpectrogram bands and create the resulting rippled signal by adding all the resulting bands.

Ripple program

In the biofysica toolbox in Gitlab\biofysica\utilities\RL_ripple_stimulus_generator you can find the program RL_ripple_stimulus_generator.m. The program is based on the an implementation of the Strategy Design Patern. The program can create a signal consisting of a left and right part. The left or right parts can be noise or any ripple sound. The rippled sounds can be ascending or descending and can be restricted to a frequency band.

References

% todo