Topics Map > Resources for collaborators
Audapter: Beyond the manual
Public Audapter manual
Here's a link to the public Audapter manual, without custom SMNG functionality: https://sites.bu.edu/guentherlab/files/2022/09/AudapterManual_2.1.5.pdf
2D Formant Perturbation
More information on 2d formant perturbation is available in the Programming a 2D Audapter experiment KB document.
High Frequency Filters (aka highFs)
Status
Programming completed, merged with master branch of blab-lab/audapter_mex repo. Pull request to main repo in progress.
Functional Changes
You can run Audapter at a higher sampling rate of 24 kHz without artifacting.
Audapter was originally designed to run on a microphone with a hardware sampling rate of 48 kHz, which would then be divided by a "downfactor" of 3, to result in Audapter receiving a 16 kHz signal. This effective sampling rate of 16 kHz was chosen to reduce Audapter's computational load, and leads to Audapter's Nyquist frequency being 8 kHz. If you want to record sounds higher than 8 kHz, you need a higher effective sampling rate, which can easily be done by dividing by a downfactor of 2 instead of 3.
Code Changes
Audapter.cpp uses a Chebyshev II filter with specific values designed to work at an effective sampling rate of 16 kHz. Now, there is another Chebyshev II filter which works at an effective sampling rate of 24 kHz. Depending on the values of p.sr (sampling rate) and p.downFact (downfactor), the appropriate filter is selected.
A boolean variable bRemakeFilter was created to track if the filter needed to be switched after initializing Audapter. If the sampling rate or downfactor variables are changed, Audapter re-evaluates which filter is appropriate.
How to use
Change the Audapter parameter downFact from 3 to 2, if you are using the default sampling rate of 48000.
Heuristics
Status
Merged with master branch of blab-lab/audapter_mex in July 2021.
What are heuristics?
When performing online status tracking (OST), heuristics determine the feature of the signal and the quality of that feature Audapter uses when determining if the OST status should change. For example, the INTENSITY_FALL heuristic, which specifies two arguments, denotes that Audapter will look for the RMS to be below parameter 1 for the duration of parameter 2. Compare with the INTENSITY_RATIO_RISE heuristic, which specifies that when the RMS ratio is above parameter 1 for the duration of parameter 2, the OST status will advance.
Functional changes
In October 2020, bugs in two heuristics were fixed: see pull request here.
In May 2021, work started on several new heuristics. Currently, all prior heuristics were kept the same in both name and functionality. The new heuristics and their functionality appears below:
- INTENSITY_SLOPE_BELOW_THRESH. RMS slope below a certain threshold (param1) for a certain duration (param2).
- INTENSITY_BELOW_THRESH_NEG_SLOPE. RMS below a certain threshold (param1) for a certain duration (param2), all while the RMS slope is a negative value.
- INTENSITY_RATIO_BELOW_THRESH_NEG_SLOPE. RMS ratio below a certain threshold (param1) for a certain duration (param2), all while the RMS slope is negative.
- INTENSITY_RATIO_ABOVE_THRESH_WITH_RMS_FLOOR. RMS ratio above a certain threshold (param1) for a certain duration (param2), all while the RMS is above a very low baseline value of 0.0003. Note: this can reasonably be used in place of INTENSITY_RATIO_RISE in any circumstance.
- INTENSITY_AND_RATIO_ABOVE_THRESH. RMS above a certain threshold (param1), and RMS ratio above a certain threshold (param2) for a certain duration (param3).
- INTENSITY_AND_RATIO_BELOW_THRESH. RMS below a certain threshold (param1), and RMS ratio below a certain threshold (param2) for a certain duration (param3). Added in version b2.3.
- INTENSITY_RATIO_SLOPE_ABOVE_THRESH. RMS ratio slope above a certain threshold (param1) for a certain duration (param2).
- INTENSITY_RATIO_SLOPE_BELOW_THRESH. RMS ratio slope below a certain threshold (param1) for a certain duration (param2).
Code changes
The majority of changes took place in ost.cpp and ost.h and are relatively straightforward. Typically code can be copied and pasted from an existing heuristic and modified as needed.
The intensity_ratio_slope heuristics required additional changes in Audapter.cpp and audapter.h, so that the RMS ratio could be stored over time. This was necessary for calculating a slope. RMS ratio and RMS ratio slope were now being stored over time, those values could also be sent back to Matlab during an Audapter('getData') call. All of the data that gets sent to Matlab is packaged in Audapter.cpp in the data_recorder variable. The AudapterIO.m function in Matlab handles how that data is rolled out and repacked into specific fields in data.mat. The new field names are data.rms_ratio and data.rms_ratio_slope to match the data.rms_slope convention.
INTENSITY_AND_RATIO_ABOVE_THRESH was the first heuristic to use 3 input parameters. ost.cpp already had the framework for supporting a heuristic with 3 input parameters (instead of the normal 2), but some minor changes to ost.cpp were made to fully support it.
How to use
Anywhere that uses heuristics can use the new heuristics.
Formant clamping
Status
Merged from taimComp branch to master branch of blab-lab/audapter_mex in August 2022, coinciding with version b2.2.
Functional changes
Users can specify what the output signal's F1 and F2 values should be during a range of OST statuses.
New parameters:
- bclampformants. Default: 0. A binary switch (0 or 1) specifying if clamping should be used. Similar to bFormantShift.
- clamp_osts. Default: [0 0]. A 1x2 array of integers, where the first number specifies the OST status when clamping should begin, and the second number specifies the OST status when clamping should stop and veridical feedback returns.
- clampf1. Default: 1x2048 array of zeros. Numeric array of size 1x2048 which specifies the F1 value of the output signal during the clamp (the period between the clamp OSTs).
- clampf2. Default: 1x2048 array of zeros. Numeric array of size 1x2048 which specifies the F2 value of the output signal during the clamp (the period between the clamp OSTs).
Parameters are set in Matlab the same way as existing parameters, with the 'setParam' action: Audapter('setParam', 'clampf1', arrayWithValues, 1). getAudapterDefaultParams.m has been updated to use the above default values. (Defaults to not using clamping functionality.)
How to use
Let's say you want your F1 values to be increasing from [501 502 503 ... 600] and your F2 values to be decreasing from [1999 1998 1997 ... 1900] during OST statuses 2 and 3. You would need to set bclampformants to 1 and set clamp_osts to [2 4]. (Note that clamping stops as soon as the 2nd-indexed clamp_osts value is reached.) Since clampf1 and clampf2 need to be 2048 units long and your specified formant arrays are only 100 units long, you would need to pad the end of each of those arrays with zeros. (The SMNG function clamp_padwithzeros.m accomplishes this.)
Other functional behavior
Each value in clampf1/clampf2 represents what the output signal's formant value should be on a single Audapter frame. Frames are typically 2ms long (always 2ms in SMNG experiments). Thus, in the above example, the user has only specified what should happen during the first 100 frames (200 milliseconds) of the clamp. If the 2nd-indexed clamp_osts value is reached before the 101st frame, then the clamp will simply end early. Meaning, if OST status 4 is reached 21 frames after OST status 2, then the final clamped F1 and F2 values would be [520] and [1980] from frame #20, and on that same frame #21, veridical feedback would be presented instead.
If instead OST status 4 is reached on frame #151, what happens between frames #101 and #150? The last non-zero value in each of clampf1 and clampf2 is repeated indefinitely. So, F1 would be [600] and F2 would be [1900] for 51 frames in a row (frames #100-150).
Let's say you're alternating between perturbation and non-perturbation trials. How should you accomplish this? You have many methods: you can set bclampformants to 0, or you can set clamp_osts to [0 0], or you can set clampf1 and clampf2 to zeros(1, 2048). The final option was chosen for taimComp, since clampf1 and clampf2 values were already being altered on each trial.
Code changes
Audapter.cpp and Audapter.h were changed to add the above parameters, which are all relatively straightforward. A new section was added to the p.bShift block in Audapter.cpp which deals with all of the perturbation types. An Audapter-internal variable clampIx iterates through clampf1 and clampf2 to keep track of which formant value to output. If the clamp goes longer than expected (ie, clampIx would index into a 0 in clampf1), clampIx instead just keeps pointing at the last non-zero value.
The constant integer maxNClampFrames is initialized in Audapter.h to store the max length of a clamp (2048).
Limitations
You cannot clamp only one formant while getting veridical feedback for the other formant. This seems doable with code changes; you'd probably mostly need to rearrange different chunks within the p.bShift block.
You cannot specify a clamp longer than 2048 frames (~4.1 seconds). This number was totally arbitrary and could be very simply made larger if necessary. You would do this by initializing maxNClampFrames to a different value, and padding clampf1 and clampf2 to that commensurate length. Note that if your clamped duration ends up going longer than 2048 frames, it will still work as specified above and will not crash. (For example if you specify values for the first 1000 frames, but the time between clamp OSTs is 3000 frames.)
You must pad clampf1 and clampf2 to be 2048 units long. This was a programmer limitation, not a C++ limitation. There are ways to initialize arrays to a variable length, but I couldn't get it to work properly.