Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paul se #6

Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open

Paul se #6

wants to merge 30 commits into from

Conversation

poyupaulchen
Copy link
Collaborator

@poyupaulchen poyupaulchen commented Jan 12, 2023

  • Fixed bug for ns-nnsp.a.
  • Update workable SE for EVB.

To run SE on EVB, follow the steps below:
From the nnsp/evb/ directory:

  1. make clean
  2. make NNSP_MODE=se GUI_ENABLE=1
  3. make deploy Prepare two USB cables. Ensure your board is connected via both the JLINK USB port and the audio USB port. Then turn on the power on EVB.
  4. make view will provide SWO output as the device is running, showing
    predicted slots/intents etc.
  5. On your cmd, type
    $ python ../python/tools/audioview_se.py --tty=/dev/tty.usbmodem1234561
    • You should see a GUI popping out.
      • On your GUI, prress record to start recording and stop to stop recording.
    • You might need to change the option --tty depending on your OS.
  6. Check the two recording files under the folder nnsp/evb/audio_result/:
    • audio_raw.wav: the raw PCM data recorded from your mic.
    • audio_se.wav: the enhanced speech.


#include <arm_math.h>

#define RUN_SPEED_TESTING 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you want this enabled by default?

.length = LEN_STFT_HOP * sizeof(int16_t),
.dType = uint8_e,
.description = msg_store,
.cmd = write_cmd,
.buffer = {.data = (uint8_t *) g_in16AudioDataBuffer, // point this to audio buffer
.dataLength = LEN_STFT_HOP * sizeof(int16_t)}};
// Block sent to PC for computation
dataBlock computeBlock = {
dataBlock computeBlock = { // this block is useless here actually
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an RPC call for this (but I just realized I don't have example code for it, will add): ns_rpc_data_fetchBlockFromPC(&block)


#ifdef DEF_GUI_ENABLE
static char msg_store[30] = "Audio16bPCM_to_WAV";
char msg_compute[30] = "CalculateMFCC_Please";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This string can be anything you want, as long as it matches your python script.

#endif
};

const ns_power_config_t ns_lp_audio = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can delete this I think


detected = NNSPClass_exec(pt_nnsp, pcmbuf_chunk);
for (int i = 0; i < SAMPLES_FRM_NNCNTRL_CLASS; i++)
se_output[i] = pt_nnsp->pt_se_out[i];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memcpy?

for (i = 0; i < ps->hop; i++)
output[i] = (int16_t) MIN(MAX(ps->odataBuffer[i], INT16_MIN), INT16_MAX);;

for (i = 0; i < ps->len_win - ps->hop; i++)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memcpy?

}

pt_out = ps->odataBuffer + ps->len_win - ps->hop;
for (i = 0; i < ps->hop; i++)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memset?

spec_c = np.loadtxt('dump/file_feat_c', dtype=int)
fname = "file_feat"
spec_py = np.loadtxt(f'dump/{fname}_py', dtype=int)
spec_c = np.loadtxt(f'C:\\Users\\Paul Chen\\Documents\\nnsp_sol\\nnsp_simu\\dump\\{fname}_c', dtype=int)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably want a less personal file path :)

*/
static int16_t pcmbuf_chunk[SAMPLES_FRM_NNCNTRL_CLASS];
NNSPClass *pt_nnsp = (NNSPClass*) pt_inst->pt_nnsp;
int16_t detected = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is throwing a compile warning ("set but not used")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants