轉(zhuǎn)載自:http://blog.csdn.net/dxpqxb/article/details/7928591
為什么需要聲學(xué)回聲消除呢?在一般的VOIP軟件或視頻會(huì)議系統(tǒng)中,假設(shè)我們只有A和B兩個(gè)人在通話,首先,A的聲音傳給B,B然后用喇叭放出來,而這時(shí)B的MIC呢則會(huì)采集到喇叭放出來的聲音,然后傳回給A,如果這個(gè)傳輸?shù)倪^程中時(shí)延足夠大,A就會(huì)聽到一個(gè)和自己剛才說過的話一樣的聲音,這就是回聲,聲學(xué)回聲消除器的作用就是在B端對B采集到的聲音進(jìn)行處理,把采集到聲音包含的A的聲音去掉再傳給A,這樣,A就不會(huì)聽到自己說過的話了。
聲學(xué)回聲消除的原理我就不說了,這在網(wǎng)上有很多文檔,網(wǎng)上缺少的是實(shí)現(xiàn),所以,我在這把一個(gè)開源的聲學(xué)回聲消除器介紹一下,希望對有些有人用,如果有人知道怎么把這消除器用的基于實(shí)時(shí)流的VOIP軟件中,希望能一起分享一下。
這個(gè)聲學(xué)回聲消除器是一個(gè)著名的音頻編解碼器speex中的一部分,1.1.9版本后的回聲消除器才起作用,以前版本的都不行,我用的也是這個(gè)版本,測試表明,用同一個(gè)模擬文件,它有效果比INTEL IPP庫4.1版中的聲學(xué)回聲消除器的還要好。
先說編譯。首先,從www.speex.org上下載speex1.1.9的源代碼,解壓,打開speex\win32\libspeex中的libspeex.dsw,這個(gè)工作區(qū)里有兩個(gè)工程,一個(gè)是 libspeex,另一個(gè)是libspeex_dynamic。然后,將libspeex中的mdf.c文件添加到工程libspeex中,編譯即可。
以下是我根據(jù)文檔封裝的一個(gè)類,里面有一個(gè)測試程序: //file name: speexEC.h
#ifndef SPEEX_EC_H
#define SPEEX_EC_H
#include <stdio.h>
#include <stdlib.h>
#include "speex/speex_echo.h"
#include "speex/speex_preprocess.h"
class CSpeexEC
{
public:
CSpeexEC();
~CSpeexEC();
void Init(int frame_size=160, int filter_length=1280, int sampling_rate=8000);
void DoAEC(short *mic, short *ref, short *out);
protected:
void Reset();
private:
bool m_bHasInit;
SpeexEchoState* m_pState;
SpeexPreprocessState* m_pPreprocessorState;
int m_nFrameSize;
int m_nFilterLen;
int m_nSampleRate;
float* m_pfNoise;
};
#endif
//fine name:speexEC.cpp
#include "SpeexEC.h"
CSpeexEC::CSpeexEC()
{
m_bHasInit = false;
m_pState = NULL;
m_pPreprocessorState = NULL;
m_nFrameSize = 160;
m_nFilterLen = 160*8;
m_nSampleRate = 8000;
m_pfNoise = NULL;
}
CSpeexEC::~CSpeexEC()
{
Reset();
}
void CSpeexEC::Init(int frame_size, int filter_length, int sampling_rate)
{
Reset();
if (frame_size<=0 || filter_length<=0 || sampling_rate<=0)
{
m_nFrameSize =160;
m_nFilterLen = 160*8;
m_nSampleRate = 8000;
}
else
{
m_nFrameSize =frame_size;
m_nFilterLen = filter_length;
m_nSampleRate = sampling_rate;
}
m_pState = speex_echo_state_init(m_nFrameSize, m_nFilterLen);
m_pPreprocessorState = speex_preprocess_state_init(m_nFrameSize, m_nSampleRate);
m_pfNoise = new float[m_nFrameSize+1];
m_bHasInit = true;
}
void CSpeexEC::Reset()
{
if (m_pState != NULL)
{
speex_echo_state_destroy(m_pState);
m_pState = NULL;
}
if (m_pPreprocessorState != NULL)
{
speex_preprocess_state_destroy(m_pPreprocessorState);
m_pPreprocessorState = NULL;
}
if (m_pfNoise != NULL)
{
delete []m_pfNoise;
m_pfNoise = NULL;
}
m_bHasInit = false;
}
void CSpeexEC:DoAEC(short* mic, short* ref, short* out)
{
if (!m_bHasInit)
return;
speex_echo_cancel(m_pState, mic, ref, out, m_pfNoise);
speex_preprocess(m_pPreprocessorState, (__int16 *)out, m_pfNoise);
}
可以看出,這個(gè)回聲消除器類很簡單,只要初始化一下就可以調(diào)用了。但是,要注意的是,傳給回聲消除器的兩個(gè)聲音信號,必須同步得非常的好,就是說,在B端,接收到A說的話以后,要把這些話音數(shù)據(jù)傳給回聲消除器做參考,然后再傳給聲卡,聲卡再放出來,這有一段延時(shí),這時(shí),B再采集,然后傳給回聲消除器,與那個(gè)參考數(shù)據(jù)比較,從采集到的數(shù)據(jù)中把頻域和參考數(shù)據(jù)相同的部分消除掉。如果傳給消除器的兩個(gè)信號同步得不好,即兩個(gè)信號找不到頻域相同的部分,就沒有辦法進(jìn)行消除了。
測試程序:
#define NN 160
void main()
{
FILE* ref_fd, *mic_fd, *out_fd;
short ref[NN], mic[NN], out[NN];
ref_fd = fopen ("ref.pcm", "rb"); //打開參考文件,即要消除的聲音
mic_fd = fopen ("mic.pcm", "rb");//打開mic采集到的聲音文件,包含回聲在里面
out_fd = fopen ("echo.pcm", "wb");//消除了回聲以后的文件
CSpeexEC ec;
ec.Init();
while (fread(mic, 1, NN*2, mic_fd))
{
fread(ref, 1, NN*2, ref_fd);
ec.DoAEC(mic, ref, out);
fwrite(out, 1, NN*2, out_fd);
}
fclose(ref_fd);
fclose(mic_fd);
fclose(out_fd);
}
以上的程序是用文件來模擬回聲和MIC,但在實(shí)時(shí)流中是大不一樣的,在一般的VOIP軟件中,接收對方的聲音并傳到聲卡中播放是在一個(gè)線程中進(jìn)行的,而采集本地的聲音并傳送到對方又是在另一個(gè)線程中進(jìn)行的,而聲學(xué)回聲消除器在對采集到的聲音進(jìn)行回聲消除的同時(shí),還需要播放線程中的數(shù)據(jù)作為參考,而要同步這兩個(gè)線程中的數(shù)據(jù)是非常困難的,因?yàn)樯陨杂行┎煌剑晫W(xué)回聲消除器中的自適應(yīng)濾波器就會(huì)發(fā)散,不但消除不了回聲,還會(huì)破壞原始采集到的聲音,使被破壞的聲音難以分辨。我做過好多嘗試,始終無法用軟件來實(shí)現(xiàn)對這兩個(gè)線程中的數(shù)據(jù)進(jìn)行同步,導(dǎo)致實(shí)現(xiàn)失敗,希望有經(jīng)驗(yàn)的網(wǎng)友們一起分享一下這方面的經(jīng)驗(yàn)。
示例代碼:
Sample code
This section shows sample code for encoding and decoding speech using the Speex API. The commands can be used to encode and decode a file by calling:
% sampleenc in_file.sw | sampledec out_file.sw
where both files are raw (no header) files encoded at 16 bits per sample (in the machine natural endianness).
sampleenc.c
sampleenc takes a raw 16 bits/sample file, encodes it and outputs a Speex stream to stdout. Note that the packing used is NOT compatible with that of speexenc/speexdec.
#include <speex/speex.h>
#include <stdio.h>
#define FRAME_SIZE 160
int main(int argc, char **argv)
{
char *inFile;
FILE *fin;
short in[FRAME_SIZE];
float input[FRAME_SIZE];
char cbits[200];
int nbBytes;
void *state;
SpeexBits bits;
int i, tmp;
state = speex_encoder_init(&speex_nb_mode);
tmp=8;
speex_encoder_ctl(state, SPEEX_SET_QUALITY, &tmp);
inFile = argv[1];
fin = fopen(inFile, "r");
speex_bits_init(&bits);
while (1)
{
fread(in, sizeof(short), FRAME_SIZE, fin);
if (feof(fin))
break;
for (i=0;i<FRAME_SIZE;i++)
input[i]=in[i];
speex_bits_reset(&bits);
speex_encode(state, input, &bits);
nbBytes = speex_bits_write(&bits, cbits, 200);
fwrite(&nbBytes, sizeof(int), 1, stdout);
fwrite(cbits, 1, nbBytes, stdout);
}
speex_encoder_destroy(state);
speex_bits_destroy(&bits);
fclose(fin);
return 0;
}
sampledec.c
sampledec reads a Speex stream from stdin, decodes it and outputs it to a raw 16 bits/sample file. Note that the packing used is NOT compatible with that of speexenc/speexdec.
#include <speex/speex.h>
#include <stdio.h>
#define FRAME_SIZE 160
int main(int argc, char **argv)
{
char *outFile;
FILE *fout;
short out[FRAME_SIZE];
float output[FRAME_SIZE];
char cbits[200];
int nbBytes;
void *state;
SpeexBits bits;
int i, tmp;
state = speex_decoder_init(&speex_nb_mode);
tmp=1;
speex_decoder_ctl(state, SPEEX_SET_ENH, &tmp);
outFile = argv[1];
fout = fopen(outFile, "w");
speex_bits_init(&bits);
while (1)
{
fread(&nbBytes, sizeof(int), 1, stdin);
fprintf (stderr, "nbBytes: %d\n", nbBytes);
if (feof(stdin))
break;
fread(cbits, 1, nbBytes, stdin);
speex_bits_read_from(&bits, cbits, nbBytes);
speex_decode(state, &bits, output);
for (i=0;i<FRAME_SIZE;i++)
out[i]=output[i];
fwrite(out, sizeof(short), FRAME_SIZE, fout);
}
speex_decoder_destroy(state);
speex_bits_destroy(&bits);
fclose(fout);
return 0;
}
開源 H323 協(xié)議中封裝的使用參考代碼:
#include <ptlib.h>
#ifdef __GNUC__
#pragma implementation "speexcodec.h"
#endif
#include "speexcodec.h"
#include "h323caps.h"
#include "h245.h"
#include "rtp.h"
extern "C" {
#include "speex/libspeex/speex.h"
};
#define new PNEW
#define XIPH_COUNTRY_CODE 0xB5 // (181) Country code for United States
#define XIPH_T35EXTENSION 0
#define XIPH_MANUFACTURER_CODE 0x0026 // Allocated by Delta Inc
#define EQUIVALENCE_COUNTRY_CODE 9 // Country code for Australia
#define EQUIVALENCE_T35EXTENSION 0
#define EQUIVALENCE_MANUFACTURER_CODE 61 // Allocated by Australian Communications Authority, Oct 2000
#define SAMPLES_PER_FRAME 160
#define SPEEX_BASE_NAME "Speex"
#define SPEEX_NARROW2_H323_NAME SPEEX_BASE_NAME "Narrow-5.95k{sw}"
#define SPEEX_NARROW3_H323_NAME SPEEX_BASE_NAME "Narrow-8k{sw}"
#define SPEEX_NARROW4_H323_NAME SPEEX_BASE_NAME "Narrow-11k{sw}"
#define SPEEX_NARROW5_H323_NAME SPEEX_BASE_NAME "Narrow-15k{sw}"
#define SPEEX_NARROW6_H323_NAME SPEEX_BASE_NAME "Narrow-18.2k{sw}"
H323_REGISTER_CAPABILITY(SpeexNarrow2AudioCapability, SPEEX_NARROW2_H323_NAME);
H323_REGISTER_CAPABILITY(SpeexNarrow3AudioCapability, SPEEX_NARROW3_H323_NAME);
H323_REGISTER_CAPABILITY(SpeexNarrow4AudioCapability, SPEEX_NARROW4_H323_NAME);
H323_REGISTER_CAPABILITY(SpeexNarrow5AudioCapability, SPEEX_NARROW5_H323_NAME);
H323_REGISTER_CAPABILITY(SpeexNarrow6AudioCapability, SPEEX_NARROW6_H323_NAME);
#define XIPH_SPEEX_NARROW2_H323_NAME SPEEX_BASE_NAME "Narrow-5.95k(Xiph){sw}"
#define XIPH_SPEEX_NARROW3_H323_NAME SPEEX_BASE_NAME "Narrow-8k(Xiph){sw}"
#define XIPH_SPEEX_NARROW4_H323_NAME SPEEX_BASE_NAME "Narrow-11k(Xiph){sw}"
#define XIPH_SPEEX_NARROW5_H323_NAME SPEEX_BASE_NAME "Narrow-15k(Xiph){sw}"
#define XIPH_SPEEX_NARROW6_H323_NAME SPEEX_BASE_NAME "Narrow-18.2k(Xiph){sw}"
H323_REGISTER_CAPABILITY(XiphSpeexNarrow2AudioCapability, XIPH_SPEEX_NARROW2_H323_NAME);
H323_REGISTER_CAPABILITY(XiphSpeexNarrow3AudioCapability, XIPH_SPEEX_NARROW3_H323_NAME);
H323_REGISTER_CAPABILITY(XiphSpeexNarrow4AudioCapability, XIPH_SPEEX_NARROW4_H323_NAME);
H323_REGISTER_CAPABILITY(XiphSpeexNarrow5AudioCapability, XIPH_SPEEX_NARROW5_H323_NAME);
H323_REGISTER_CAPABILITY(XiphSpeexNarrow6AudioCapability, XIPH_SPEEX_NARROW6_H323_NAME);
/////////////////////////////////////////////////////////////////////////
static int Speex_Bits_Per_Second(int mode) {
void *tmp_coder_state;
int bitrate;
tmp_coder_state = speex_encoder_init(&speex_nb_mode);
speex_encoder_ctl(tmp_coder_state, SPEEX_SET_QUALITY, &mode);
speex_encoder_ctl(tmp_coder_state, SPEEX_GET_BITRATE, &bitrate);
speex_encoder_destroy(tmp_coder_state);
return bitrate;
}
static int Speex_Bytes_Per_Frame(int mode) {
int bits_per_frame = Speex_Bits_Per_Second(mode) / 50; // (20ms frame size)
return ((bits_per_frame+7)/8); // round up
}
OpalMediaFormat const OpalSpeexNarrow_5k95(OPAL_SPEEX_NARROW_5k95,
OpalMediaFormat::DefaultAudioSessionID,