语音识别,语义理解一站式解决(android平台&olami sdk)

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了语音识别,语义理解一站式解决(android平台&olami sdk)相关的知识,希望对你有一定的参考价值。

  

olami sdk语音识别语义理解做在线听书

olamisdk实现了把录音或者文字转化为用户可以理解的json字符串,本文使用olami sdk做了一个在线听书的demo,用的是喜马拉雅的在线听书sdk.基于eclipse开发环境,libs目录下jarso文件如下:

olami-android-sdk.jar//olami sdk jar

afinal_0.5.1_bin.jar

litepal.jar

gson-2.2.4.jar

okhttp-2.4.0.jar

okhttp-urlconnection-2.2.0.jar

okio-1.4.0.jar

opensdk.jar          //上面这几个都是喜马拉雅需要的jar

libspeex.so          //olami sdk 需要用到speex压缩功能

libxmediaplayer.so   // 喜马拉雅so

libxmediaplayer_x.so// 喜马拉雅so

概述 
VoiceSdkService中定义了OlamiVoiceRecognizer语音识别引擎,通过点击MusicActivity的开始button启动录音,录音结果在VoiceSdkService中的onResult()回调中拿到识别的Json字符串,在processServiceMessage()函数中处理后找到要听书的名称,然后进入BookUtil进行搜索,搜索到结果后通知VoiceSdkService进行播放,并通知MusicActivity更新播放进度等信息。

 

 

 

 

 

 

1.AndroidManifest.xml配置

<?xml version="1.0"encoding="utf-8"?>

    <manifestxmlns:android="http://schemas.android.com/apk/res/android"

       package="com.olami.musicdemo"

       android:versionCode="1"

       android:versionName="1.0" >

 

       <uses-sdk

           android:minSdkVersion="8"

           android:targetSdkVersion="21" />

 

       <uses-permissionandroid:name="android.permission.RECORD_AUDIO"/>

       <uses-permissionandroid:name="android.permission.INTERNET"/>

       <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>

       <uses-permissionandroid:name="android.permission.ACCESS_WIFI_STATE"/>

       <uses-permissionandroid:name="android.permission.READ_PHONE_STATE"/>

       <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>

       <uses-permissionandroid:name="android.permission.WRITE_EXTERNAL_STORAGE" />

 

       <application

           android:name="com.olami.musicdemo.OlamiApplication"

           android:allowBackup="true"

           android:icon="@drawable/ic_launcher"

           android:label="@string/app_name"

           android:theme="@style/AppTheme" >

 

           <!--喜马拉雅听书测试账号app_key-->

           <meta-data

               android:name="app_key"

                android:value="b617866c20482d133d5de66fceb37da3"/>

           <!--喜马拉雅听书测试账号包名-->

           <meta-data

               android:name="pack_id"

               android:value="com.app.test.android" />

 

           <activity

               android:name=".MusicActivity"

               android:label="@string/app_name" >

                <intent-filter>

                    <actionandroid:name="android.intent.action.MAIN" />

 

                    <categoryandroid:name="android.intent.category.LAUNCHER" />

                </intent-filter>

           </activity>

 

            <!--注册olami sdk service-->

           <service

               android:name=".VoiceSdkService"

               android:exported="true" >

           </service>

 

           <!--注册喜马拉雅听书service-->

            <service               

              android:name=

            "com.ximalaya.ting.android.opensdk.player.service.XmPlayerService"

           />

       </application>

 

</manifest>

2.layout布局文件

layout_musicview.xml

TextView 有两个,tv_name显示听书的名称, tv_totoal_time显示听书的总时间。 
ProgressBar
实时刷新显示听书的进度

<RelativeLayoutxmlns:android="http://schemas.android.com/apk/res/android"

 xmlns:tools="http://schemas.android.com/tools"

  android:layout_width="fill_parent"

 android:layout_height="wrap_content"

  android:background="@android:color/transparent">

 

 

    <TextView

        android:text="name"

       android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_marginTop="5dp"

        android:layout_centerHorizontal="true"

       android:id="@+id/tv_name"/>

 

    <ProgressBar

       style="?android:attr/progressBarStyleHorizontal"

       android:layout_width="fill_parent"

       android:layout_height="wrap_content"

       android:layout_below="@+id/tv_name"

       android:layout_marginTop="10dp"

       android:layout_marginLeft="20dp"

       android:layout_marginRight="20dp"

       android:id="@+id/progressbar_music"/>

 

    <TextView

        android:text="total_time"

        android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_below="@+id/progressbar_music"

       android:layout_marginTop="10dp"

       android:layout_centerHorizontal="true"

       android:id="@+id/tv_total_time"/>

 

</RelativeLayout>

 

activity_music.xml

<RelativeLayoutxmlns:android="http://schemas.android.com/apk/res/android"

   xmlns:tools="http://schemas.android.com/tools"

   android:layout_width="match_parent"

   android:layout_height="match_parent"

   android:paddingLeft="@dimen/activity_horizontal_margin"

   android:paddingRight="@dimen/activity_horizontal_margin"

   tools:context="com.olami.musicdemo.MusicActivity" >

 

   <TextView

       android:id="@+id/tv_inputText"

       android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_marginTop="5dp"

       android:text="输入:" />

 

   <TextView

       android:id="@+id/tv_volume"

       android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_alignLeft="@+id/tv_inputText"

       android:layout_below="@+id/tv_inputText"

       android:layout_marginTop="40dp"

       android:text="音量:" />

 

   <TextView

       android:id="@+id/tv_result"

       android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_below="@+id/tv_volume"

       android:layout_marginTop="20dp"

       android:maxLines="15"

       android:ellipsize="end"

       android:text="服务器返回sentence:"

       android:visibility="visible" />

 

   <com.olami.musicdemo.MusicView

       android:id="@+id/music_view"

       android:layout_width="fill_parent"

       android:layout_height="80dp"

       android:layout_centerInParent="true"

       >

   </com.olami.musicdemo.MusicView>

 

   <EditText

       android:id="@+id/et_content"

       android:layout_width="wrap_content"

       android:layout_height="40dp"

       android:layout_above="@+id/btn_stop"

       android:layout_alignLeft="@+id/tv_inputText"

       android:layout_marginBottom="10dp"

       android:layout_toLeftOf="@+id/btn_send"

       android:background="#E7E7E7"

       android:singleLine="true"

       android:text="上海的天气" />   

 

    <Button

       android:id="@+id/btn_send"

        android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_alignBaseline="@+id/et_content"

       android:layout_alignBottom="@+id/et_content"

       android:layout_alignParentRight="true"

       android:text="提交" />

 

   <Button

       android:id="@+id/btn_start"

       android:layout_width="wrap_content"

       android:layout_height="wrap_content" 

       android:layout_alignParentBottom="true"

       android:layout_centerHorizontal="true"

       android:text="开始" />

 

   <Button

       android:id="@+id/btn_stop"

       android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_alignLeft="@+id/et_content"

       android:layout_alignParentBottom="true"

       android:text="停止" />

 

   <Button

       android:id="@+id/btn_cancel"

       android:layout_width="wrap_content"

       android:layout_height="wrap_content"

       android:layout_alignParentBottom="true"

       android:layout_toRightOf="@+id/et_content"

       android:text="取消" />

 

</RelativeLayout>

 

自定义MusicView比较简单,代码如下:

public class MusicView extendsRelativeLayout{

   private Context mContext;

   private Handler mHandler;

   private TextView mTextViewName;

   private TextView mTextViewTotalTime;

   private ProgressBar mProgressBar;

   public MusicView(Context context,AttributeSet attrs) {

       super(context,attrs);

       LayoutInflater inflater =(LayoutInflater) context.getSystemService(

                                            context.LAYOUT_INFLATER_SERVICE);

       RelativeLayout view = (RelativeLayout) inflater.inflate

(R.layout.layout_musicview,this,true);

       mTextViewName = (TextView) view.findViewById(R.id.tv_name);

       mTextViewTotalTime = (TextView) view.findViewById(R.id.tv_total_time);

       mProgressBar = (ProgressBar)view.findViewById(R.id.progressbar_music);

 

    }

 

   public void initMusicView(Context context,Handler handler)

    {

       mContext = context;

       mHandler = handler;

    }

 

   public void setMusicName(String name)

   {//设置播放名称

       mTextViewName.setText(name);

    }

 

   public void setProgress(int progress)

   {//设置播放进度

       mProgressBar.setProgress(progress);

    }

 

   public void setTotalTime(String time)

   {//设置播放总时间

       mTextViewTotalTime.setText(time);

    }

}

布局效果图如下: 

技术分享

 

 

3.MusicActivityVoiceSdkService通信

本文没有用bindservice的方式实现activityservice的消息通信。 
MusicAcitity
VoiceSdkService中分别实现了一个CommunicationAssist的接口

 

public interface CommunicationAssist {

   public void callBack(int what, int arg1, int arg2,Bundle data, Objectobj);

}

然后把他们分别实现CommunicationAssist接口的变量注册到OlamiApplication,这样通过OlamiApplication实现了MusicAcitity VoiceSdkService桥接。

 

3.1OlamiApplication

1) 注册MusicActivityVoiceSdkService的回调

public voidsetActivityToServiceListener(CommunicationAssist listener)

{

  ActivityToServiceListener = listener;

}

这个是在VoiceSdkService中调用setActivityToServiceListener(),VoiceSdkService中的VoiceSdkComAssist注册到application中,MusicActivity中可以通过getActivityToServiceListener 
这个函数回调向VoiceSdkService发送消息。

 

2) 注册 VoiceSdkServiceMusicActivity回调

public voidsetServiceToActivityListener(CommunicationAssist listener)

{

 mServiceToActivityListener = listener;

}

 

这个是在MusicAcitivity中调用setServiceToActivityListener(),这样在VoiceSdkService中就可以通过getServiceToActivityListener()获得回调向MusciActivity发送消息。

 

3.2MusicActivity

public class MusicActivity extends Activity{

   private Handler mHandler;

   private Handler mInComingHandler;

   private ActivityComAssist mActivityComAssist;

   private Button mBtnStart;

   private Button mBtnStop;

   private Button mBtnCancel;

   private Button mBtnSend;

   private EditText mEditText;

   private TextView mTextView;

   private TextView mInputTextView;

   private TextView mTextViewVolume;

   private BookUtil mBookUtil = null;

   private MusicView mMusicView = null;

 

   @Override

   protected void onCreate(Bundle savedInstanceState) {

       super.onCreate(savedInstanceState);

       setContentView(R.layout.activity_music);

       initHandler();//初始化handler用于内部消息处理

       initInComingHandler();//用于处理来自VoiceSdkService的消息

       initCommunicationAssist();//application注册消息回调,VoiceSdkSerive可以

                //通过getServiceToActivityListener()获得回调向MusicActivity发送消息

       initView();//初始化view控件

       Intent intent = new Intent();

       intent.setClass(MusicActivity.this, VoiceSdkService.class);

       startService(intent);//启动后台服务

 

    }

 

   private void initView()

    {

       mBtnStart = (Button) findViewById(R.id.btn_start);

       mBtnStop = (Button) findViewById(R.id.btn_stop);

       mBtnCancel = (Button) findViewById(R.id.btn_cancel);

       mBtnSend = (Button) findViewById(R.id.btn_send);

       mInputTextView = (TextView) findViewById(R.id.tv_inputText);

       mEditText = (EditText) findViewById(R.id.et_content);

       mTextView = (TextView) findViewById(R.id.tv_result);

       mTextViewVolume = (TextView) findViewById(R.id.tv_volume);

 

        mBtnStart.setOnClickListener(newOnClickListener(){

 

           @Override

           public void onClick(View v) {

               sendMessageToService(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);

           }          

       });

 

       mBtnStop.setOnClickListener(new OnClickListener(){

 

           @Override

           public void onClick(View v) {

               sendMessageToService(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);

                mBtnStart.setText("开始");

               Log.i("led","MusicActivity mBtnStop onclick 开始");

           }          

       });

 

       mBtnCancel.setOnClickListener(new OnClickListener(){

 

           @Override

           public void onClick(View v) {

                sendMessageToService(MessageConst.CLIENT_ACTION_CANCEL_RECORED,0,0,null,null);

           }          

       });

 

       mBtnSend.setOnClickListener(new OnClickListener(){

 

           @Override

                public void onClick(View v) {

           sendMessageToService(

           MessageConst.CLIENT_ACTION_SENT_TEXT,0,0,null,mEditText.getText());

                mInputTextView.setText("文字:"+mEditText.getText());

           }          

       });

 

       mMusicView = (MusicView) findViewById(R.id.music_view);

       //if(mMusicView != null)

           //mMusicView.initMusicView(MusicActivity.this,mHandler);

 

    }

 

   private void initHandler()

    {

       mHandler = new Handler(){

           @Override

           public voidhandleMessage(Message msg)

           {

                switch (msg.what){

                caseMessageConst.CLIENT_ACTION_START_RECORED:

                    break;

                default:

                    break; 

                }

            }

       };

    }

   //InComingHandler 收到来自VoiceSdkService的消息用于更新界面,

   //包括开始录音,结束录音,播放的书的名称和进度,音量等信息。

   private void initInComingHandler()

    {

       mInComingHandler = new Handler(){

           @Override

           public void handleMessage(Message msg)

           {

                switch (msg.what){

                caseMessageConst.CLIENT_ACTION_START_RECORED:

                    mBtnStart.setText("录音中");

                   Log.i("led","MusicActivity 录音中");

                    break;

                caseMessageConst.CLIENT_ACTION_STOP_RECORED:

                    mBtnStart.setText("识别中");

                   Log.i("led","MusicActivity 识别中");

                    break;

                caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

                    mBtnStart.setText("开始");

                    mTextView.setText("已取消");

                    break;

                caseMessageConst.CLIENT_ACTION_ON_ERROR:

                    mTextView.setText("错误代码:"+msg.arg1);

                    mBtnStart.setText("开始");

                    break;

                caseMessageConst.CLIENT_ACTION_UPDATA_VOLUME:

                   mTextViewVolume.setText("音量: "+msg.arg1);

                    break;

                caseMessageConst.SERVER_ACTION_RETURN_RESULT:

                   //mTextView.setText(msg.obj.toString());

                    mBtnStart.setText("开始");

                    break;

                caseMessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:

                    mBtnStart.setText("开始");

                    mBookUtil =BookUtil.getInstance();

                    mBookUtil.play(msg.arg1);

                    break;

                caseMessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:

                    mMusicView.setMusicName(msg.obj.toString());

                    break;

                caseMessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:

                    int current = msg.arg1;

                    int duration = msg.arg2;

                   mMusicView.setProgress(current*100/duration);

                    float time =duration/1000/60;

                   mMusicView.setTotalTime("总时间:"+time);

                    break;

                caseMessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT:

                    if(msg.obj != null)

                      mInputTextView.setText("文字:"+msg.obj.toString());

                    break;

                caseMessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE:

                    if(msg.obj != null)

                        mTextView.setText("服务器返回sentence:"+msg.obj.toString());

                    break;

                default:

                    break;

                }

           }

       };

    }

 

   private void initCommunicationAssist()

   {//Application注册VoiceSdkServiceMusicActivity的回调

       mActivityComAssist = new ActivityComAssist();

       OlamiApplication.getInstance().setServiceToActivityListener(mActivityComAssist);

    }

 

   private void sendMessageToService(int what, int arg1, int arg2, Bundledata, Object obj)

   {//VoiceSdkService发送消息

       if(OlamiApplication.getInstance().getActivityToServiceListener() !=null)

           OlamiApplication.getInstance().getActivityToServiceListener().callBack

           (what, arg1, arg2, data, obj);

    }

 

   private class ActivityComAssist implements CommunicationAssist{

   //实现CommunicationAssist借口,用于回调VoiceSdkService发送过来的消息

       @Override

       public void callBack(int what, int arg1, int arg2, Bundle data,Objectobj) {

           Message msg = Message.obtain(null, what);

           msg.arg1 = arg1;

           msg.arg2 = arg2;

           if (data != null)

                msg.setData(data);

           if (obj != null)

                msg.obj = obj;

           mInComingHandler.sendMessage(msg);

       }       

    }

 

   @Override

   public void onDestroy() {

       //退出应用,停止VoiceSdkService,会进行资源的释放

       super.onDestroy();

       Intent intent = new Intent();

       intent.setClass(MusicActivity.this, VoiceSdkService.class);

       stopService(intent);

    }

}

3.3VoiceSdkService

@Override

public void onCreate() {

   initHandler();//用于内部消息处理

   initInComingHandler();//用于处理来自MusicActivity的消息

   initCommunicationAssist();//application注册消息回调,这样MusicActivity

   //以通过getActivityToServiceListener()回调向VoiceSdkService发送消息

   initViaVoiceRecognizerListener();//初始化录音识别回调listener

   init();//olami录音识别引擎初始化

   initXmly();//喜马拉雅初始化

}

 

public void init()

{

   initHandler();

   mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);

   TelephonyManager telephonyManager=(TelephonyManager)this.getSystemService(

   (this.getBaseContext().TELEPHONY_SERVICE);

   String imei=telephonyManager.getDeviceId();

   mOlamiVoiceRecognizer.init(imei);//设置身份标识,可以填null

 

   mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);//设置识别结果回调listener

   mOlamiVoiceRecognizer.setLocalization(

   OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);//设置支持的语音类型,优先选择中文简体

   mOlamiVoiceRecognizer.setAuthorization("51a4bb56ba954655a4fc834bfdc46af1",

                           "asr","68bff251789b426896e70e888f919a6d","nli"); 

   //注册Appkey,在olami官网注册应用后生成的appkey

   //注册api,请直接填写“asr”,标识语音识别类型

   //注册secret,在olami官网注册应用后生成的secret

   //注册seq ,请填写“nli

 

   mOlamiVoiceRecognizer.setVADTailTimeout(2000);//录音时尾音结束时间,建议填//2000ms

   //设置经纬度信息,不愿上传位置信息,可以填0

   mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009);

}

 

定义OlamiVoiceRecognizerListener

onError(int errCode)//出错回调,可以对比官方文档错误码看是什么错误 
onEndOfSpeech()//
录音结束 
onBeginningOfSpeech()//
录音开始 
onResult(String result, int type)//result
是识别结果JSON字符串 
onCancel()//
取消识别,不会再返回识别结果 
onUpdateVolume(int volume)//
录音时的音量,1-12个级别大小音量

下面是VoiceSdkService完整代码:

 

public class VoiceSdkService extends Service{

 

   private Handler mHandler;

   private Handler mInComingHandler;

   private VoiceSdkComAssist mVoiceSdkComAssist;

   private OlamiVoiceRecognizer mOlamiVoiceRecognizer;

   private OlamiVoiceRecognizerListener mOlamiVoiceRecognizerListener;

    privateBookUtil mBookUtil = null;

   private boolean mIsRecordPause = false;

 

   @Override

   public void onCreate() {

       initHandler();

       initInComingHandler();

       initCommunicationAssist();

       initViaVoiceRecognizerListener();

       init();

       initXmly();

    }

 

   @Override

   public IBinder onBind(Intent intent) {

       // TODO Auto-generated method stub

       return null;

    }

 

   public void init()

    {

       initHandler();

       mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);

       TelephonyManager telephonyManager=(TelephonyManager)this.getSystemService(

       this.getBaseContext().TELEPHONY_SERVICE);

       String imei=telephonyManager.getDeviceId();

       mOlamiVoiceRecognizer.init(imei);//set null if you do not want to notifyolami server.

 

       mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);

       mOlamiVoiceRecognizer.setLocalization(

                                        OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);

       mOlamiVoiceRecognizer.setAuthorization(

           "51a4bb56ba954655a4fc834bfdc46af1",

           "asr","68bff251789b426896e70e888f919a6d","nli");       

       mOlamiVoiceRecognizer.setVADTailTimeout(2000);

       mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009);

    }

 

   private void initHandler()

    {

       mHandler = new Handler(){

           @Override

           public void handleMessage(Message msg)

           {

                switch (msg.what){

                caseMessageConst.CLIENT_ACTION_START_RECORED:

                   sendMessageToActivity(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);

                    break;

                case MessageConst.CLIENT_ACTION_STOP_RECORED:

                   sendMessageToActivity(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);

                    break;

                caseMessageConst.CLIENT_ACTION_ON_ERROR:

                    sendMessageToActivity(MessageConst.CLIENT_ACTION_ON_ERROR,msg.arg1,0,null,null);

                    break;

                caseMessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:

                   sendMessageToActivity(MessageConst.

                    CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH,msg.arg1, 0, null, msg.obj);

                    break;

                caseMessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:

                   sendMessageToActivity(MessageConst.

                   CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME, msg.arg1, 0, null, msg.obj);

                    break;

                caseMessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:

                   sendMessageToActivity(MessageConst.

                    CLIENT_ACTION_UPDATE_BOOK_PROGRESS,msg.arg1, msg.arg2, null, null);

                    break;

                caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

                   sendMessageToActivity(MessageConst.

                    CLIENT_ACTION_CANCEL_RECORED,msg.arg1, msg.arg2, null, null);

                    break;

                default:

                    break;

                }

           }

       };

    }

 

   private void initInComingHandler()

    {

       mInComingHandler = new Handler(){

           @Override

           public void handleMessage(Message msg)

           {

                switch (msg.what){

                caseMessageConst.CLIENT_ACTION_START_RECORED:

                    if(mOlamiVoiceRecognizer !=null)

                       mOlamiVoiceRecognizer.start(); 

                    break;

                caseMessageConst.CLIENT_ACTION_STOP_RECORED:

                    if(mOlamiVoiceRecognizer !=null)

                       mOlamiVoiceRecognizer.stop();  

                    break;

                caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

                    if(mOlamiVoiceRecognizer !=null)

                       mOlamiVoiceRecognizer.cancel();

                    break;

                case MessageConst.CLIENT_ACTION_SENT_TEXT:

                    if(mOlamiVoiceRecognizer !=null)

                       mOlamiVoiceRecognizer.sendText(msg.obj.toString());                

                    break;

                }

           }

       };

    }

 

   private void initViaVoiceRecognizerListener()

    {

       mOlamiVoiceRecognizerListener = new OlamiVoiceRecognizerListener();

    }

 

   private class OlamiVoiceRecognizerListener implementsIOlamiVoiceRecognizerListener{

 

       @Override

       public void onError(int errCode) {

           mHandler.sendMessage(mHandler.obtainMessage(

           MessageConst.CLIENT_ACTION_ON_ERROR,errCode,0));

 

       }

 

       @Override

       public void onEndOfSpeech() {

           mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_STOP_RECORED);

           if(mIsRecordPause)

           {

                mIsRecordPause = false;

                mBookUtil.resumePlay();

           }

 

       }

 

       @Override

       public void onBeginningOfSpeech() {

           if(mBookUtil.isPlaying())

           {

                mBookUtil.pause();

                mIsRecordPause = true;

           }

           mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_START_RECORED);

 

       }

 

       @Override

       public void onResult(String result, int type) {    

           sendMessageToActivity(MessageConst.SERVER_ACTION_RETURN_RESULT,type,0,null,result);

           processServiceMessage(result);

       }

 

       @Override

       public void onCancel() {

           mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_CANCEL_RECORED);

 

       }

 

       @Override

       public void onUpdateVolume(int volume) {

 

       sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_VOLUME,volume,0,null,null);

 

       }

 

    }

 

   private void initCommunicationAssist()

    {

       mVoiceSdkComAssist = new VoiceSdkComAssist();

       OlamiApplication.getInstance().setActivityToServiceListener(mVoiceSdkComAssist);

    }

 

   private void initXmly()

    {

       if(mBookUtil == null)

       {

           mBookUtil = BookUtil.getInstance();

           mBookUtil.init(VoiceSdkService.this);

           mBookUtil.setHandler(mHandler);

       }

    }

 

   private void processServiceMessage(String message)

    {

       String input = null;

       String serverMessage = null;

       try{

           JSONObject jsonObject = new JSONObject(message);

           JSONArray jArrayNli = jsonObject.optJSONObject("data").optJSONArray("nli");

           JSONObject jObj = jArrayNli.optJSONObject(0);

           JSONArray jArraySemantic = null;

           if(message.contains("semantic"))

              jArraySemantic =jObj.getJSONArray("semantic");

           else{

               input =jsonObject.optJSONObject("data").optJSONObject("asr").

                optString("result");

               sendMessageToActivity(MessageConst.

                                    CLIENT_ACTION_UPDATA_INPUT_TEXT, 0, 0, null, input);

                serverMessage =jObj.optJSONObject("desc_obj").opt("result").toString();

               sendMessageToActivity(MessageConst.

                       CLIENT_ACTION_UPDATA_SERVER_MESSAGE, 0, 0, null, serverMessage);

                return;

           }

           JSONObject jObjSemantic;

           JSONArray jArraySlots;

           JSONArray jArrayModifier;

           String type = null;

           String songName = null;

           String singer = null;

 

 

           if(jObj != null) {

                type =jObj.optString("type");

               if("musiccontrol".equals(type))

                {

                    jObjSemantic =jArraySemantic.optJSONObject(0);

                    input =jObjSemantic.optString("input");

                    jArraySlots =jObjSemantic.optJSONArray("slots");

                    jArrayModifier =jObjSemantic.optJSONArray("modifier");

                    String modifier =(String)jArrayModifier.opt(0);

                    if((jArrayModifier != null)&& ("play".equals(modifier)))

                    {

                        if(jArraySlots != null)

                           for(inti=0,k=jArraySlots.length(); i<k; i++)

                           {

                               JSONObject obj =jArraySlots.getJSONObject(i);

                               String name =obj.optString("name");

                              if("singer".equals(name))

                                   singer =obj.optString("value");

                               elseif("songname".equals(name))

                                   songName =obj.optString("value");

 

                           }

                    }else if((modifier != null)&& ("stop".equals(modifier)))

                    {

                        if(mBookUtil != null)

                           if(mBookUtil.isPlaying())

                               mBookUtil.stop();

                    }else if((modifier != null)&& ("pause".equals(modifier)))

                    {

                        if(mBookUtil != null)

                           if(mBookUtil.isPlaying())

                               mBookUtil.pause();

                    }else if((modifier != null)&& ("resume_play".equals(modifier)))

                    {

                        if(mBookUtil != null)

                           mBookUtil.resumePlay();

                    }else if((modifier != null)&& ("add_volume".equals(modifier)))

                    {

                        if(mBookUtil != null)

                            mBookUtil.addVolume();

                    }else if((modifier != null)&& ("del_volume".equals(modifier)))

                    {

                        if(mBookUtil != null)

                           mBookUtil.delVolume();

                    }else if((modifier != null)&& ("next".equals(modifier)))

                    {

                        if(mBookUtil != null)

                            mBookUtil.next();

                    }else if((modifier != null)&& ("previous".equals(modifier)))

                    {

                       if(mBookUtil !=null)

                            mBookUtil.prev();

                    }else if((modifier != null)&& ("play_index".equals(modifier)))

                    {

                        int position = 0;

                        if(jArraySlots != null)

                               for(inti=0,k=jArraySlots.length(); i<k; i++)

                               {

                                   JSONObjectobj = jArraySlots.getJSONObject(i);

                                   JSONObjectjNumDetial = obj.getJSONObject("num_detail");

                                   String index= jNumDetial.optString("recommend_value");

                                   position =Integer.parseInt(index) - 1;

                               }

                        if(mBookUtil != null)

                           mBookUtil.skipTo(position);

                    }

                }

           }

           if(songName != null)

           {

                if(singer != null)

                {

 

                }else{

                   mBookUtil.searchBookAndPlay(songName,0,0);

                }

           }else if(singer != null)

           {

               mBookUtil.searchBookAndPlay(songName,0,0);

           }

           serverMessage =jObj.optJSONObject("desc_obj").opt("result").toString();

       }

       catch (Exception e)

       {

           e.printStackTrace();

       }

       //发送消息更新语音识别的文字

       sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT, 0,0, null, input);

       //发送消息更新服务器返回的结果字符串

       sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE,

                                                   0, 0, null, serverMessage);

 

    }

 

   private void sendMessageToActivity(int what, int arg1, int arg2, Bundledata, Object obj)

    {

       if(OlamiApplication.getInstance().getServiceToActivityListener() !=null)

           OlamiApplication.getInstance().getServiceToActivityListener().

                                            callBack(what, arg1, arg2, data, obj);

    }

 

   private class VoiceSdkComAssist implements CommunicationAssist{

 

       @Override

       public void callBack(int what, int arg1, int arg2, Bundle data,Objectobj) {

           Message msg = Message.obtain(null, what);

           msg.arg1 = arg1;

           msg.arg2 = arg2;

           if (data != null)

                msg.setData(data);

           if (obj != null)

                msg.obj = obj;

            mInComingHandler.sendMessage(msg);

       }      

    }

 

   @Override

   public void onDestroy(){

       super.onDestroy();

       if(mOlamiVoiceRecognizer != null)

           mOlamiVoiceRecognizer.destroy();

       if(mBookUtil != null)

       {

           mBookUtil.destroy();

       }

 

    }

 

}

 

3.4VoiceSdkServiceonResult的回调处理

在VoiceSdkService.java中processServiceMessage(String message)用于处理onResult的回调数据。例如“我要听三国演义”返回如下数据:

 

{

    "data": {

        "asr": {

            "result": "我要听三国演义",

            "speech_status": 0,

            "final": true,

            "status": 0

        },

        "nli": [

            {

                "desc_obj": {

                    "result": "正在努力搜索中,请稍等",

                    "status": 0

                },

                "semantic": [

                    {

                        "app": "musiccontrol",

                        "input": "我要听三国演义",

                        "slots": [

                            {

                                "name": "songname",

                               "value": "三国演义"

                            }

                        ],

                        "modifier": [

                            "play"

                        ],

                        "customer": "58df512384ae11f0bb7b487e"

                    }

                ],

                "type": "musiccontrol"

            }

        ]

    },

    "status": "ok"

}

1)解析出nlitype类型是musiccontrol,这是语法返回app的类型,而这个在线听书的demo只关心musiccontrolapp类型,其他的忽略。

2)用户说的话转成文字是在asr中的result中获取 
3
)在nli中的semantic中,input值是用户说的话,同asr中的result 
modifier
代表返回的行为动作,此处可以看到是play就是要求播放,slots中的数据表示歌曲名称是三国演义。 
那么动作是play,内容是歌曲名称是三国演义,在这个demo中调用 
mBookUtil.searchBookAndPlay(songName,0,0);
会先查询,查询到结果会再发播放消息要求播放,我要听三国演义这个流程就走完了。

 

4.BookUtil

说一下搜索听书的实现过程

public void searchBookInfo(StringbookName,final int index,final boolean isNeedPlay)

{

   mBookName = bookName;

   Map<String, String> param = new HashMap<String, String>();

   param.put(DTransferConstants.SEARCH_KEY, bookName);

   param.put(DTransferConstants.CATEGORY_ID, "" + 3);//此处3代表搜索的是听书

   //param.put(DTransferConstants.PAGE, "" + mPageId);

   param.put(DTransferConstants.SORT, "asc");//返回列表的排序是正序还是逆序

   param.put(DTransferConstants.PAGE_SIZE, "" + PAGE_SIZE);//每页能返回多少个查询结果

   mPage = (index/PAGE_SIZE)+1;//当前在第几页

 

   mPlayerManager = XmPlayerManager.getInstance(mContext);//喜马拉雅初始化部分

   mPlayerManager.init(mNotificationId, null);

   mPlayerManager.addPlayerStatusListener(mPlayerStatusListener);

   mPlayerManager.addAdsStatusListener(mAdsListener);

 

   CommonRequest.getSearchedAlbums(param, newIDataCallBack<SearchAlbumList>()

    {

 

       @Override

       public void onSuccess(SearchAlbumList object)  

       {                  

           if (object != null && object.getAlbums() != null

                    &&object.getAlbums().size() != 0)

           {

                if (mSearchAlbumList == null)

                {

                    mSearchAlbumList = object;

                }

                else

               {

                   mSearchAlbumList.getAlbums().addAll(object.getAlbums());

                }

               //mTrackAdapter.notifyDataSetChanged();

 

                Map<String, String> map =new HashMap<String, String>();

 

                map.put(

                DTransferConstants.ALBUM_ID,""+object.getAlbums().get(0).getId());

               map.put(DTransferConstants.SORT, "asc");

               map.put(DTransferConstants.PAGE, "" + mPage);

               map.put(DTransferConstants.PAGE_SIZE, "" + PAGE_SIZE);

 

                CommonRequest.getTracks(map,new IDataCallBack<TrackList>()

                {

 

                        @Override

                        public voidonSuccess(TrackList object)

                        {

                            mTrackList =object;

                            mTotalCount =mTrackList.getTotalCount();

                           if(mTrackList.getTracks().size() <= 0)

                                return;

                            String str = "专辑:"+mTrackList.getAlbumTitle()+

                                           get(0).getTrackTitle().toString();

                            if(isNeedPlay)

                            {

                                mPosition =index % PAGE_SIZE;

                               mHandler.sendMessage(mHandler.obtainMessage                               MessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH,

                             index %PAGE_SIZE,0));//此处mTrackList中已经查询出结果

                                //VoiceSdkService发送消息进行播放

                            }

                            else

                               sendBookInfoToServer();                                

                        }

                        @Override

                        public void onError(intcode, String message)

                        {

                           Log.i("ppp","error: "+message);

                           sendBookInfoToServer();

                        }

                });

 

            }

       }

 

       @Override

       public void onError(int code, String message)

       {

             Log.i("ppp","error: "+message);

              sendBookInfoToServer();

       }

   });

}

5.demo中支持的说法

我想听西游记 
我要听西游记 
播放西游记 
听西游记 
我想听西游记这本书 
上一首 
上一回 
下一首 
下一回 
暂停/暂停播放 
继续/继续播放 
声音大一点 
声音小一点 
关闭/关闭播放

用的是喜马拉雅测试账号,只支持听书的功能,查找歌曲的结果返回为空。

6.源码下载链接

https://github.com/ls0609/olami_musicdemo

7.相关博客

语音识别语义理解一站式解决(android平台&olami sdk

http://blog.csdn.net/ls0609/article/details/71519203

 

olami开放平台语法编写简介:http://blog.csdn.net/ls0609/article/details/71624340

olami开放平台语法官方介绍:https://cn.olami.ai/wiki/?mp=nli&content=nli2.html

csdn博文:http://blog.csdn.net/ls0609/article/details/71519203 


本文出自 “ls0609” 博客,请务必保留此出处http://ls0609.blog.51cto.com/12943469/1929273

以上是关于语音识别,语义理解一站式解决(android平台&olami sdk)的主要内容,如果未能解决你的问题,请参考以下文章

用Kotlin开发android平台语音识别,语义理解应用(olamisdk)

基于javascript用olamisdk实现web端语音识别语义理解(speex压缩)

零代码技能平台技术实践探索!

集成讯飞语音

恒源云_语音识别与语义处理领域之 NAG 优化器

恒源云(GPUSHARE)_语音识别与语义处理领域之[机器翻译] 21.7 mRASP2