语音识别，语义理解一站式解决（android平台&olami sdk）

Posted 2020-09-19

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了语音识别，语义理解一站式解决（android平台&olami sdk）相关的知识，希望对你有一定的参考价值。

用olami sdk语音识别语义理解做在线听书

olamisdk实现了把录音或者文字转化为用户可以理解的json字符串，本文使用olami sdk做了一个在线听书的demo，用的是喜马拉雅的在线听书sdk.基于eclipse开发环境，libs目录下jar和so文件如下：

olami-android-sdk.jar//olami sdk 的jar

afinal_0.5.1_bin.jar

litepal.jar

gson-2.2.4.jar

okhttp-2.4.0.jar

okhttp-urlconnection-2.2.0.jar

okio-1.4.0.jar

opensdk.jar //上面这几个都是喜马拉雅需要的jar

libspeex.so //olami sdk 需要用到speex压缩功能

libxmediaplayer.so // 喜马拉雅so

libxmediaplayer_x.so// 喜马拉雅so

概述：
VoiceSdkService中定义了OlamiVoiceRecognizer语音识别引擎，通过点击MusicActivity的开始button启动录音，录音结果在VoiceSdkService中的onResult()回调中拿到识别的Json字符串，在processServiceMessage()函数中处理后找到要听书的名称，然后进入BookUtil进行搜索，搜索到结果后通知VoiceSdkService进行播放，并通知MusicActivity更新播放进度等信息。

1.AndroidManifest.xml配置

<?xml version="1.0"encoding="utf-8"?>

<manifestxmlns:android="http://schemas.android.com/apk/res/android"

package="com.olami.musicdemo"

android:versionCode="1"

android:versionName="1.0" >

<uses-sdk

android:minSdkVersion="8"

android:targetSdkVersion="21" />

<uses-permissionandroid:name="android.permission.RECORD_AUDIO"/>

<uses-permissionandroid:name="android.permission.INTERNET"/>

<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>

<uses-permissionandroid:name="android.permission.ACCESS_WIFI_STATE"/>

<uses-permissionandroid:name="android.permission.READ_PHONE_STATE"/>

<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>

<uses-permissionandroid:name="android.permission.WRITE_EXTERNAL_STORAGE" />

<application

android:name="com.olami.musicdemo.OlamiApplication"

android:allowBackup="true"

android:icon="@drawable/ic_launcher"

android:label="@string/app_name"

android:theme="@style/AppTheme" >

<meta-data

android:name="app_key"

android:value="b617866c20482d133d5de66fceb37da3"/>

<meta-data

android:name="pack_id"

android:value="com.app.test.android" />

<activity

android:name=".MusicActivity"

android:label="@string/app_name" >

<intent-filter>

<actionandroid:name="android.intent.action.MAIN" />

<categoryandroid:name="android.intent.category.LAUNCHER" />

</intent-filter>

</activity>

<service

android:name=".VoiceSdkService"

android:exported="true" >

</service>

<service

android:name=

"com.ximalaya.ting.android.opensdk.player.service.XmPlayerService"

</application>

</manifest>

2.layout布局文件

layout_musicview.xml

TextView 有两个，tv_name显示听书的名称, tv_totoal_time显示听书的总时间。
ProgressBar 实时刷新显示听书的进度

<RelativeLayoutxmlns:android="http://schemas.android.com/apk/res/android"

xmlns:tools="http://schemas.android.com/tools"

android:layout_width="fill_parent"

android:layout_height="wrap_content"

android:background="@android:color/transparent">

<TextView

android:text="name"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_marginTop="5dp"

android:layout_centerHorizontal="true"

android:id="@+id/tv_name"/>

<ProgressBar

style="?android:attr/progressBarStyleHorizontal"

android:layout_width="fill_parent"

android:layout_height="wrap_content"

android:layout_below="@+id/tv_name"

android:layout_marginTop="10dp"

android:layout_marginLeft="20dp"

android:layout_marginRight="20dp"

android:id="@+id/progressbar_music"/>

<TextView

android:text="total_time"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_below="@+id/progressbar_music"

android:layout_marginTop="10dp"

android:layout_centerHorizontal="true"

android:id="@+id/tv_total_time"/>

</RelativeLayout>

activity_music.xml

<RelativeLayoutxmlns:android="http://schemas.android.com/apk/res/android"

xmlns:tools="http://schemas.android.com/tools"

android:layout_width="match_parent"

android:layout_height="match_parent"

android:paddingLeft="@dimen/activity_horizontal_margin"

android:paddingRight="@dimen/activity_horizontal_margin"

tools:context="com.olami.musicdemo.MusicActivity" >

<TextView

android:id="@+id/tv_inputText"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_marginTop="5dp"

android:text="输入：" />

<TextView

android:id="@+id/tv_volume"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_alignLeft="@+id/tv_inputText"

android:layout_below="@+id/tv_inputText"

android:layout_marginTop="40dp"

android:text="音量：" />

<TextView

android:id="@+id/tv_result"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_below="@+id/tv_volume"

android:layout_marginTop="20dp"

android:maxLines="15"

android:ellipsize="end"

android:text="服务器返回sentence："

android:visibility="visible" />

<com.olami.musicdemo.MusicView

android:id="@+id/music_view"

android:layout_width="fill_parent"

android:layout_height="80dp"

android:layout_centerInParent="true"

</com.olami.musicdemo.MusicView>

<EditText

android:id="@+id/et_content"

android:layout_width="wrap_content"

android:layout_height="40dp"

android:layout_above="@+id/btn_stop"

android:layout_alignLeft="@+id/tv_inputText"

android:layout_marginBottom="10dp"

android:layout_toLeftOf="@+id/btn_send"

android:background="#E7E7E7"

android:singleLine="true"

android:text="上海的天气" />

<Button

android:id="@+id/btn_send"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_alignBaseline="@+id/et_content"

android:layout_alignBottom="@+id/et_content"

android:layout_alignParentRight="true"

android:text="提交" />

<Button

android:id="@+id/btn_start"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_alignParentBottom="true"

android:layout_centerHorizontal="true"

android:text="开始" />

<Button

android:id="@+id/btn_stop"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_alignLeft="@+id/et_content"

android:layout_alignParentBottom="true"

android:text="停止" />

<Button

android:id="@+id/btn_cancel"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_alignParentBottom="true"

android:layout_toRightOf="@+id/et_content"

android:text="取消" />

</RelativeLayout>

自定义MusicView比较简单，代码如下：

public class MusicView extendsRelativeLayout{

private Context mContext;

private Handler mHandler;

private TextView mTextViewName;

private TextView mTextViewTotalTime;

private ProgressBar mProgressBar;

public MusicView(Context context,AttributeSet attrs) {

super(context,attrs);

LayoutInflater inflater =(LayoutInflater) context.getSystemService(

context.LAYOUT_INFLATER_SERVICE);

RelativeLayout view = (RelativeLayout) inflater.inflate

(R.layout.layout_musicview,this,true);

mTextViewName = (TextView) view.findViewById(R.id.tv_name);

mTextViewTotalTime = (TextView) view.findViewById(R.id.tv_total_time);

mProgressBar = (ProgressBar)view.findViewById(R.id.progressbar_music);

}

public void initMusicView(Context context,Handler handler)

{

mContext = context;

mHandler = handler;

}

public void setMusicName(String name)

{//设置播放名称

mTextViewName.setText(name);

}

public void setProgress(int progress)

{//设置播放进度

mProgressBar.setProgress(progress);

}

public void setTotalTime(String time)

{//设置播放总时间

mTextViewTotalTime.setText(time);

}

布局效果图如下：

3.MusicActivity和VoiceSdkService通信

本文没有用bindservice的方式实现activity和service的消息通信。
MusicAcitity 和 VoiceSdkService中分别实现了一个CommunicationAssist的接口

public interface CommunicationAssist {

public void callBack(int what, int arg1, int arg2,Bundle data, Objectobj);

}

然后把他们分别实现CommunicationAssist接口的变量注册到OlamiApplication，这样通过OlamiApplication实现了MusicAcitity 和VoiceSdkService桥接。

3.1OlamiApplication

1) 注册MusicActivity到VoiceSdkService的回调

public voidsetActivityToServiceListener(CommunicationAssist listener)

{

ActivityToServiceListener = listener;

}

这个是在VoiceSdkService中调用setActivityToServiceListener(),把VoiceSdkService中的VoiceSdkComAssist注册到application中，MusicActivity中可以通过getActivityToServiceListener
这个函数回调向VoiceSdkService发送消息。

2) 注册 VoiceSdkService到MusicActivity回调

public voidsetServiceToActivityListener(CommunicationAssist listener)

{

mServiceToActivityListener = listener;

}

这个是在MusicAcitivity中调用setServiceToActivityListener(),这样在VoiceSdkService中就可以通过getServiceToActivityListener()获得回调向MusciActivity发送消息。

3.2MusicActivity

public class MusicActivity extends Activity{

private Handler mHandler;

private Handler mInComingHandler;

private ActivityComAssist mActivityComAssist;

private Button mBtnStart;

private Button mBtnStop;

private Button mBtnCancel;

private Button mBtnSend;

private EditText mEditText;

private TextView mTextView;

private TextView mInputTextView;

private TextView mTextViewVolume;

private BookUtil mBookUtil = null;

private MusicView mMusicView = null;

@Override

protected void onCreate(Bundle savedInstanceState) {

super.onCreate(savedInstanceState);

setContentView(R.layout.activity_music);

initHandler();//初始化handler用于内部消息处理

initInComingHandler();//用于处理来自VoiceSdkService的消息

initCommunicationAssist();//向application注册消息回调，VoiceSdkSerive可以

//通过getServiceToActivityListener()获得回调向MusicActivity发送消息

initView();//初始化view控件

Intent intent = new Intent();

intent.setClass(MusicActivity.this, VoiceSdkService.class);

startService(intent);//启动后台服务

}

private void initView()

{

mBtnStart = (Button) findViewById(R.id.btn_start);

mBtnStop = (Button) findViewById(R.id.btn_stop);

mBtnCancel = (Button) findViewById(R.id.btn_cancel);

mBtnSend = (Button) findViewById(R.id.btn_send);

mInputTextView = (TextView) findViewById(R.id.tv_inputText);

mEditText = (EditText) findViewById(R.id.et_content);

mTextView = (TextView) findViewById(R.id.tv_result);

mTextViewVolume = (TextView) findViewById(R.id.tv_volume);

mBtnStart.setOnClickListener(newOnClickListener(){

@Override

public void onClick(View v) {

sendMessageToService(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);

}

});

mBtnStop.setOnClickListener(new OnClickListener(){

@Override

public void onClick(View v) {

sendMessageToService(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);

mBtnStart.setText("开始");

Log.i("led","MusicActivity mBtnStop onclick 开始");

}

});

mBtnCancel.setOnClickListener(new OnClickListener(){

@Override

public void onClick(View v) {

sendMessageToService(MessageConst.CLIENT_ACTION_CANCEL_RECORED,0,0,null,null);

}

});

mBtnSend.setOnClickListener(new OnClickListener(){

@Override

public void onClick(View v) {

sendMessageToService(

MessageConst.CLIENT_ACTION_SENT_TEXT,0,0,null,mEditText.getText());

mInputTextView.setText("文字:"+mEditText.getText());

}

});

mMusicView = (MusicView) findViewById(R.id.music_view);

//if(mMusicView != null)

//mMusicView.initMusicView(MusicActivity.this,mHandler);

}

private void initHandler()

{

mHandler = new Handler(){

@Override

public voidhandleMessage(Message msg)

{

switch (msg.what){

caseMessageConst.CLIENT_ACTION_START_RECORED:

break;

default:

break;

}

};

}

//InComingHandler 收到来自VoiceSdkService的消息用于更新界面，

//包括开始录音，结束录音，播放的书的名称和进度，音量等信息。

private void initInComingHandler()

{

mInComingHandler = new Handler(){

@Override

public void handleMessage(Message msg)

{

switch (msg.what){

caseMessageConst.CLIENT_ACTION_START_RECORED:

mBtnStart.setText("录音中");

Log.i("led","MusicActivity 录音中");

break;

caseMessageConst.CLIENT_ACTION_STOP_RECORED:

mBtnStart.setText("识别中");

Log.i("led","MusicActivity 识别中");

break;

caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

mBtnStart.setText("开始");

mTextView.setText("已取消");

break;

caseMessageConst.CLIENT_ACTION_ON_ERROR:

mTextView.setText("错误代码："+msg.arg1);

mBtnStart.setText("开始");

break;

caseMessageConst.CLIENT_ACTION_UPDATA_VOLUME:

mTextViewVolume.setText("音量: "+msg.arg1);

break;

caseMessageConst.SERVER_ACTION_RETURN_RESULT:

//mTextView.setText(msg.obj.toString());

mBtnStart.setText("开始");

break;

caseMessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:

mBtnStart.setText("开始");

mBookUtil =BookUtil.getInstance();

mBookUtil.play(msg.arg1);

break;

caseMessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:

mMusicView.setMusicName(msg.obj.toString());

break;

caseMessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:

int current = msg.arg1;

int duration = msg.arg2;

mMusicView.setProgress(current*100/duration);

float time =duration/1000/60;

mMusicView.setTotalTime("总时间："+time);

break;

caseMessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT:

if(msg.obj != null)

mInputTextView.setText("文字:"+msg.obj.toString());

break;

caseMessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE:

if(msg.obj != null)

mTextView.setText("服务器返回sentence:"+msg.obj.toString());

break;

default:

break;

}

};

}

private void initCommunicationAssist()

{//向Application注册VoiceSdkService到MusicActivity的回调

mActivityComAssist = new ActivityComAssist();

OlamiApplication.getInstance().setServiceToActivityListener(mActivityComAssist);

}

private void sendMessageToService(int what, int arg1, int arg2, Bundledata, Object obj)

{//向VoiceSdkService发送消息

if(OlamiApplication.getInstance().getActivityToServiceListener() !=null)

OlamiApplication.getInstance().getActivityToServiceListener().callBack

(what, arg1, arg2, data, obj);

}

private class ActivityComAssist implements CommunicationAssist{

//实现CommunicationAssist借口，用于回调VoiceSdkService发送过来的消息

@Override

public void callBack(int what, int arg1, int arg2, Bundle data,Objectobj) {

Message msg = Message.obtain(null, what);

msg.arg1 = arg1;

msg.arg2 = arg2;

if (data != null)

msg.setData(data);

if (obj != null)

msg.obj = obj;

mInComingHandler.sendMessage(msg);

}

@Override

public void onDestroy() {

//退出应用，停止VoiceSdkService，会进行资源的释放

super.onDestroy();

Intent intent = new Intent();

intent.setClass(MusicActivity.this, VoiceSdkService.class);

stopService(intent);

}

3.3VoiceSdkService

@Override

public void onCreate() {

initHandler();//用于内部消息处理

initInComingHandler();//用于处理来自MusicActivity的消息

initCommunicationAssist();//向application注册消息回调，这样MusicActivity可

//以通过getActivityToServiceListener()回调向VoiceSdkService发送消息

initViaVoiceRecognizerListener();//初始化录音识别回调listener

init();//olami录音识别引擎初始化

initXmly();//喜马拉雅初始化

}

public void init()

{

initHandler();

mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);

TelephonyManager telephonyManager=(TelephonyManager)this.getSystemService(

(this.getBaseContext().TELEPHONY_SERVICE);

String imei=telephonyManager.getDeviceId();

mOlamiVoiceRecognizer.init(imei);//设置身份标识，可以填null

mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);//设置识别结果回调listener

mOlamiVoiceRecognizer.setLocalization(

OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);//设置支持的语音类型，优先选择中文简体

mOlamiVoiceRecognizer.setAuthorization("51a4bb56ba954655a4fc834bfdc46af1",

"asr","68bff251789b426896e70e888f919a6d","nli");

//注册Appkey，在olami官网注册应用后生成的appkey

//注册api，请直接填写“asr”，标识语音识别类型

//注册secret，在olami官网注册应用后生成的secret

//注册seq ，请填写“nli”

mOlamiVoiceRecognizer.setVADTailTimeout(2000);//录音时尾音结束时间，建议填//2000ms

//设置经纬度信息，不愿上传位置信息，可以填0

mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009);

}

定义OlamiVoiceRecognizerListener

onError(int errCode)//出错回调，可以对比官方文档错误码看是什么错误
onEndOfSpeech()//录音结束
onBeginningOfSpeech()//录音开始
onResult(String result, int type)//result是识别结果JSON字符串
onCancel()//取消识别，不会再返回识别结果
onUpdateVolume(int volume)//录音时的音量，1-12个级别大小音量

下面是VoiceSdkService完整代码：

public class VoiceSdkService extends Service{

private Handler mHandler;

private Handler mInComingHandler;

private VoiceSdkComAssist mVoiceSdkComAssist;

private OlamiVoiceRecognizer mOlamiVoiceRecognizer;

private OlamiVoiceRecognizerListener mOlamiVoiceRecognizerListener;

privateBookUtil mBookUtil = null;

private boolean mIsRecordPause = false;

@Override

public void onCreate() {

initHandler();

initInComingHandler();

initCommunicationAssist();

initViaVoiceRecognizerListener();

init();

initXmly();

}

@Override

public IBinder onBind(Intent intent) {

// TODO Auto-generated method stub

return null;

}

public void init()

{

initHandler();

mOlamiVoiceRecognizer = new OlamiVoiceRecognizer(VoiceSdkService.this);

TelephonyManager telephonyManager=(TelephonyManager)this.getSystemService(

this.getBaseContext().TELEPHONY_SERVICE);

String imei=telephonyManager.getDeviceId();

mOlamiVoiceRecognizer.init(imei);//set null if you do not want to notifyolami server.

mOlamiVoiceRecognizer.setListener(mOlamiVoiceRecognizerListener);

mOlamiVoiceRecognizer.setLocalization(

OlamiVoiceRecognizer.LANGUAGE_SIMPLIFIED_CHINESE);

mOlamiVoiceRecognizer.setAuthorization(

"51a4bb56ba954655a4fc834bfdc46af1",

"asr","68bff251789b426896e70e888f919a6d","nli");

mOlamiVoiceRecognizer.setVADTailTimeout(2000);

mOlamiVoiceRecognizer.setLatitudeAndLongitude(31.155364678184498,121.34882432933009);

}

private void initHandler()

{

mHandler = new Handler(){

@Override

public void handleMessage(Message msg)

{

switch (msg.what){

caseMessageConst.CLIENT_ACTION_START_RECORED:

sendMessageToActivity(MessageConst.CLIENT_ACTION_START_RECORED,0,0,null,null);

break;

case MessageConst.CLIENT_ACTION_STOP_RECORED:

sendMessageToActivity(MessageConst.CLIENT_ACTION_STOP_RECORED,0,0,null,null);

break;

caseMessageConst.CLIENT_ACTION_ON_ERROR:

sendMessageToActivity(MessageConst.CLIENT_ACTION_ON_ERROR,msg.arg1,0,null,null);

break;

caseMessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH:

sendMessageToActivity(MessageConst.

CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH,msg.arg1, 0, null, msg.obj);

break;

caseMessageConst.CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME:

sendMessageToActivity(MessageConst.

CLIENT_ACTION_UPDATA_PLAYING_BOOK_NAME, msg.arg1, 0, null, msg.obj);

break;

caseMessageConst.CLIENT_ACTION_UPDATE_BOOK_PROGRESS:

sendMessageToActivity(MessageConst.

CLIENT_ACTION_UPDATE_BOOK_PROGRESS,msg.arg1, msg.arg2, null, null);

break;

caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

sendMessageToActivity(MessageConst.

CLIENT_ACTION_CANCEL_RECORED,msg.arg1, msg.arg2, null, null);

break;

default:

break;

}

};

}

private void initInComingHandler()

{

mInComingHandler = new Handler(){

@Override

public void handleMessage(Message msg)

{

switch (msg.what){

caseMessageConst.CLIENT_ACTION_START_RECORED:

if(mOlamiVoiceRecognizer !=null)

mOlamiVoiceRecognizer.start();

break;

caseMessageConst.CLIENT_ACTION_STOP_RECORED:

if(mOlamiVoiceRecognizer !=null)

mOlamiVoiceRecognizer.stop();

break;

caseMessageConst.CLIENT_ACTION_CANCEL_RECORED:

if(mOlamiVoiceRecognizer !=null)

mOlamiVoiceRecognizer.cancel();

break;

case MessageConst.CLIENT_ACTION_SENT_TEXT:

if(mOlamiVoiceRecognizer !=null)

mOlamiVoiceRecognizer.sendText(msg.obj.toString());

break;

}

};

}

private void initViaVoiceRecognizerListener()

{

mOlamiVoiceRecognizerListener = new OlamiVoiceRecognizerListener();

}

private class OlamiVoiceRecognizerListener implementsIOlamiVoiceRecognizerListener{

@Override

public void onError(int errCode) {

mHandler.sendMessage(mHandler.obtainMessage(

MessageConst.CLIENT_ACTION_ON_ERROR,errCode,0));

}

@Override

public void onEndOfSpeech() {

mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_STOP_RECORED);

if(mIsRecordPause)

{

mIsRecordPause = false;

mBookUtil.resumePlay();

}

@Override

public void onBeginningOfSpeech() {

if(mBookUtil.isPlaying())

{

mBookUtil.pause();

mIsRecordPause = true;

}

mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_START_RECORED);

}

@Override

public void onResult(String result, int type) {

sendMessageToActivity(MessageConst.SERVER_ACTION_RETURN_RESULT,type,0,null,result);

processServiceMessage(result);

}

@Override

public void onCancel() {

mHandler.sendEmptyMessage(MessageConst.CLIENT_ACTION_CANCEL_RECORED);

}

@Override

public void onUpdateVolume(int volume) {

sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_VOLUME,volume,0,null,null);

}

private void initCommunicationAssist()

{

mVoiceSdkComAssist = new VoiceSdkComAssist();

OlamiApplication.getInstance().setActivityToServiceListener(mVoiceSdkComAssist);

}

private void initXmly()

{

if(mBookUtil == null)

{

mBookUtil = BookUtil.getInstance();

mBookUtil.init(VoiceSdkService.this);

mBookUtil.setHandler(mHandler);

}

private void processServiceMessage(String message)

{

String input = null;

String serverMessage = null;

try{

JSONObject jsonObject = new JSONObject(message);

JSONArray jArrayNli = jsonObject.optJSONObject("data").optJSONArray("nli");

JSONObject jObj = jArrayNli.optJSONObject(0);

JSONArray jArraySemantic = null;

if(message.contains("semantic"))

jArraySemantic =jObj.getJSONArray("semantic");

else{

input =jsonObject.optJSONObject("data").optJSONObject("asr").

optString("result");

sendMessageToActivity(MessageConst.

CLIENT_ACTION_UPDATA_INPUT_TEXT, 0, 0, null, input);

serverMessage =jObj.optJSONObject("desc_obj").opt("result").toString();

sendMessageToActivity(MessageConst.

CLIENT_ACTION_UPDATA_SERVER_MESSAGE, 0, 0, null, serverMessage);

return;

}

JSONObject jObjSemantic;

JSONArray jArraySlots;

JSONArray jArrayModifier;

String type = null;

String songName = null;

String singer = null;

if(jObj != null) {

type =jObj.optString("type");

if("musiccontrol".equals(type))

{

jObjSemantic =jArraySemantic.optJSONObject(0);

input =jObjSemantic.optString("input");

jArraySlots =jObjSemantic.optJSONArray("slots");

jArrayModifier =jObjSemantic.optJSONArray("modifier");

String modifier =(String)jArrayModifier.opt(0);

if((jArrayModifier != null)&& ("play".equals(modifier)))

{

if(jArraySlots != null)

for(inti=0,k=jArraySlots.length(); i<k; i++)

{

JSONObject obj =jArraySlots.getJSONObject(i);

String name =obj.optString("name");

if("singer".equals(name))

singer =obj.optString("value");

elseif("songname".equals(name))

songName =obj.optString("value");

}

}else if((modifier != null)&& ("stop".equals(modifier)))

{

if(mBookUtil != null)

if(mBookUtil.isPlaying())

mBookUtil.stop();

}else if((modifier != null)&& ("pause".equals(modifier)))

{

if(mBookUtil != null)

if(mBookUtil.isPlaying())

mBookUtil.pause();

}else if((modifier != null)&& ("resume_play".equals(modifier)))

{

if(mBookUtil != null)

mBookUtil.resumePlay();

}else if((modifier != null)&& ("add_volume".equals(modifier)))

{

if(mBookUtil != null)

mBookUtil.addVolume();

}else if((modifier != null)&& ("del_volume".equals(modifier)))

{

if(mBookUtil != null)

mBookUtil.delVolume();

}else if((modifier != null)&& ("next".equals(modifier)))

{

if(mBookUtil != null)

mBookUtil.next();

}else if((modifier != null)&& ("previous".equals(modifier)))

{

if(mBookUtil !=null)

mBookUtil.prev();

}else if((modifier != null)&& ("play_index".equals(modifier)))

{

int position = 0;

if(jArraySlots != null)

for(inti=0,k=jArraySlots.length(); i<k; i++)

{

JSONObjectobj = jArraySlots.getJSONObject(i);

JSONObjectjNumDetial = obj.getJSONObject("num_detail");

String index= jNumDetial.optString("recommend_value");

position =Integer.parseInt(index) - 1;

}

if(mBookUtil != null)

mBookUtil.skipTo(position);

}

if(songName != null)

{

if(singer != null)

{

}else{

mBookUtil.searchBookAndPlay(songName,0,0);

}

}else if(singer != null)

{

mBookUtil.searchBookAndPlay(songName,0,0);

}

serverMessage =jObj.optJSONObject("desc_obj").opt("result").toString();

}

catch (Exception e)

{

e.printStackTrace();

}

//发送消息更新语音识别的文字

sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_INPUT_TEXT, 0,0, null, input);

//发送消息更新服务器返回的结果字符串

sendMessageToActivity(MessageConst.CLIENT_ACTION_UPDATA_SERVER_MESSAGE,

0, 0, null, serverMessage);

}

private void sendMessageToActivity(int what, int arg1, int arg2, Bundledata, Object obj)

{

if(OlamiApplication.getInstance().getServiceToActivityListener() !=null)

OlamiApplication.getInstance().getServiceToActivityListener().

callBack(what, arg1, arg2, data, obj);

}

private class VoiceSdkComAssist implements CommunicationAssist{

@Override

public void callBack(int what, int arg1, int arg2, Bundle data,Objectobj) {

Message msg = Message.obtain(null, what);

msg.arg1 = arg1;

msg.arg2 = arg2;

if (data != null)

msg.setData(data);

if (obj != null)

msg.obj = obj;

mInComingHandler.sendMessage(msg);

}

@Override

public void onDestroy(){

super.onDestroy();

if(mOlamiVoiceRecognizer != null)

mOlamiVoiceRecognizer.destroy();

if(mBookUtil != null)

{

mBookUtil.destroy();

}

3.4VoiceSdkService中onResult的回调处理

在VoiceSdkService.java中processServiceMessage(String message)用于处理onResult的回调数据。例如“我要听三国演义”返回如下数据：

{

"data": {

"asr": {

"result": "我要听三国演义",

"speech_status": 0,

"final": true,

"status": 0

"nli": [

{

"desc_obj": {

"result": "正在努力搜索中，请稍等",

"status": 0

"semantic": [

{

"app": "musiccontrol",

"input": "我要听三国演义",

"slots": [

{

"name": "songname",

"value": "三国演义"

}

"modifier": [

"play"

"customer": "58df512384ae11f0bb7b487e"

}

"type": "musiccontrol"

}

]

"status": "ok"

}

1）解析出nli中type类型是musiccontrol,这是语法返回app的类型，而这个在线听书的demo只关心musiccontrol这个app类型，其他的忽略。

2）用户说的话转成文字是在asr中的result中获取
3）在nli中的semantic中，input值是用户说的话，同asr中的result。
modifier代表返回的行为动作，此处可以看到是play就是要求播放，slots中的数据表示歌曲名称是三国演义。
那么动作是play，内容是歌曲名称是三国演义，在这个demo中调用
mBookUtil.searchBookAndPlay(songName,0,0);会先查询，查询到结果会再发播放消息要求播放，我要听三国演义这个流程就走完了。

4.BookUtil

说一下搜索听书的实现过程

public void searchBookInfo(StringbookName,final int index,final boolean isNeedPlay)

{

mBookName = bookName;

Map<String, String> param = new HashMap<String, String>();

param.put(DTransferConstants.SEARCH_KEY, bookName);

param.put(DTransferConstants.CATEGORY_ID, "" + 3);//此处3代表搜索的是听书

//param.put(DTransferConstants.PAGE, "" + mPageId);

param.put(DTransferConstants.SORT, "asc");//返回列表的排序是正序还是逆序

param.put(DTransferConstants.PAGE_SIZE, "" + PAGE_SIZE);//每页能返回多少个查询结果

mPage = (index/PAGE_SIZE)+1;//当前在第几页

mPlayerManager = XmPlayerManager.getInstance(mContext);//喜马拉雅初始化部分

mPlayerManager.init(mNotificationId, null);

mPlayerManager.addPlayerStatusListener(mPlayerStatusListener);

mPlayerManager.addAdsStatusListener(mAdsListener);

CommonRequest.getSearchedAlbums(param, newIDataCallBack<SearchAlbumList>()

{

@Override

public void onSuccess(SearchAlbumList object)

{

if (object != null && object.getAlbums() != null

&&object.getAlbums().size() != 0)

{

if (mSearchAlbumList == null)

{

mSearchAlbumList = object;

}

else

{

mSearchAlbumList.getAlbums().addAll(object.getAlbums());

}

//mTrackAdapter.notifyDataSetChanged();

Map<String, String> map =new HashMap<String, String>();

map.put(

DTransferConstants.ALBUM_ID,""+object.getAlbums().get(0).getId());

map.put(DTransferConstants.SORT, "asc");

map.put(DTransferConstants.PAGE, "" + mPage);

map.put(DTransferConstants.PAGE_SIZE, "" + PAGE_SIZE);

CommonRequest.getTracks(map,new IDataCallBack<TrackList>()

{

@Override

public voidonSuccess(TrackList object)

{

mTrackList =object;

mTotalCount =mTrackList.getTotalCount();

if(mTrackList.getTracks().size() <= 0)

return;

String str = "专辑："+mTrackList.getAlbumTitle()+

get(0).getTrackTitle().toString();

if(isNeedPlay)

{

mPosition =index % PAGE_SIZE;

mHandler.sendMessage(mHandler.obtainMessage MessageConst.CLIENT_ACTION_PLAY_BOOK_AFTER_SEARCH,

index %PAGE_SIZE,0));//此处mTrackList中已经查询出结果

//向VoiceSdkService发送消息进行播放

}

else

sendBookInfoToServer();

}

@Override

public void onError(intcode, String message)

{

Log.i("ppp","error: "+message);

sendBookInfoToServer();

}

});

}

@Override

public void onError(int code, String message)

{

Log.i("ppp","error: "+message);

sendBookInfoToServer();

}

});

}

5.demo中支持的说法

我想听西游记
我要听西游记
播放西游记
听西游记
我想听西游记这本书
上一首
上一回
下一首
下一回
暂停/暂停播放
继续/继续播放
声音大一点
声音小一点
关闭/关闭播放

用的是喜马拉雅测试账号，只支持听书的功能，查找歌曲的结果返回为空。

6.源码下载链接

https://github.com/ls0609/olami_musicdemo

7.相关博客

语音识别语义理解一站式解决（android平台&olami sdk）

http://blog.csdn.net/ls0609/article/details/71519203

olami开放平台语法编写简介：http://blog.csdn.net/ls0609/article/details/71624340

olami开放平台语法官方介绍：https://cn.olami.ai/wiki/?mp=nli&content=nli2.html

csdn博文：http://blog.csdn.net/ls0609/article/details/71519203

本文出自 “ls0609” 博客，请务必保留此出处http://ls0609.blog.51cto.com/12943469/1929273

以上是关于语音识别，语义理解一站式解决（android平台&olami sdk）的主要内容，如果未能解决你的问题，请参考以下文章

用Kotlin开发android平台语音识别，语义理解应用（olamisdk）

基于javascript用olamisdk实现web端语音识别语义理解（speex压缩）

零代码技能平台技术实践探索！

集成讯飞语音

恒源云_语音识别与语义处理领域之 NAG 优化器

恒源云(GPUSHARE)_语音识别与语义处理领域之[机器翻译] 21.7 mRASP2