试图通过 TCP 流式传输 2 路音频？

Posted

技术标签:

【中文标题】试图通过 TCP 流式传输 2 路音频？【英文标题】：Trying to stream 2 way audio over TCP? 【发布时间】：2013-12-08 19:27:33 【问题描述】：

我正在尝试制作一个视频会议应用程序（用 c# 编写），它允许 2 个用户使用 TCP 进行视频会议。此外，用户可以单独进行文字聊天。现在，我有一个工作的视频流，但还没有音频工作。我不确定如何访问麦克风，使用 TCP 流式传输它，然后在其他用户的扬声器上播放它，因为我对 c# 相对较新，并且对使用媒体是全新的。

如果有人可以向我指出示例代码，请帮助我了解如何访问麦克风，或者您认为对我有帮助的任何其他内容，那就太好了。

我按原样附上我的代码以供参考。

WEBCAM.cs

using System;
using System.IO;
using System.Linq;
using System.Text;
using WebCam_Capture;
using System.Windows.Controls;
using System.Collections.Generic;
using System.Windows.Media.Imaging;
using System.Net;
using System.Net.Sockets;
using System.Windows; 

namespace DuckTalk


class WebCam
       
    const int TEXT_VIDEO_NUM = 45674;
    private System.Windows.Controls.TextBox _hostIpAddressBox;


    private WebCamCapture webcam;
    private int FrameNumber = 30;





    public void InitializeWebCam(ref System.Windows.Controls.TextBox hostIpAddressBox)
    

        webcam = new WebCamCapture();
        webcam.FrameNumber = ((ulong)(0ul));
        webcam.TimeToCapture_milliseconds = FrameNumber;
        webcam.ImageCaptured += new WebCamCapture.WebCamEventHandler(webcam_ImageCaptured);


        _hostIpAddressBox = hostIpAddressBox;            
    



    void webcam_ImageCaptured(object source, WebcamEventArgs e)
    
        TcpClient connection = null;
        NetworkStream stream = null;
        byte[] imgBytes;

        try
        
            //Set up IPAddress
            IPAddress ipAddress = IPAddress.Parse(_hostIpAddressBox.Text);
            IPEndPoint ipLocalEndPoint = new IPEndPoint(ipAddress, TEXT_VIDEO_NUM);

            //Connect to TCP
            connection = new TcpClient();
            connection.Connect(ipLocalEndPoint);

            // Get a client stream for reading and writing. 
            stream = connection.GetStream();

            //Send image as bytes
            imgBytes = ImageByteConverter.ImageToBytes((System.Drawing.Bitmap)e.WebCamImage);
            stream.Write(imgBytes, 0, imgBytes.Length);
        
        catch (Exception error)
        
            MessageBox.Show("ERROR: " + error.Message);
        
        finally
        
            // Close everything.
            if (connection != null)
                connection.Close();

            if (stream != null)
                stream.Close();
                    
    


    public void Start()
    
        webcam.TimeToCapture_milliseconds = FrameNumber;
        webcam.Start(0);
    


    public void Stop()
    
        webcam.Stop();

ImageByteConverter.cs

using System;
using System.IO;
using System.Linq;
using System.Text;
using System.Drawing;
using System.Collections.Generic;
using System.Windows.Media.Imaging;


namespace DuckTalk

class ImageByteConverter
               
    public static byte[] ImageToBytes(System.Drawing.Bitmap bitmap)
    
        byte[] byteArray;

        using (MemoryStream stream = new MemoryStream())
        
            bitmap.Save(stream, System.Drawing.Imaging.ImageFormat.Png);
            stream.Close();

            byteArray = stream.ToArray();
        

        return byteArray;
    

    public static BitmapImage BytesToImage(byte[] imgBytes)
    
        var image = new BitmapImage();

        image.BeginInit();
        image.StreamSource = new System.IO.MemoryStream(imgBytes);
        image.EndInit();

        return image;

Window1.xaml.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using System.ComponentModel;
using System.Net;
using System.Net.Sockets;

namespace DuckTalk

/// <summary>
/// Interaction logic for Window1.xaml
/// </summary>
public partial class MainWindow : Window

    const int TEXT_PORT_NUM = 45673;
    const int TEXT_VIDEO_NUM = 45674;
    WebCam webcam;

    public MainWindow()
    
        InitializeComponent();
    

    private void mainWindow_Loaded(object sender, System.Windows.RoutedEventArgs e)
    
        webcam = new WebCam();
        webcam.InitializeWebCam(ref xaml_hostTextBox);

        var _backgroundIMWorker = new BackgroundWorker();
        var _backgroundVidWorker = new BackgroundWorker();

        _backgroundIMWorker.WorkerReportsProgress = true;
        _backgroundVidWorker.WorkerReportsProgress = true;

        // Set up the Background Worker Events
        _backgroundIMWorker.DoWork += new DoWorkEventHandler(keepListeningForInstantMessages);
        _backgroundVidWorker.DoWork += new DoWorkEventHandler(keepListeningForVideoMessages);

        // Run the Background Workers
        _backgroundIMWorker.RunWorkerAsync();
        _backgroundVidWorker.RunWorkerAsync(); 
    

    ///////////////////////////////////////////////////////////////////////////////////////////////
    //
    //
    //  The next 2 functions take care of the instant messaging part of the program
    //
    //
    //
    ///////////////////////////////////////////////////////////////////////////////////////////////
    private void keepListeningForInstantMessages(object sender, DoWorkEventArgs e)
    
        Action<string> displayIncomingMessage = (incomingMsg) =>
        
            xaml_incomingTextBox.Text += "\n\nINCOMING MESSAGE:   " + incomingMsg;
            xaml_incomingTextScroll.ScrollToBottom();
        ;

        Socket connection;
        Byte[] data;
        String msg;

        // create the socket
        Socket listenSocket = new Socket(AddressFamily.InterNetwork,
                                         SocketType.Stream,
                                         ProtocolType.Tcp);

        // bind the listening socket to the port
        IPEndPoint ep = new IPEndPoint(IPAddress.Any, TEXT_PORT_NUM);
        listenSocket.Bind(ep);

        while (true)
        
            msg = "";
            data = new Byte[3000];

            // start listening
            listenSocket.Listen(1);

            //Received a connection
            connection = listenSocket.Accept();

            //Get Data
            connection.Receive(data);

            //Get the message in string format
            msg = System.Text.Encoding.Default.GetString(data);
            msg = msg.Substring(0,msg.IndexOf((char)0));

            //Send message to the UI
            xaml_incomingTextBox.Dispatcher.BeginInvoke(displayIncomingMessage, msg);

            connection.Close();
        
    //end of keepListeningForInstantMessages

    void SendInstantMsg(object sender, RoutedEventArgs e)
    
        TcpClient connection = null;
        NetworkStream stream = null;
        byte[] data;

        xaml_incomingTextBox.Text += "\n\nOUTGOING MESSAGE:   " + xaml_outgoingTextBox.Text;

        try
        
            //Set up IPAddress
            IPAddress ipAddress = IPAddress.Parse(xaml_hostTextBox.Text);
            IPEndPoint ipLocalEndPoint = new IPEndPoint(ipAddress, TEXT_PORT_NUM);

            //Connect to TCP
            connection = new TcpClient();
            connection.Connect(ipLocalEndPoint);

            //Convert text to bytes
            data = System.Text.Encoding.ASCII.GetBytes(xaml_outgoingTextBox.Text);                               

            // Get a client stream for reading and writing. 
            stream = connection.GetStream();

            // Send the message to the connected TcpServer. 
            stream.Write(data, 0, data.Count());

            xaml_outgoingTextBox.Text = "";
        
        catch (Exception error)
        
            MessageBox.Show("ERROR: " + error.Message);
        
        finally
        
            // Close everything.
            if (connection != null)
                connection.Close();

            if (stream != null)
                stream.Close();
        
    //end of SendInstantMsg

    ///////////////////////////////////////////////////////////////////////////////////////////////
    //
    //
    //  The next 2 functions take care of the video part of the program
    //
    //
    //
    ///////////////////////////////////////////////////////////////////////////////////////////////
    private void keepListeningForVideoMessages(object sender, DoWorkEventArgs e)
    
        Action<Byte[]> displayIncomingVideo = (incomingImgBytes) =>
        
            xaml_incomingVideo.Source = ImageByteConverter.BytesToImage(incomingImgBytes);
        ;

        Socket connection;
        Byte[] incomingBytes;
        Byte[] data;
        int offset;
        int numOfBytesRecieved;

        // create the socket
        Socket listenSocket = new Socket(AddressFamily.InterNetwork,
                                         SocketType.Stream,
                                         ProtocolType.Tcp);

        // bind the listening socket to the port
        IPEndPoint ep = new IPEndPoint(IPAddress.Any, TEXT_VIDEO_NUM);
        listenSocket.Bind(ep);

        while (true)
        
            offset = 0;
            numOfBytesRecieved = -1;
            incomingBytes = new Byte[300000];

            // start listening
            listenSocket.Listen(1);

            //Received a connection
            connection = listenSocket.Accept();

            //Get all the data from the connection stream
            while (numOfBytesRecieved != 0)
            
                numOfBytesRecieved = connection.Receive(incomingBytes, offset, 10000, SocketFlags.None);

                offset += numOfBytesRecieved;
            

            data = new Byte[offset];
            Array.Copy(incomingBytes, data, offset);

            //Send image to the UI
            xaml_incomingTextBox.Dispatcher.BeginInvoke(displayIncomingVideo, data);

            connection.Close();
        
    //end of keepListeningForVideoMessages


    private void SendVideoMsg(object sender, RoutedEventArgs e)
    
        xaml_incomingVideo.Visibility   = Visibility.Visible;
        xaml_StopVideoButton.Visibility = Visibility.Visible;
        xaml_SendTextButton.Visibility  = Visibility.Hidden;
        webcam.Start(); 
    


    private void StopVideoMsg(object sender, RoutedEventArgs e)
    
        xaml_incomingVideo.Visibility   = Visibility.Hidden;
        xaml_StopVideoButton.Visibility = Visibility.Hidden;
        xaml_SendTextButton.Visibility  = Visibility.Visible;
        webcam.Stop();
    

//end of Class
//end of NameSpace

【问题讨论】：

I'm relatively new to c# and brand new to using media 然后，我建议搜索一些库。从头开始编写它会很困难。 【参考方案1】：

您应该尝试下载 NAudio，它是一个开源音频库。它附带了一个演示如何使用 UDP 将音频从一台电脑传输到另一台电脑；它位于名为“网络聊天”的 NAudioDemo 应用程序中。

http://naudio.codeplex.com/

您当前使用的 TCP 并不真正推荐用于音频。改用 UDP

http://www.onsip.com/about-voip/sip/udp-versus-tcp-for-voip

或者，如果您不想重新发明***（我不确定您的项目的最终目标是什么），您可以尝试下载我们的 iConf.NET 视频会议 SDK，它可以为您完成大部分工作（但不是免费的）

http://avspeed.com/

【讨论】：

以上是关于试图通过 TCP 流式传输 2 路音频？的主要内容，如果未能解决你的问题，请参考以下文章

通过 tcp 套接字流式传输 PCM 音频

是否可以通过多个设备流式传输音频通道

通过WiFi在Android手机之间流式传输语音

从 MIC 录制并流式传输到 TCP 服务器； MediaRecorder：启动失败：-38

将音频流式传输到其他 iOS 设备（使用多点）并通过本地设备播放音频

通过 socket.io 流式传输实时音频