使用 Audiotrack 在 Android 中播放 javacv-ffmpeg 解码的音频

Posted 2023-02-25

【中文标题】使用 Audiotrack 在 Android 中播放 javacv-ffmpeg 解码的音频

【英文标题】：Playing javacv-ffmpeg decoded audio in Android with Audiotrack

【发布时间】：2014-03-31 11:24:37

【问题描述】：

我正在开发 android 应用程序，我需要在其中播放来自 Red5 服务器的 AAC 实时音频流。 我已经使用 javacv-ffmpeg 成功解码了音频流。 但我的问题是如何播放解码样本中的音频。 我已经尝试过以下方式

int len = avcodec.avcodec_decode_audio4( audio_c,  samples_frame,  got_frame,  pkt2);
if (len <= 0)
    this.pkt2.size(0);
 else 
    if (this.got_frame[0] != 0) 
            long pts = avutil.av_frame_get_best_effort_timestamp(samples_frame);
            int sample_format = samples_frame.format();
                   int planes = avutil.av_sample_fmt_is_planar(sample_format) != 0 ? samples_frame.channels() : 1;
                   int data_size = avutil.av_samples_get_buffer_size((IntPointer)null, audio_c.channels(), samples_frame.nb_samples(), audio_c.sample_fmt(), 1) / planes;

                   if ((samples_buf == null) || (samples_buf.length != planes)) 

                       samples_ptr = new BytePointer[planes];
                       samples_buf = new Buffer[planes];
                   
                           BytePointer ptemp = samples_frame.data(0);
               BytePointer[] temp_ptr = new BytePointer[1];
               temp_ptr[0] = ptemp.capacity(sample_size);
               ByteBuffer btemp = ptemp.asBuffer();
               byte[] buftemp = new byte[sample_size];
               btemp.get(buftemp, 0, buftemp.length);

                           play the buftemp[] with audiotrack.....
        

但是只听到扬声器发出噪音，是否需要对我们从decode_audio4(...) 得到的AVFrame 进行任何处理。 传入的音频流使用 AAC 编解码器正确编码。 任何帮助，建议表示赞赏。 提前致谢。

【问题讨论】：

我也面临同样的问题，.:(

@Ichigo Kurosaki 你找到解决办法了吗？？？如果你找到了，请在这里分享

【参考方案1】：

您可以使用 FFmpegFrameGrabber 类来捕获流。并使用 FloatBuffer 类提取音频。这是一个java示例

public class PlayVideoAndAudio extends Application

    private static final Logger LOG = Logger.getLogger(JavaFxPlayVideoAndAudio.class.getName());
    private static final double SC16 = (double) 0x7FFF + 0.4999999999999999;

    private static volatile Thread playThread;

    public static void main(String[] args)
    
        launch(args);
    

    @Override
    public void start(Stage primaryStage) throws Exception
    
        String source = "rtsp://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov";

        StackPane root = new StackPane();
        ImageView imageView = new ImageView();

        root.getChildren().add(imageView);
        imageView.fitWidthProperty().bind(primaryStage.widthProperty());
        imageView.fitHeightProperty().bind(primaryStage.heightProperty());

        Scene scene = new Scene(root, 640, 480);

        primaryStage.setTitle("Video + audio");
        primaryStage.setScene(scene);
        primaryStage.show();

        playThread = new Thread(() -> 
            try 
                FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(source);
                grabber.start();
                primaryStage.setWidth(grabber.getImageWidth());
                primaryStage.setHeight(grabber.getImageHeight());
                AudioFormat audioFormat = new AudioFormat(grabber.getSampleRate(), 16, grabber.getAudioChannels(), true, true);

                DataLine.Info info = new DataLine.Info(SourceDataLine.class, audioFormat);
                SourceDataLine soundLine = (SourceDataLine) Audiosystem.getLine(info);
                soundLine.open(audioFormat);
                soundLine.start();

                Java2DFrameConverter converter = new Java2DFrameConverter();

                ExecutorService executor = Executors.newSingleThreadExecutor();

                while (!Thread.interrupted()) 
                    Frame frame = grabber.grab();
                    if (frame == null) 
                        break;
                    
                    if (frame.image != null) 
                        Image image = SwingFXUtils.toFXImage(converter.convert(frame), null);
                        Platform.runLater(() -> 
                            imageView.setImage(image);
                        );
                     else if (frame.samples != null) 
                        FloatBuffer channelSamplesFloatBuffer = (FloatBuffer) frame.samples[0];
                        channelSamplesFloatBuffer.rewind();

                        ByteBuffer outBuffer = ByteBuffer.allocate(channelSamplesFloatBuffer.capacity() * 2);

                        for (int i = 0; i < channelSamplesFloatBuffer.capacity(); i++) 
                            short val = (short)((double) channelSamplesFloatBuffer.get(i) * SC16);
                            outBuffer.putShort(val);
                        

                        /**
                         * We need this because soundLine.write ignores
                         * interruptions during writing.
                         */
                        try 
                            executor.submit(() -> 
                                soundLine.write(outBuffer.array(), 0, outBuffer.capacity());
                                outBuffer.clear();
                            ).get();
                         catch (InterruptedException interruptedException) 
                            Thread.currentThread().interrupt();
                        
                    
                
                executor.shutdownNow();
                executor.awaitTermination(10, TimeUnit.SECONDS);
                soundLine.stop();
                grabber.stop();
                grabber.release();
                Platform.exit();
             catch (Exception exception) 
                LOG.log(Level.SEVERE, null, exception);
                System.exit(1);
            
        );
        playThread.start();
    

    @Override
    public void stop() throws Exception
    
        playThread.interrupt();
    

【参考方案2】：

因为你得到的buftemp[]的数据是这个AV_SAMPLE_FMT_FLTP格式的，你必须用SwrContext把它改成AV_SAMPLE_FMT_S16格式，然后你的问题就解决了。