码迷,mamicode.com
首页 > 移动开发 > 详细

How to decode a H.264 frame on iOS by hardware decoding?

时间:2015-11-19 01:55:54      阅读:1197      评论:0      收藏:0      [点我收藏+]

标签:

来源:http://stackoverflow.com/questions/25197169/how-to-decode-a-h-264-frame-on-ios-by-hardware-decoding

 

技术分享技术分享

I have been used ffmpeg to decode every single frame that I received from my ip cam. The brief code looks like this:

-(void) decodeFrame:(unsigned char *)frameData frameSize:(int)frameSize{
   AVFrame frame;
   AVPicture picture;
   AVPacket pkt;
   AVCodecContext *context;
   pkt.data = frameData;
   pat.size = frameSize;
   avcodec_get_frame_defaults(&frame);
   avpicture_alloc(&picture, PIX_FMT_RGB24, targetWidth, targetHeight);
   avcodec_decode_video2(&context, &frame, &got_picture, &pkt);
}

The code woks fine, but it‘s software decoding. I want to enhance the decoding performance by hardware decoding. After lots of research, I know it may be achieved by AVFoundation framework. The AVAssetReader class may help, but I can‘t figure out what‘s the next.Could anyone points out the following steps for me? Any help would be appreciated.

shareimprove this question
 

2 Answers

up vote 11 down vote accepted

iOS does not provide any public access directly to the hardware decode engine, because hardware is always used to decode H.264 video on iOS.

Therefore, session 513 gives you all the information you need to allow frame-by-frame decoding on iOS. In short, per that session:

  • Generate individual network abstraction layer units (NALUs) from your H.264 elementary stream. There is much information on how this is done online. VCL NALUs (IDR and non-IDR) contain your video data and are to be fed into the decoder.
  • Re-package those NALUs according to the "AVCC" format, removing NALU start codes and replacing them with a 4-byte NALU length header.
  • Create a CMVideoFormatDescriptionRef from your SPS and PPS NALUs via CMVideoFormatDescriptionCreateFromH264ParameterSets()
  • Package NALU frames as CMSampleBuffers per session 513.
  • Create a VTDecompressionSessionRef, and feed VTDecompressionSessionDecodeFrame() with the sample buffers
    • Alternatively, use AVSampleBufferDisplayLayer, whose -enqueueSampleBuffer: method obviates the need to create your own decoder.
shareimprove this answer
 
2  
This works as of iOS 8. Note that the 4-byte NALU length header is in big-endian format, so if you have a UInt32 value it must be byte-swapped before copying to the CMBlockBuffer (use CFSwapInt32). – 12on Dec 12 ‘14 at 16:17
    
Thank you 12on, I was banging my head against decode errors for a long time until I tried swapping the bytes like you said. – Greg Feb 27 at 23:30
    
@rpj - can you explain the 3rd (packaging) step? How many NALU frames should I pack (for example all with the same frame number)? – Tomasz Wójcik May 7 at 8:57
    
This link provide more detail explanation on how to decode h.264 step by step: stackoverflow.com/a/29525001/3156169 – ChihHao May 22 at 2:40
 

Edit:

This link provide more detail explanation on how to decode h.264 step by step: stackoverflow.com/a/29525001/3156169

Original answer:

I watched the session 513 "Direct Access to Video Encoding and Decoding" in WWDC 2014 yesterday, and got the answer of my own question.

The speaker says:

We have Video Toolbox(in iOS 8). Video Toolbox has been there on OS X for a while, but now it‘s finally populated with headers on iOS.This provides direct access to encoders and decoders.

So, there is no way to do hardware decoding frame by frame in iOS 7, but it can be done in iOS 8.

Is there anyone figure out how to directly access to video encoding and decoding frame by frame in iOS 8?

shareimprove this answer
 
    
Thank you so much for: "Direct Access to Video Encoding and Decoding" – user1748502 Jun 11 at 22:49

How to decode a H.264 frame on iOS by hardware decoding?

标签:

原文地址:http://www.cnblogs.com/sunminmin/p/4976391.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!