一、RTP视频数据包读取并组帧
jitter_buffer.cc webrtc::FrameList::InsertFrame(webrtc::VCMFrameBuffer * frame) 行 58 C++
jitter_buffer.cc webrtc::VCMJitterBuffer::InsertPacket receiver.cc webrtc::VCMReceiver::InsertPacket video_receiver.cc webrtc::vcm::VideoReceiver::IncomingPacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::OnReceivedPayloadData rtp_receiver_video.cc webrtc::RTPReceiverVideo::ParseRtpPacket rtp_receiver_impl.cc webrtc::RtpReceiverImpl::IncomingRtpPacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::ReceivePacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::OnRecoveredPacket ulpfec_receiver_impl.cc webrtc::UlpfecReceiverImpl::ProcessReceivedFec rtp_stream_receiver.cc webrtc::RtpStreamReceiver::ParseAndHandleEncapsulatingHeader rtp_stream_receiver.cc webrtc::RtpStreamReceiver::ReceivePacket rtp_stream_receiver.cc webrtc::RtpStreamReceiver::OnRtpPacket video_receive_stream.cc webrtc::internal::VideoReceiveStream::OnRtpPacket call.cc webrtc::internal::Call::DeliverRtp call.cc webrtc::internal::Call::DeliverPacket webrtcvideoengine2.cc cricket::WebRtcVideoChannel2::OnPacketReceived channel.cc cricket::BaseChannel::OnPacketReceived rtc::MethodFunctor3 rtc::FireAndForgetAsyncClosure<rtc::MethodFunctor3 rtc::AsyncInvoker::OnMessage(rtc::Message * msg) rtc::MessageQueue::Dispatch(rtc::Message * pmsg) rtc::Thread::ProcessMessages(int cmsLoop) rtc::Thread::Run() rtc::Thread::PreRun(void * pv)
1. VideoReceiveStream和AudioReceiveStream都派生自Syncable;
2. VCMJitterBuffer视频帧管理类,是VCMReceiver的成员变量,负责接收网络到来的RTP数据包,并组帧,包含几个frame list:
decodable_frames_:可解码的帧
incomplete_frames_:不完整的帧
当jitterbuffer InsertPacket时,它会首先找到该packet所属于的frame。
3. VCMFrameBuffer代表一帧视频帧
4. VCMSessionInfo成员变量,表示该帧的状态,每接收一帧都会UpdateCompleteSession更新该帧的状态,判断帧完整的依据是:是否有首包和尾包,并且首包和尾包之间的序号是否连续;
5. RTPReceiverVideo类负责解析RTP;
二、RTP音频数据包读取
packet_buffer.cc webrtc::PacketBuffer::InsertPacket packet_buffer.cc webrtc::PacketBuffer::InsertPacketList neteq_impl.cc webrtc::NetEqImpl::InsertPacketInternal neteq_impl.cc webrtc::NetEqImpl::InsertPacket acm_receiver.cc webrtc::acm2::AcmReceiver::InsertPacket audio_coding_module.cc webrtc::AudioCodingModuleImpl::IncomingPacket channel.cc webrtc::voe::Channel::OnReceivedPayloadData rtp_receiver_audio.cc webrtc::RTPReceiverAudio::ParseAudioCodecSpecific rtp_receiver_audio.cc webrtc::RTPReceiverAudio::ParseRtpPacket rtp_receiver_impl.cc webrtc::RtpReceiverImpl::IncomingRtpPacket channel.cc webrtc::voe::Channel::ReceivePacket channel.cc webrtc::voe::Channel::ReceivedRTPPacket channel_proxy.cc webrtc::voe::ChannelProxy::ReceivedRTPPacket audio_receive_stream.cc webrtc::internal::AudioReceiveStream::DeliverRtp call.cc webrtc::internal::Call::DeliverRtp webrtc::internal::Call::DeliverPacket cricket::WebRtcVoiceMediaChannel::OnPacketReceived cricket::BaseChannel::OnPacketReceived rtc::MethodFunctor3 rtc::FireAndForgetAsyncClosure<rtc::MethodFunctor3 rtc::AsyncInvoker::OnMessage rtc::MessageQueue::Dispatch rtc::Thread::ProcessMessages rtc::Thread::Run() rtc::Thread::PreRun
1,音频包读取和视频包读取是一个线程;
三、音频数据解码及渲染线程
packet_buffer.cc webrtc::PacketBuffer::GetNextPacket neteq_impl.cc webrtc::NetEqImpl::ExtractPackets neteq_impl.cc webrtc::NetEqImpl::GetDecision neteq_impl.cc webrtc::NetEqImpl::GetAudioInternal neteq_impl.cc webrtc::NetEqImpl::GetAudio acm_receiver.cc webrtc::acm2::AcmReceiver::GetAudio webrtc::AudioCodingModuleImpl::PlayoutData10Ms webrtc::voe::Channel::GetAudioFrameWithMuted webrtc::voe::Channel::GetAudioFrameWithInfo webrtc::voe::ChannelProxy::GetAudioFrameWithInfo webrtc::AudioReceiveStream::GetAudioFrameWithInfo webrtc::AudioMixerImpl::GetAudioFromSources webrtc::AudioMixerImpl::Mix webrtc::AudioTransportProxy::NeedMorePlayData webrtc::AudioDeviceBuffer::RequestPlayoutData webrtc::AudioDeviceWindowsCore::DoRenderThread webrtc::AudioDeviceWindowsCore::WSAPIRenderThread
1. 该线程主要是从NetEQ模块里取出原始的RTP数据,解码后进行一系列操作,最终渲染出去,可以参考文档;
四、视频数据解码线程
video_render_frames.cc webrtc::VideoRenderFrames::AddFrame incoming_video_stream.cc webrtc::IncomingVideoStream::OnFrame video_stream_decoder.cc webrtc::VideoStreamDecoder::FrameToRender generic_decoder.cc webrtc::VCMDecodedFrameCallback::Decoded generic_decoder.cc webrtc::VCMDecodedFrameCallback::Decoded generic_decoder.cc webrtc::VCMDecodedFrameCallback::Decoded webrtc::VP8DecoderImpl::ReturnFrame webrtc::VP8DecoderImpl::Decode webrtc::VCMGenericDecoder::Decode webrtc::vcm::VideoReceiver::Decode webrtc::vcm::VideoReceiver::Decode webrtc::VideoReceiveStream::Decode webrtc::VideoReceiveStream::DecodeThreadFunction rtc::PlatformThread::Run() rtc::PlatformThread::StartThread
1. 解码后的frame调用VideoRenderFrames的AddFrame函数,存放在incoming_frames_变量;
2. 渲染线程调用VideoRenderFrames的FrameToRender函数,取出incoming_frames_中的解码后的frame;