Android – WebRTC video Android and iOS client with H264 decoding capabilities

androiddecoderh.264ioswebrtc

We are looking for someone who could help us develop a client application for Android and iOS that has the ability to properly decode H264. The application can be developed in any way as long as it can be integrated to our existing Android and iOS apps. The main requirement is to render a h264 WebRTC stream on iOS and android device

We have already developed the signaling mechanism using the Janus WebRTC Gateway and the Streaming plugin. The video stream is sent to Janus using a standard UVC camera with hardware H264 encoding and Gstreamer with the following pipeline:

gst-launch-1.0 -v v4l2src device=/dev/video1 ! video/x-h264, profile=baseline, width=1280, height=720, framerate=30/1 ! h264parse ! rtph264pay config-interval=1 pt=126 ! udpsink host=x.x.x.x port=xxxx

The stream can be viewed using a browser on a computer or using Firefox on Android or Safari on iOS. However, no video is displayed using the Native WebRTC library or ReactNative on Android.

This is an example of the offer SDP:

{
 "type":"offer",
 "sdp":
    "v=0
    no=- 1515855541838524 1515855541838524 IN IP4 188.166.53.100
    ns=Mountpoint 99
    nt=0 0
    na=group:BUNDLE video
    na=msid-semantic: WMS janus
    nm=video 9 RTP/SAVPF 126
    nc=IN IP4 188.166.53.100
    na=sendonly
    na=mid:video
    na=rtcp-mux
    na=ice-ufrag:Ip+m
    na=ice-pwd:nKEVyaCWWbgGSz1FHuohS7
    na=ice-options:trickle
    na=fingerprint:sha-256 D2:B9:31:8F:DF:24:D8:0E:ED:D2:EF:25:9E:AF:6F:B8:34:AE:53:9C:E6:F3:8F:F2:64:15:FA:E8:7F:53:2D:38
    na=setup:actpass
    na=rtpmap:126 H264/90000
    na=fmtp:126 profile-level-id=42e01f;packetization-mode=1
    na=rtcp-fb:126 nack
    na=rtcp-fb:126 goog-remb
    na=ssrc:309447223 cname:janusvideo
    na=ssrc:309447223 msid:janus janusv0
    na=ssrc:309447223 mslabel:janus
    na=ssrc:309447223 label:janusv0
    na=candidate:1 1 udp 2013266431 188.166.53.100 58606 typ host
    na=candidate:2 1 udp 2013266431 10.18.0.5 36151 typ host
    na=candidate:3 1 udp 2013266431 10.0.20.1 34608 typ host
    na=end-of-candidates
    n"
}

The ReactNative code can be found here: https://github.com/Nefalas/VideoWebRTC
I have tested it on a Samsung Galaxy S7 and no video is rendered, but I get the following message:

Unrecognized profile 2130706433 for video/avc

Any help would be greatly appreciated! We are ready to remunerate your efforts if you are able to find a solution quickly.

Best Answer

Firefox ships with a software H.264 codec on Android, whereas Chrome and the native WebRTC library for Android do not. This is why Firefox can decode your stream but the native library or React Native (which I assume relies on the Chrome engine) can't.

Chrome and the native WebRTC library do support certain hardware H.264 encoders on Android but it looks like your particular device doesn't have one. Unfortunately, there's no software codec for Chrome or native WebRTC to fall back on in this situation.

Edit: Galaxy S7 should have a hardware H.264 decoder supported by WebRTC, so the issue probably has to do with the parameters of the original H.264 stream. Still, when there's no supported hardware H.264 encoder/decoder on the device, the native WebRTC library as well as Chrome won't be able to render the stream.

Related Topic