(Kusan) kyamarar gidan yanar gizo mara amfani daga mai lilo. Sashe na 2. WebRTC

Ko ta yaya a daya daga tsoffin labaran da aka yi watsi da su, na rubuta game da yadda sauƙi da sauƙi za ku iya watsa bidiyo daga zane ta hanyar yanar gizo. A cikin wannan labarin, na ɗan yi magana game da yadda ake ɗaukar bidiyo daga kyamara da sauti daga makirufo ta amfani da MediaStream API, yadda ake ɓoye rafin da aka karɓa kuma aika ta ta shafukan yanar gizo zuwa uwar garken. Duk da haka, a gaskiya ba sa yin wannan, don watsa shirye-shirye suna amfani da ko dai software na musamman wanda ke buƙatar shigarwa da kuma daidaita shi: kashe-kashe, wannan na iya zama Bude Fasahar Watsa shirye-shirye, ko kuma suna amfani da WebRTC, wanda ke aiki kai tsaye daga cikin akwatin, wato, baya buƙatar shigar da kowane plugins a la flash player, wanda za a yanke shi daga mai binciken Chromium a cikin Disamba.

Yau za mu yi magana game da WebRTC.

Sadarwar Yanar Gizo ta Real-Time (WebRTC) ba ƙa'ida ɗaya ba ce, tarin ƙa'idodi ne, ka'idoji da JavaScript API waɗanda tare suke ba da haɗin kai-da-tsara na ainihin-lokacin sadarwar bidiyo-audio, kuma ana iya amfani da su don canja wurin kowane abu. binary data. Yawancin lokaci browsers suna aiki azaman abokan aiki, amma kuma yana iya zama aikace-aikacen hannu, misali. Don tsara sadarwar p2p tsakanin abokan ciniki, ana buƙatar tallafin mai bincike don nau'ikan bidiyo da rikodin sauti daban-daban, tallafi ga ka'idodin hanyoyin sadarwa iri-iri, tabbatar da hulɗar kayan aiki tare da mai binciken (ta hanyar Layer OS): kyamarar gidan yanar gizo, katunan sauti. Duk wannan hodgepodge na fasaha yana ɓoye a bayan JavaScript API abstraction don dacewa da mai haɓakawa.

Duk ya ragu zuwa API guda uku:

• MediaStream API — An wargaje a ƙarshe, a yau zan ɗan ƙara yin rubutu game da shi. Yana aiki don karɓar rafukan bidiyo / sauti daga kayan masarufi

• RTCPeer Connection - Yana ba da sadarwa tsakanin abokan ciniki biyu (p2p)

• RTCDataChannel - yana aiki don canja wurin bayanan sabani tsakanin abokan ciniki biyu

Ana shirya rafukan sauti da bidiyo don watsawa

Duk yana farawa da "ɗaukar" kyamarar gidan yanar gizon da rafukan watsa labarai na makirufo. Tabbas, rafukan rafuka ba su dace da shirya taron tarho ba, kowane rafi yana buƙatar sarrafa shi: haɓaka inganci, daidaita sauti tare da bidiyo, sanya alamun aiki tare a cikin rafi na bidiyo, da tabbatar da bitrate daidai da canjin bandwidth koyaushe na tashar. . Mai binciken yana kula da duk wannan, mai haɓakawa ba ya ma damu game da samar da ɓoye don rafukan watsa labarai. A cikin burauzar zamani, akwai riga-kafi na software don ɗauka, inganta inganci (cire sautin murya da amo daga sauti, inganta hoto), rikodin bidiyo da sauti. Ana nuna tsarin Layer a cikin fig. 1:

Shinkafa 1. Yadudduka na sarrafa sauti da bidiyo a cikin browser

Duk aiki yana faruwa kai tsaye a cikin burauzar kanta, babu ƙari. babu plugins da ake buƙata. Koyaya, har yanzu abubuwa ba su da ƙarfi don 2020. Akwai masu bincike waɗanda har yanzu ba su goyi baya sosai ba MediaStream API, zaku iya bin hanyar haɗin yanar gizo kuma ku ga teburin dacewa a ƙasan ƙasa. IE musamman yana sake ban takaici.

Kuna iya yin abubuwa masu ban sha'awa sosai tare da rafukan da aka karɓa: zaku iya clone, canza ƙudurin bidiyo, sarrafa ingancin sauti, zaku iya ɗauka da "ƙugiya" rafin Media Stream zuwa yi alama kuma dubi kanka a shafin html. Ko kuma za ku iya zana rafi a kan zane, sannan ku saita WebGL ko CSS3, sannan ku yi amfani da tacewa iri-iri zuwa bidiyo, ɗaukar bidiyon da aka sarrafa daga zane sannan ku aika ta hanyar sadarwar zuwa uwar garken, canza lambar kuma buga wa kowa (sannu bigo live, twitch da sauransu). Anan ba zan yi nazarin yadda ake yin irin waɗannan abubuwa ba, zan ba da misalai biyu da aka samu a gidan yanar gizon:

https://jeeliz.com/ - Mutanen suna yin CV na ainihi a cikin Javascript. Suna da duka arsenal dakunan karatu na js daban-daban don aiki tare da rafi na bidiyo akan zane: gano fuskoki, abubuwa, amfani da tacewa (mask, kamar a cikin Instagram), da sauransu. Kyakkyawan misali na yadda zaku iya aiwatar da bidiyo na ainihin lokacin kai tsaye a cikin mai bincike ba tare da ƙarin plugins ba.

API ɗin Canvas kama StreamStream - Takardun API don ɗaukar rafin bidiyo daga zane. An riga an goyi bayan Chrome, Opera da Firefox

RTCPeer Connection

Don haka mun zo ga batun, amma ta yaya za a zahiri canja wurin bidiyo zuwa wani mai amfani? Ya zo kan gaba RTCPeer Connection. A takaice, a kusan wannan matakin kuna buƙatar ƙirƙirar abu na RTCPeerConnection:

const peerConnection = new RTCPeerConnection({
  iceServers: [{
    urls: 'stun:stun.l.google.com:19302'
  }]
});

Mun saka iceServers a matsayin ɗaya daga cikin zaɓuɓɓuka - wannan sabar ce da ke taimakawa wajen samar da haɗin kai tsakanin masu bincike biyu a bayan NAT'om. Wato, an magance matsalar anan: ta yaya za a gano ip na interlocutor idan yana bayan NAT na mai ba da shi? Ka'idar ICE ta zo don ceto, a zahiri, ICE ba ta shafi WebRTC kwata-kwata, amma ƙari akan wancan daga baya.

A baya, mun sami rafukan mai amfani:

navigator.mediaDevices.getUserMedia({ video: true, audio: true }).then(stream => {
  // Usermedia-потоки, обычно это видео и аудио 
  const tracks = stream.getTracks();

   for (const track of tracks) {
     // каждый трек присоединяем к peerConnection
     peerConnection.addTrack(track);
   }
}).catch(console.error);

Bayan haka, taron tattaunawar da ake buƙata ya kunna wuta akan haɗin haɗin gwiwa, a cikin mai kula da shi dole ne mu ƙirƙiri tayin (cikin sharuddan SDP - Protocol Bayanin Zama) kuma sanya shi zuwa haɗin haɗin gwiwa ta hanyar saitiLocalDescription. Game da SDP - menene kuma game da tayin da tsarin amsa - za mu kara magana.

Bayan sanya LocalDescription peerConnection, mai binciken yana "tattarawa" 'yan takarar kankara, wato, ya sami hanyoyi daban-daban don sadarwa ta hanyar NAT. Lamarin da ya barke a jihar baki daya. A cikin mai kula da canjin jiha, muna ba da damar haɗi zuwa rafin webrtc-signaling-server don musanya Bayanin Zama tsakanin takwarorina:

peerConnection.oniceconnectionstatechange = (event) => {
      console.log('Connection state: ', peerConnection.iceConnectionState);

      if (peerConnection.iceConnectionState === 'connected') {
        // Можем активировать кнопку Start broadcast
        setBroadcasting(true);
        setBroadcastingBtnActive(true);
      }
    };

// Событие срабатывает сразу, как только добавился медаиапоток в peerConnection
peerConnection.onnegotiationneeded = (event) => {
      // Создаем и назначаем SDP offer
      peerConnection.createOffer().
        then((offer) => peerConnection.setLocalDescription(offer)).
        catch(console.error);
    };

// Событие срабатывает каждый раз, как появляется ICE кандидат
peerConnection.onicegatheringstatechange = (ev) => {
      let connection = ev.target;

      // Now we can activate broadcast button
      if (connection.iceGatheringState === 'complete') {
        let delay = 50;
        let tries = 0;
        let maxTries = 3;

        let timerId = setTimeout(function allowStreaming() {
          if (isOnline) {
            setBroadcastingBtnActive(true);
            return;
          }

          if (tries < maxTries) {
            tries += 1;
            delay *= 2;
            timerId = setTimeout(allowStreaming, delay);
          } else {
            // TODO: show user notification
            console.error("Can't connect to server");

            alert("Can't connect to server");
          }
        }, delay);
      }
    };

Webrtc-signaling-server shine uwar garken da ake buƙata don musanya bayanin zaman tsakanin takwarorinsu biyu, yana iya zama mafi sauƙi websocket ko xhr-uwar garken akan kowane PL. Ayyukansa mai sauƙi ne: karɓar bayanin zama daga wani ɗan'uwan kuma canza shi zuwa wani.

Bayan musayar bayanin Zama, bangarorin biyu suna shirye don watsa shirye-shiryen watsa shirye-shiryen bidiyo da karɓar rafukan bidiyo, a gefen da ke karɓar rafi na bidiyo, an kunna taron ontrack akan peerConnection, a cikin mai sarrafa wanda, za a iya sanya waƙoƙin da aka karɓa zuwa kuma duba mafi so interlocutor. Ƙarin ka'idar da cikakkun bayanai.

Hanyoyin haɗi da wallafe-wallafe:

https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection - takardun shaida RTCPeer Connection

https://github.com/pion/webrtc - aiwatar da ka'idojin WebRTC akan tafiya

https://webrtcforthecurious.com/ - littafi daga mahaliccin pion

https://hpbn.co/ - Littattafan Babban Ayyukan Browser Networking. An tattauna batutuwan tabbatar da babban aiki na aikace-aikacen yanar gizo dalla-dalla. A ƙarshe, an kwatanta WebRTC. Tabbas littafin ya tsufa (2013), amma bai rasa dacewarsa ba.

A cikin sashi na gaba, Ina so in ba da ɗan ƙaramin kaso na ka'idar kuma a aikace don nazarin liyafar da sarrafa rafin bidiyo akan sabar ta amfani da pion, canzawa zuwa HLS ta hanyar ffmpeg don watsa shirye-shirye na gaba ga masu kallo a cikin mai binciken.

Ga marasa hakuri: Nau'in ɗanyen samfurina na watsa bidiyo daga kyamarar gidan yanar gizo don amsawa ta hanyar sabar tushen pion a cikin twitch (wannan gwaji ne kawai).

source: www.habr.com