Classically, a media stream may be sent over communication networks using different transmission protocols. For instance, the User Datagram Protocol (hereafter UDP) uses a simple transmission model with a minimum of protocol mechanism so that media data are provided quickly. This is because UDP does not require prior communications (“handshaking dialogues”) to set up special transmission channels or data paths prior to sending data packets. However, UDP does not ensure that packets are well received and that they are received in the right order, and only once each (i.e. there are no duplicate thereof). Indeed, packets are sent individually and are checked for integrity only if they arrive.
The Real Time Protocol (hereafter RTP), which is based on UDP, is more adapted to video and audio streaming between a server and a client device, over the communication network. According to RTP, payload data are provided with an additional header including an identifier of the data source, a timestamp for synchronization, sequence numbers for managing packet loss and reordering, and a payload type specifying the type (audio/video) and the format (codec) of the transported media.
In order to handle the transmission of packets, RTP also defines the Real Time Control Protocol (hereafter RTCP). RTCP messages are defined for exchanging control messages with different purposes such as monitoring of transmission, correction of transmission errors, and control of the codec.
For instance, Receiver Report (RR) and Sender Report (SR) are RTCP messages sent for estimating the network conditions (e.g. network jitter, loss rate, round trip time (RTT)). Negative Acknowledgment (NACK) messages are for requesting the retransmission of non-received packets. Slice Loss Indication (SLI) and Picture Loss Indication (PLI) are for indicating a loss of a specific part of a video sequence. A Full Intra Request (FIR) is for requiring the encoder to encode a full intra picture in order to limit error propagation in a video sequence and a Temporary Maximum Media Stream Bit Rate Request (TMMBR) is for indicating a bit rate limit to be respected by the encoder. Depending on the events observed during the transmission, an application managing the RTP media session generates adapted RTCP messages.
In the context of video (and audio) streaming, communication devices having several network interfaces are preferably used so that they can perform multi-path streaming (MP-RTP for multi-path RTP). Given the important amount of data to be transmitted over the network, it is particularly advantageous to allow packets (media data or RTCP message(s)) to be sent in parallel over several paths forming several subflows. In the following description, a path is defined as a communication link set between a network interface of a communication device (e.g. server) and another network interface of another communication device (e.g. client).
In general terms, each path between one of the communication interfaces of the server and the communication interface of the client has specific characteristics (bandwidth, latency, jitter, etc.) that are generally different from the characteristics of another path between another communication interface of the server and the communication interface of the client. Also, delivery requirements (transmission requirements) are different depending on the type of packets or messages exchanged. For instance, in terms of bandwidth, the RTCP stream bitrate must be bounded at 5% of the media stream (RTP) path bandwidth. This bitrate has to be shared among the different paths of the multi-path.
In order to achieve an efficient transmission over these different paths depending on their own characteristics, MP-RTP standard, as standardized by the IETF (acronym for Internet Engineering Task Force), provides means for controlling the streaming on each path based on the exchange of specific RTCP messages called multi-path RTCP feedback messages (MP-RTCP) and by adding an extension header into the RTP header containing a path identifier and a sequence number dedicated to a path. An MP-RTCP feedback message aggregates data regarding the performance of each path of the multi-path, and thus allows the server to determine the relative performance (network conditions) of each path. Based on the determined performances, the traffic on each path of the multi-path may be balanced as appropriate (this is called load balancing).
An MP-RTP communication may occur in environment composed of heterogeneous networks, for instance between communication devices having a different number of communication interfaces connected to different types of networks such as wired networks, wireless or mobile networks (e.g. Wi-Fi, 3G/4G). The present invention lies in this context. More specifically, a server-client implementation is considered, with a server able to communicate using a plurality of communication interfaces in parallel while the client (called legacy RTP client) is not aware of multi-path signaling.
An issue is that a legacy RTP client is not compliant with MP-RTP, meaning that it is not aware of multi-path related fields of MP-RTP packets and MP-RTCP messages. Therefore, the specific MP-RTCP messages sent to control each path during the streaming are ignored by the legacy client. Furthermore, no MP-RTCP feedback messages are transmitted to the server whereas these messages are important to control streaming over the different paths of the multi-path RTP session as mentioned. In order to get feedback messages from the legacy client, the server must thus transmit RTCP messages focusing on the whole multi-path RTP session rather than MP-RTCP messages. Thus the server is unable to obtain efficiently network conditions or parameters of each individual path.
Another issue is that when receiving a message (RTCP or data (RTP)) over a given path, the legacy client may transmit a feedback message over anyone of the paths, including on a different path. This may skew network conditions estimations by the server. For instance, when receiving an SR message, the legacy client may respond by transmitting an RR message on the same path or on another path. In the last case, the RTT calculation may be biased since this duration depends on the quality of the path on which the SR has been transmitted but also on the quality of the path on which the RR has been transmitted.
This shows that determining the network conditions of each individual path in case a legacy client is involved is not simple.
Therefore, there is a need for a method of efficiently estimating the network conditions of a multi-path connection between a server and a client not aware of multi-path signaling.
USPTO Backgrounds
USPTO Backgrounds