1 rtmp是什么:
Adobe’s Real Time Messaging Protocol (RTMP) provides a bidirectional message multiplex service over a reliable stream transport, such as TCP [RFC0793], intended to carry parallel streams of video, audio, and data messages, with associated timing information, between a pair of communicating peers. Implementations typically assign different priorities to different classes of messages, which can effect the order in which messages are enqueued to the underlying stream transport when transport capacity is constrained.
说白了就是基于tcp等可靠协议的实Adobe开发的实时信息协议,主要用于流媒体如音频视频等上,携带时间信息来做同步。
rtmp基于tcp
2 rtmp中的元素:
- Payload: The data contained in a packet, for example audio samples or compressed video data. The payload format and interpretation are beyond the scope of this document.
- Packet: A data packet consists of fixed header and payload data. Some underlying protocols may require an encapsulation of the packet to be defined.
- Port: The “abstraction that transport protocols use to distinguish among multiple destinations within a given host computer. TCP/IP protocols identify ports using small positive integers.” The transport selectors (TSEL) used by the OSI transport layer are equivalent to ports.
- Transport address: The combination of a network address and port that identifies a transport-level endpoint, for example an IP address and a TCP port. Packets are transmitted from a source transport address to a destination transport address.
- Message stream: A logical channel of communication in which messages flow.
- Message stream ID: Each message has an ID associated with it to identify the message stream in which it is flowing.
- Chunk: A fragment of a message. The messages are broken into smaller parts and interleaved before they are sent over the network. The chunks ensure timestamp-ordered end-to-end delivery of all messages, across multiple streams.
- Chunk stream: A logical channel of communication that allows flow of chunks in a particular direction. The chunk stream can travel from the client to the server and reverse.
- Chunk stream ID: Every chunk has an ID associated with it to identify the chunk stream in which it is flowing.
- Multiplexing: Process of making separate audio/video data into one coherent audio/video stream, making it possible to transmit several video and audio simultaneously.
- DeMultiplexing: Reverse process of multiplexing, in which interleaved audio and video data are assembled to form the original audio and video data.
- Remote Procedure Call (RPC): A request that allows a client or a server to call a subroutine or procedure at the peer end.
- Metadata: A description about the data. The metadata of a movie includes the movie title, duration, date of creation, and so on.
- Application Instance: The instance of the application at the server with which the clients connect by sending the connect request.
- Action Message Format (AMF): A compact binary format that is used to serialize ActionScript object graphs. AMF has two versions: AMF 0 [AMF0] and AMF 3 [AMF3].
3 字节序,对齐和时间格式:
所有整数字段都以网络字节顺序传送,字节零是显示的第一个字节,位零是字或字段中的最高有效位。这种字节顺序通常称为大端。传输顺序在 Internet 协议 [RFC0791] 中有详细描述。除非另有说明,否则本文档中的数字常量为十进制(以 10 为基数)。
除非另有说明,RTMP中的所有数据都是字节对齐的;例如,16 位字段可能位于奇数字节偏移处。在指示填充的地方,填充字节应该具有零值。
RTMP 中的时间戳以相对于未指定时期的整数毫秒数给出。通常,每个流将以 0 的时间戳开始,但这不是必需的,只要两个端点在 epoch 上达成一致即可。请注意,这意味着跨多个流(尤其是来自不同主机)的任何同步都需要 RTMP 之外的一些额外机制。
因为时间戳是 32 位长,它们每 49 天 17 小时 2 分钟 47.296 秒翻转一次。因为流被允许连续运行,可能持续数年,RTMP 应用程序在处理时间戳时应该使用序列号算法 [RFC1982],并且应该能够处理回绕。例如,应用程序假设所有相邻的时间戳都在 2^31 - 1 毫秒内,因此 10000 在 4000000000 之后,而 3000000000 在 4000000000 之前。
时间戳增量也指定为相对于前一个时间戳的无符号整数毫秒数。时间戳增量可以是 24 位或 32 位长。
4 RTMP CHUNK STREAM:
- 本节指定实时消息传递协议块流(RTMP 块流)。它为更高级别的多媒体流协议提供多路复用和打包服务。
- 虽然 RTMP 块流被设计为与实时消息传递协议(第 6 节)一起使用,但它可以处理任何发送消息流的协议。每条消息都包含时间戳和有效负载类型标识。 RTMP Chunk Stream 和 RTMP 一起适用于各种音视频应用,从一对一和一对多直播到视频点播服务再到交互式会议应用。
- 当与可靠的传输协议(如 TCP [RFC0793])一起使用时,RTMP 块流提供跨多个流的所有消息的有保证的时间戳顺序端到端传递。 RTMP 块流不提供任何优先级或类似形式的控制,但可以被更高级别的协议用于提供此类优先级。
- 例如,实时视频服务器可能会根据发送时间或确认每条消息的时间,选择为慢速客户端丢弃视频消息,以确保及时接收音频消息。
- RTMP Chunk Stream 包含自己的带内协议控制消息,并且还提供了一种机制让上层协议嵌入用户控制消息。
4.1 Message Format:
The format of a message that can be split into chunks to support multiplexing depends on a higher level protocol.
The message format SHOULD however contain the following fields which are necessary for creating the chunks.
可以拆分为块以支持多路复用的消息格式取决于更高级别的协议。 然而,消息格式应该包含创建块所必需的以下字段。
1 | Timestamp: Timestamp of the message. This field can transport 4 bytes. |
4.2 握手:
- RTMP 连接以握手开始。 握手与协议的其余部分不同; 它由三个静态大小的块组成,而不是由带有标题的可变大小的块组成。 客户端(发起连接的端点)和服务器各自发送相同的三个块。 为了说明,这些块在客户端发送时将被指定为 C0、C1 和 C2; 由服务器发送时的 S0、S1 和 S2。
Handshake Sequence
- 握手从客户端发送 C0 和 C1 块开始。 客户端必须等到收到 S1 后再发送 C2。 客户端必须等到收到 S2 后再发送任何其他数据。
- 服务器必须等到收到 C0 后再发送 S0 和 S1,也可以等到 C1 之后。 服务器必须等到收到 C1 后再发送 S2。 服务器必须等到收到 C2 后再发送任何其他数据。
C0 and S0 Format
The C0 and S0 packets are a single octet, treated as a single 8-bit integer field:
1 | 0 1 2 3 4 5 6 7 |
C1 and S1 Format
The C1 and S1 packets are 1536 octets long, consisting of the following fields:
1 | 0 1 2 3 |
C2 and S2 Format
The C2 and S2 packets are 1536 octets long, and nearly an echo of S1 and C1 (respectively), consisting of the following fields:
1 | 0 1 2 3 |
交互图
1 | +-------------+ +-------------+ |
注意就是整个rtmp握手包的格式,没有额外的头;
5 chunking:
在握手之后,该连接多路复用一个或多个chunk streams.每个chunk stream从一个消息流携带一种类型的消息。每个chunk 被创建时有一个独立的ID,关联到,称为chunk streamID。这个chunks被网络进行传输。
当传输时,每个chunk 必须被完全发送,在发送下个chunk之前。 在接收方,多个chunks被组装为message,基于chunk ID。
chunking允许将高层协议的大message分解为小的messages, 例如,为了避免大的低优先级的message(如视频),阻塞了小的高优先级message(如音频或控制)
chunking也允许小message携带少的头发送信息,as the chunk header contains a compressed representation of information that would otherwise have to be included in the message itself.
chunk size是可配置的。他能被Set Chunk Size 控制message设置。 更大的chunk size 减少了cpu的使用,但也导致更大的写入,这可能会延迟较低带宽上的其他内容。
Smaller chunks are not good for high bit rate streaming. Chunk size is maintained independently for each direction.
chunk头格式
1 | +--------------+----------------+--------------------+--------------+ |
几个问题:chunk 大小,和tcp数据流的关系,rtmp能不能基于udp? chunk大小和实际流有关吗?一个chunk可以包含多个message?
rtmp 基于流的,streamid,分message chunk, 有message header,chunk header字段basic header:
1
2
3
4
5
6
71) basic header 的大小取决于chunk stream id的大小;它编码了chunk stream ID和chunk type(由fmt域进行配置)。
chunk type决定了编码messag header的格式。chunk basic 头可能是1,2,3 字节,取决于chunk stream ID大小。实现应该使用最小的能表示ID的。
2)协议支持到65597个streamID ,为3-65599值。 IDs 0,1,2,为保留。
3)IDs为0时即cs id这个字节为0 意思是两个字节来形成一个ID,在范围 64-319,(即第二个字节+64 :255+64->309). 字节值为1表示3个字节来形成这个ID:64-65599((the third byte)*256 + the second byte + 64).
4)值为: 3-63 represent the complete stream ID。一个字节足已。Chunk Stream ID with value 2 is reserved for low-level
5)The bits 0-5 (least significant) in the chunk basic header represent the chunk stream ID.Chunk stream IDs 2-63 can be encoded in the 1-byte version of this field.
1
2
3
4
5
60 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|fmt| cs id |
+-+-+-+-+-+-+-+-+
Chunk basic header 1Chunk stream IDs 64-319 can be encoded in the 2-byte form of the header. ID is computed as (the second byte + 64).
1
2
3
4
5
6
70 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|fmt| 0 | cs id - 64 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Chunk basic header 2Chunk stream IDs 64-65599 can be encoded in the 3-byte version of this field. ID is computed as ((the third byte)*256 + (the second byte) + 64).
1
2
3
4
5
60 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|fmt| 1 | cs id - 64 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Chunk basic header 3
chunk message header:
fmt两位决定了四种类型00,01,,10,11: (实现应使用每个块消息标头的最紧凑的表示。)
这里的fmt永远是两位,则决定了 chunk message header的类型;注意不是message的类型;
主要指示流中的首个message chunk和message和chunk的切分情况1
2
3
4
5
6
7
8
9
10
11
12
13type 0-3;
/**
* parse the message header.
* 3bytes: timestamp delta, fmt=0,1,2 即0,1,2都有的字段
* 3bytes: payload length, fmt=0,1 即0,1都有的字段
* 1bytes: message type, fmt=0,1
* 4bytes: stream id, fmt=0
* where:
* fmt=0, 0x0X
* fmt=1, 0x4X
* fmt=2, 0x8X
* fmt=3, 0xCX
*/type 0: 是chunk stream的开始使用以及每当流时间戳回滚?11Bytes。 This type MUST be used at the start of a chunk stream, and whenever the stream timestamp goes backward (e.g., because of a backward seek).
1 | 0 1 2 3 |
type 1: 不含streamid,意味着和前一个chunk的streamid相同,所以第一个不能是1,带message length和type意味着Streams with variable-sized messages (for example, many video formats) SHOULD use this format for the first chunk of each new message after the first.
7Bytes1
2
3
4
5
6
7
8
90 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp delta |message length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| message length (cont) |message type id|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Chunk Message Header - Type 1type 2:只含timestamp delta: 意味着streamid从前一个拿,message length也和前一个chunk相同,一般在固定长度的chunksize下用,非可变chunk. 和type
3bytes:1
2
3
4
5
6
70 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp delta |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Chunk Message Header - Type 2type 3: 不带头,即type3的chunk ,和前一个chunk属于同一个message;它的streamid和其他字段和前面的一样,意味着其实timestamp应该加上之前的chunk;
message组织方式:
- 当一个message被拆成chunk时,除了第一个message chunk外的所有message chunk都用这个类型;Refer to Example 2 (Section 5.3.2.2). A stream consisting of messages of exactly the same size, stream ID and spacing in time SHOULD use this type for all chunks after a chunk of Type 2. Refer to Example 1 (Section 5.3.2.1). If the delta between the first message and the second message is same as the timestamp of the first message, then a chunk of Type 3 could immediately follow the chunk of Type 0 as there is no need for a chunk of Type 2 to register the delta. If a Type 3 chunk follows a Type 0 chunk, then the timestamp delta for this Type 3 chunk is the same as the timestamp of the Type 0 chunk. 看github文档例子
- 请注意,从两个示例中可以看出,块类型 3 可以以两种不同的方式使用。第一个是指定消息的延续。第二个是指定一个新消息的开始,它的头可以从现有的状态数据中导出。
- 总结: 流开始为type 0 ,中间依情况为1,2位message的首chunk,当message 被分割为多个chunk时,出现type3,此时当第一个chunk为type0时,message的长度在第一个chunk指示;可以算出来总长;或者当type0,type2,type3的情况时,type0chunk 的message length==type2==type3;
- 其他情况根据具体实现决定;比如type1会带message size此时可能和0-3一样;
1 | messageheader的 实际取的是chunk message header; |
Example 1
This example shows a simple stream of audio messages. This example demonstrates the redundancy of information.
1 | +---------+-----------------+-----------------+-----------------+ |
PS:
有点奇怪这里,chunk3 和chunk4是message3和message4的,他们时间戳不同,这里按理说应该是type2.但是是type3,虽然解释说有优化,但是还是奇怪。
The next table shows chunks produced in this stream. From message 3 onward, data transmission is optimized. There is only 1 byte of overhead per message beyond this point.
example2: 这里假设chunk的最大是128,这里的message超出了;会被分为多个chunk.
This example illustrates a message that is too long to fit in a 128 byte chunk and is broken into several chunks.
1 | +-----------+-------------------+-----------------+-----------------+ |
一般实现会把chunk size设置比较大,这样message不会出现上述被分割为多个chunk的情况,则chunk basic header基本只会出现一次,对一个message来说,而message header内容也是一样的,也简化了处理。
message分类等:
message type id:
1 | Protocol Control Messages: |
关于Protocol Control Messages:
set chunk size:
is used to notify the peer of a new maximum chunk size. 默认值是128bytes, 但client和server可以改变这个值。然后updates its peer using this message;
例如一个client想发送131bytes 的音频数据,然后此时的chunk size是128,这种情况下,client可以发送这个消息来通知server,现在的chunk size是131,然后client就可以在一个chunk 中发送131大小的数据了;The maximum chunk size is maintained independently for each direction.
1 | 0 1 2 3 |
- 0: This bit MUST be zero.
- chunk size (31 bits): 填新的max chunk size,字节为单位,有效长是1-2147483647 (0x7FFFFFFF) ,然而,所有大于16777215 (0xFFFFFF) 是相等的,因为
没有chunk 大于一个message,而message的最大值是16777215 bytes. 字段大小长度限制;
Abort Message detail:
is used to notify the peer if it is waiting for chunks to complete a message, then to discard the partially received message over a chunk stream
用来通知对端,若正在等待chunks 来完成一个message,则丢弃已经收到message的部分chunk.
The peer receives the chunk stream ID as this protocol message’s payload. An application may send this message when closing in order to indicate that further processing of the messages is not required.
对端收到这个后。一个app可能会发送这个消息,当有序关闭时,暗示之后对这个messages的processing不被required;
1 | 0 1 2 3 |
- chunk stream ID (32 bits): This field holds the chunk stream ID, whose current message is to be discarded.
Acknowledgement:即ack 因为rtmp目前主流实现是基于tcp的,所以这个基本没怎么用到。
客户端或服务器必须在收到等于窗口大小的字节后向对等方发送确认。窗口大小是发送方在未收到接收方确认的情况下发送的最大字节数。此消息指定序列号,即到目前为止接收到的字节数。
1 | 0 1 2 3 |
- sequence number (32 bits): This field holds the number of bytes received so far.
Window Ack size: 即多久(多少数据传输了要发ack)
和上面的关联:
客户端或服务器发送此消息以通知对等方要在发送确认之间使用的窗口大小。在发送方发送窗口大小字节后,发送方希望得到其对等方的确认。接收对等方必须在收到自上次发送确认后指定的字节数后发送确认(第 5.4.3 节),或者从会话开始(如果尚未发送确认)。、
1 | 0 1 2 3 |
set peer bandwidth:
The client or the server sends this message to limit the output bandwidth of its peer. The peer receiving this message limits its output bandwidth by limiting the amount of sent but unacknowledged data to the window size indicated in this message. The peer receiving this message SHOULD respond with a Window Acknowledgement Size message if the window size is different from the last one sent to the sender of this message.
client或server发送这个message去限制对端的output bandwidth. 对端收到这个message后,限制它的output bandwidth 通过限制已发送但未确认的数据到这个消息指示的windows size. (PS:对端输出时,需要等这个message指示的window size范围的数据被ack后才能继续发,意味着如果小了,则输出会变小).
接收方收到这个message 应该response with a window ack size message ,如果它的window size 和这个message中的不同;
1
2
3
4
5
6
7
80 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Acknowledgement Window size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Limit Type |
+-+-+-+-+-+-+-+-+
Payload for the ‘Set Peer Bandwidth’ protocol messageThe Limit Type is one of the following values:
1
2
30 - Hard: The peer SHOULD limit its output bandwidth to the indicated window size.对方应该将其输出带宽限制在指示的窗口大小。
1 - Soft: The peer SHOULD limit its output bandwidth to the the window indicated in this message or the limit already in effect, whichever is smaller.对方应该将其输出带宽限制在此消息中指示的窗口或已经生效的限制,以较小者为准。
2 - Dynamic: If the previous Limit Type was Hard, treat this message as though it was marked Hard, otherwise ignore this message.如果先前的限制类型为Hard,请处理此消息,将其视为hard. 否则忽略此消息。shared obj message detail:
共享对象是在多个客户端、实例等之间同步的 Flash 对象(名称值对的集合)。 AMF0 的消息类型 19 和 AMF3 的消息类型 16 保留用于共享对象事件。每条消息可以包含多个事件。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56+------+------+-------+-----+-----+------+-----+ +-----+------+-----+
|Header|Shared|Current|Flags|Event|Event |Event|.|Event|Event |Event|
| |Object|Version| |Type |data |data |.|Type |data |data |
| |Name | | | |length| |.| |length| |
+------+------+-------+-----+-----+------+-----+ +-----+------+-----+
| |
|<- - - - - - - - - - - - - - - - - - - - - - - - - - - - - >|
| AMF Shared Object Message body |
The shared object message format
The following event types are supported:
+---------------+--------------------------------------------------+
| Event | Description |
+---------------+--------------------------------------------------+
| Use(=1) | The client sends this event to inform the server |
| | about the creation of a named shared object. |
+---------------+--------------------------------------------------+
| Release(=2) | The client sends this event to the server when |
| | the shared object is deleted on the client side. |
+---------------+--------------------------------------------------+
| Request Change| The client sends this event to request that the |
| (=3) | change the value associated with a named |
| | parameter of the shared object. |
+---------------+--------------------------------------------------+
| Change (=4) | The server sends this event to notify all |
| | clients, except the client originating the |
| | request, of a change in the value of a named |
| | parameter. |
+---------------+--------------------------------------------------+
| Success (=5) | The server sends this event to the requesting |
| | client in response to RequestChange event if the |
| | request is accepted. |
+---------------+--------------------------------------------------+
| SendMessage | The client sends this event to the server to |
| (=6) | broadcast a message. On receiving this event, |
| | the server broadcasts a message to all the |
| | clients, including the sender. |
+---------------+--------------------------------------------------+
| Status (=7) | The server sends this event to notify clients |
| | about error conditions. |
+---------------+--------------------------------------------------+
| Clear (=8) | The server sends this event to the client to |
| | clear a shared object. The server also sends |
| | this event in response to Use event that the |
| | client sends on connect. |
+---------------+--------------------------------------------------+
| Remove (=9) | The server sends this event to have the client |
| | delete a slot. |
+---------------+--------------------------------------------------+
| Request Remove| The client sends this event to have the client |
| (=10) | delete a slot. |
+---------------+--------------------------------------------------+
| Use Success | The server sends this event to the client on a |
| (=11) | successful connection. |
+---------------+--------------------------------------------------+
Aggregate Message detail:
An aggregate message is a single message that contains a series of RTMP sub-messages using the format described follow:
1 | +---------+-------------------------+ |
聚合消息和第一个子消息的时间戳之间的差异是用于将子消息的时间戳重新规范化为流时间尺度的偏移量。偏移量被添加到每个子消息的时间戳中,以达到规范化的流时间。第一个子消息的时间戳应该与聚合消息的时间戳相同,因此偏移量应该为零。
The back pointer contains the size of the previous message including its header. It is included to match the format of FLV file and is used for backward seek.
使用这个message 有以下几个性能的好处:
- 1 chunk stream 能在一个chunk中最多发一个message,因此,增加了chunk size并使用aggregate message能减少发送chunk的次数;
- 2 sub-message被存储在连续的内存中,对系统调用发送到网络上更有效率;
关于type4 User Control Messages
- User Control Message detail:The client or the server sends this message to notify the peer about the user control events.
- 这种消息,它的Chunk basic header和message header 中: message stream id为0, chun steam id为2,timestamp被忽略
- This message carries Event type and Event data.
- event data字段的大小是可变的。然而,在消息必须通过 RTMP 块流层的情况下,最大块大小(第 5.4.1 节)应该足够大以允许这些消息适合单个块。
以下为一些协议上预定义的event type 和描述:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53+---------------+--------------------------------------------------+
| Event | Description |
+---------------+--------------------------------------------------+
| Stream Begin | The server sends this event to notify the client |
| (=0) | that a stream has become functional and can be |
| | used for communication. By default, this event |
| | is sent on ID 0 after the application connect |
| | command is successfully received from the |
| | client. The event data is 4-byte and represents |
| | the stream ID of the stream that became |
| | functional. |
+---------------+--------------------------------------------------+
| Stream EOF | The server sends this event to notify the client |
| (=1) | that the playback of data is over as requested |
| | on this stream. No more data is sent without |
| | issuing additional commands. The client discards |
| | the messages received for the stream. The |
| | 4 bytes of event data represent the ID of the |
| | stream on which playback has ended. |
+---------------+--------------------------------------------------+
| StreamDry | The server sends this event to notify the client |
| (=2) | that there is no more data on the stream. If the |
| | server does not detect any message for a time |
| | period, it can notify the subscribed clients |
| | that the stream is dry. The 4 bytes of event |
| | data represent the stream ID of the dry stream. |
+---------------+--------------------------------------------------+
| SetBuffer | The client sends this event to inform the server |
| Length (=3) | of the buffer size (in milliseconds) that is |
| | used to buffer any data coming over a stream. |
| | This event is sent before the server starts |
| | processing the stream. The first 4 bytes of the |
| | event data represent the stream ID and the next |
| | 4 bytes represent the buffer length, in |
| | milliseconds. |
+---------------+--------------------------------------------------+
| StreamIs | The server sends this event to notify the client |
| Recorded (=4) | that the stream is a recorded stream. The |
| | 4 bytes event data represent the stream ID of |
| | the recorded stream. |
+---------------+--------------------------------------------------+
| PingRequest | The server sends this event to test whether the |
| (=6) | client is reachable. Event data is a 4-byte |
| | timestamp, representing the local server time |
| | when the server dispatched the command. The |
| | client responds with PingResponse on receiving |
| | MsgPingRequest. |
+---------------+--------------------------------------------------+
| PingResponse | The client sends this event to the server in |
| (=7) | response to the ping request. The event data is |
| | a 4-byte timestamp, which was received with the |
| | PingRequest request. |
+---------------+--------------------------------------------------+
关于 RTMP Command Messages: command message detail:
类型:type of command:
类型1 NetConnection Commands
connect
The client sends the connect command to the server to request connection to a server application instance.
The command structure from the client to the server is as follows:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23+--------------+ +-------------+
| Client | | | Server |
+------+-------+ | +------+------+
| Handshaking done |
| | |
| | |
| | |
| | |
|----------- Command Message(connect) ------->|
| |
|<------- Window Acknowledgement Size --------|
| |
|<----------- Set Peer Bandwidth -------------|
| |
|-------- Window Acknowledgement Size ------->|
| |
|<------ User Control Message(StreamBegin) ---|
| |
|<------------ Command Message ---------------|
| (_result- connect response) |
| |
Message flow in the connect commandcall: The call method of the NetConnection object runs remote procedure calls (RPC) at the receiving end. The called RPC name is passed as a parameter to the call command.
req: format…
rsp: format…close
createStream
类型2 NetStream Commands
NetStream定义了流媒体音频,视频和数据消息的通道可以通过将客户端连接到服务器的NetConnection流过。 NetConnection对象可以支持多个数据流的多个NetStream。
The following commands can be sent on the NetStream by the client to the server:
- play
- play2
- deleteStream
- closeStream
- receiveAudio
- receiveVideo
- publish
- seek
- pause
The server sends NetStream status updates to the client using the “onStatus” command:服务器使用“onStatus”命令向客户端发送 NetStream 状态更新:
1 | +--------------+----------+----------------------------------------+ |
客户端向服务端请求拉流: play
client发送这个指令要求server play a stream. A playlist can also be created using this command multiple times.
若你想创建一个动态playlist 能再不同的直播或已记录流中切换。调用play多次和传递false去重置,每次。 相反,如果您想立即播放指定的流,清除排队等待播放的任何其他流,请传递 true 以进行重置。
The command structure from the client to the server is as follows:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109+--------------+----------+-----------------------------------------+
| Field Name | Type | Description |
+--------------+----------+-----------------------------------------+
| Command Name | String | Name of the command. Set to "play". |
+--------------+----------+-----------------------------------------+
| Transaction | Number | Transaction ID set to 0. |
| ID | | |
+--------------+----------+-----------------------------------------+
| Command | Null | Command information does not exist. |
| Object | | Set to null type. |
+--------------+----------+-----------------------------------------+
| Stream Name | String | Name of the stream to play. |
| | | To play video (FLV) files, specify the |
| | | name of the stream without a file |
| | | extension (for example, "sample"). To |
| | | play back MP3 or ID3 tags, you must |
| | | precede the stream name with mp3: |
| | | (for example, "mp3:sample". To play |
| | | H.264/AAC files, you must precede the |
| | | stream name with mp4: and specify the |
| | | file extension. For example, to play the|
| | | file sample.m4v,specify "mp4:sample.m4v"|
| | | |
+--------------+----------+-----------------------------------------+
| Start | Number | An optional parameter that specifies |
| | | the start time in seconds. The default |
| | | value is -2, which means the subscriber |
| | | first tries to play the live stream |
| | | specified in the Stream Name field. If a|
| | | live stream of that name is not found,it|
| | | plays the recorded stream of the same |
| | | name. If there is no recorded stream |
| | | with that name, the subscriber waits for|
| | | a new live stream with that name and |
| | | plays it when available. If you pass -1 |
| | | in the Start field, only the live stream|
| | | specified in the Stream Name field is |
| | | played. If you pass 0 or a positive |
| | | number in the Start field, a recorded |
| | | stream specified in the Stream Name |
| | | field is played beginning from the time |
| | | specified in the Start field. If no |
| | | recorded stream is found, the next item |
| | | in the playlist is played. |
+--------------+----------+-----------------------------------------+
| Duration | Number | An optional parameter that specifies the|
| | | duration of playback in seconds. The |
| | | default value is -1. The -1 value means |
| | | a live stream is played until it is no |
| | | longer available or a recorded stream is|
| | | played until it ends. If you pass 0, it |
| | | plays the single frame since the time |
| | | specified in the Start field from the |
| | | beginning of a recorded stream. It is |
| | | assumed that the value specified in |
| | | the Start field is equal to or greater |
| | | than 0. If you pass a positive number, |
| | | it plays a live stream for |
| | | the time period specified in the |
| | | Duration field. After that it becomes |
| | | available or plays a recorded stream |
| | | for the time specified in the Duration |
| | | field. (If a stream ends before the |
| | | time specified in the Duration field, |
| | | playback ends when the stream ends.) |
| | | If you pass a negative number other |
| | | than -1 in the Duration field, it |
| | | interprets the value as if it were -1. |
+--------------+----------+-----------------------------------------+
| Reset | Boolean | An optional Boolean value or number |
| | | that specifies whether to flush any |
| | | previous playlist. |
+--------------+----------+-----------------------------------------+
+-------------+ +----------+
| Play Client | | | Server |
+------+------+ | +-----+----+
| Handshaking and Application |
| connect done |
| | |
| | |
| | |
| | |
---+---- |------Command Message(createStream) ----->|
Create| | |
Stream| | |
---+---- |<---------- Command Message --------------|
| (_result- createStream response) |
| |
---+---- |------ Command Message (play) ----------->|
| | |
| |<------------ SetChunkSize --------------|
| | |
| |<---- User Control (StreamIsRecorded) ----|
Play | | |
| |<---- UserControl (StreamBegin) ----------|
| | |
| |<--Command Message(onStatus-play reset) --|
| | |
| |<--Command Message(onStatus-play start) --|
| | |
| |<-------------Audio Message---------------|
| | |
| |<-------------Video Message---------------|
| | | |
|
Keep receiving audio and video stream till finishes
Message flow in the play command流程如下:
The message flow during the execution of the command is:- The client sends the play command after receiving result of the createStream command as success from the server.
- On receiving the play command, the server sends a protocol message to set the chunk size.
- The server sends another protocol message (user control) specifying the event ’StreamIsRecorded’ and the stream ID in that message. The message carries the event type in the first 2 bytes and the stream ID in the last 4 bytes.
- The server sends another protocol message (user control) specifying the event ’StreamBegin’, to indicate beginning of the streaming to the client.
- The server sends an onStatus command messages NetStream.Play.Start & NetStream.Play.Reset if the play command sent by the client is successful. NetStream.Play.Reset is sent by the server only if the play command sent by the client has set the reset flag. If the stream to be played is not found, the Server sends the onStatus message NetStream.Play.StreamNotFound.
- After this, the server sends audio and video data, which the client plays.
play2:
不像play command.play2能切换到不同的码率流,而不用改变播放的内容的时间表。服务器为客户端可以在 play21
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57+--------------+----------+----------------------------------------+
| Field Name | Type | Description |
+--------------+----------+----------------------------------------+
| Command Name | String | Name of the command, set to "play2". |
+--------------+----------+----------------------------------------+
| Transaction | Number | Transaction ID set to 0. |
| ID | | |
+--------------+----------+----------------------------------------+
| Command | Null | Command information does not exist. |
| Object | | Set to null type. |
+--------------+----------+----------------------------------------+
| Parameters | Object | An AMF encoded object whose properties |
| | | are the public properties described |
| | | for the flash.net.NetStreamPlayOptions |
| | | ActionScript object. |
+--------------+----------+----------------------------------------+
+--------------+ +-------------+
| Play2 Client | | | Server |
+--------+-----+ | +------+------+
| Handshaking and Application |
| connect done |
| | |
| | |
| | |
| | |
---+---- |---- Command Message(createStream) --->|
Create | | |
Stream | | |
---+---- |<---- Command Message (_result) -------|
| |
---+---- |------ Command Message (play) -------->|
| | |
| |<------------ SetChunkSize ------------|
| | |
| |<--- UserControl (StreamIsRecorded)----|
Play | | |
| |<------- UserControl (StreamBegin)-----|
| | |
| |<--Command Message(onStatus-playstart)-|
| | |
| |<---------- Audio Message -------------|
| | |
| |<---------- Video Message -------------|
| | |
| |
---+---- |-------- Command Message(play2) ------>|
| | |
| |<------- Audio Message (new rate) -----|
Play2 | | |
| |<------- Video Message (new rate) -----|
| | | |
| | | |
| Keep receiving audio and video stream till finishes
|
Message flow in the play2 commanddeleteStream:
NetStream sends the deleteStream command when the NetStream object is getting destroyed.
The command structure from the client to the server is as follows:
1 | +--------------+----------+----------------------------------------+ |
The server does not send any response.
receiveAudio: NetStream发送这个command消息通知服务器是否发送音频给客户端。
- receiveVideo:和24类似
- publish: 客户端发送publish去发布一个命名流给到服务器。使用这个名字,任何client能play这个流和接收发布的音视频和data message.
- seek: client发送seek command去在媒体文件或播放列表中寻求偏移量(以毫秒为单位)。
- pause: client发送pause command告诉服务器暂停和启动playing
Message Exchange Examples
略:
总结:
rtmp节点的flv tag直接是取rtmp协议中header中的messagelen
流程: 客户端向服务端请求推流; publish
- srs_rtmp_handshake :c0c1 s0s1s2 c2 握手
- srs_rtmp_connect_app : 带各种键值对: tcurl,swfurl,… 连接
- 一些protocol control message: SetChunkSize
- 访问rtmp流: 推流端:对于推流端,客户端要向服务器发送一个releaseStream命令消息,之后是createStream命令消息,对于拉流端,则要发送play消息请求视频资源。我们来先看看createStream消息,RTMP客户端发送此消息到服务端,创建一个逻辑通道,用于消息通信。音频、视频、元数据均通过createStream创建的数据通道进行交互,而releaseStream与createStream相对应,为什么有的时候会在createStream之前先来一次releaseStream呢?这就像我们很多的服务实现中,先进行一次stop,然后再进行start一样。因为我们每次开启新的流程,并不能确保之前的流程是否正常走完,是否出现了异常情况,异常的情况是否已经处理等等,所以,做一个类似于恢复初始状态的操作,releaseStream就是这个作用。
releaseStream, FCPUBLISH , CreateStream 用于创建一个新的通道; - 等待server返回结果: 返回streamID等
- 推流,需要publish stream: 主要携带流名称;
- 推流信令的返回response
- 创建chunk 后发送媒体数据