02 - 二进制分帧层
第 02 章:二进制分帧层
HTTP/2 的基石:一切皆帧(Frame)
2.1 从文本到二进制
HTTP/1.1 是纯文本协议,请求和响应都是人类可读的 ASCII 文本。HTTP/2 彻底改变了这一点——所有通信都被组织为二进制编码的帧。
2.1.1 为什么选择二进制?
| 对比维度 | 文本协议(HTTP/1.1) | 二进制协议(HTTP/2) |
|---|---|---|
| 解析复杂度 | 需逐字符解析,处理边界 | 固定偏移量,直接读取 |
| 解析速度 | 慢(字符串解析) | 快(位运算) |
| 空间效率 | 低(文本编码冗余) | 高(紧凑二进制编码) |
| 错误检测 | 弱(需应用层处理) | 强(帧长度、类型校验) |
| 可调试性 | 高(人类可读) | 需工具辅助 |
HTTP/1.1 请求(文本):
GET /path HTTP/1.1\r\n
Host: example.com\r\n
\r\n
HTTP/2 请求(二进制帧):
┌──────────────────────────────────────────────────┐
│ Length (24 bits) │ Type (8) │ Flags (8) │ R (1) │
│ Stream ID (31 bits) │
│ Header Block Fragment (变长) │
└──────────────────────────────────────────────────┘
2.2 帧格式详解
2.2.1 帧头部(Frame Header)
每个 HTTP/2 帧都以一个 9 字节的固定头部开始:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Length (24) |
+---------------+---------------+---------------+---------------+
| Type (8) | Flags (8) |
+-+-------------+---------------+-------------------------------+
|R| Stream Identifier (31) |
+-+-------------------------------------------------------------+
| Frame Payload (变长) |
+---------------------------------------------------------------+
| 字段 | 位数 | 说明 |
|---|---|---|
| Length | 24 bits | 帧负载长度,不含帧头部 9 字节。默认最大 16384 (16KB),可通过 SETTINGS_MAX_FRAME_SIZE 调整至 16MB |
| Type | 8 bits | 帧类型(0-255),定义帧的语义 |
| Flags | 8 bits | 帧类型相关的标志位 |
| R | 1 bit | 保留位,必须为 0 |
| Stream Identifier | 31 bits | 流标识符,0 表示连接级控制帧 |
2.2.2 帧格式代码实现
import struct
from dataclasses import dataclass
from enum import IntEnum
class FrameType(IntEnum):
DATA = 0x0
HEADERS = 0x1
PRIORITY = 0x2
RST_STREAM = 0x3
SETTINGS = 0x4
PUSH_PROMISE = 0x5
PING = 0x6
GOAWAY = 0x7
WINDOW_UPDATE = 0x8
CONTINUATION = 0x9
@dataclass
class Frame:
length: int # 24 bits
type: int # 8 bits
flags: int # 8 bits
stream_id: int # 31 bits
payload: bytes
HEADER_SIZE = 9
MAX_LENGTH = 16384 # 16KB default
def encode(self) -> bytes:
"""编码帧为二进制"""
header = struct.pack(
'>I', # 32-bit unsigned int (大端序)
self.length << 8 # 前 24 位为长度
)
# 注意:前 3 字节是长度,需特殊处理
header = self.length.to_bytes(3, 'big')
header += self.type.to_bytes(1, 'big')
header += self.flags.to_bytes(1, 'big')
header += (self.stream_id & 0x7FFFFFFF).to_bytes(4, 'big')
return header + self.payload
@classmethod
def decode(cls, data: bytes) -> 'Frame':
"""从二进制解码帧"""
if len(data) < cls.HEADER_SIZE:
raise ValueError("数据不足一个帧头部")
length = int.from_bytes(data[0:3], 'big')
frame_type = data[3]
flags = data[4]
stream_id = int.from_bytes(data[5:9], 'big') & 0x7FFFFFFF
payload = data[9:9+length]
return cls(length, frame_type, flags, stream_id, payload)
# 示例:创建一个 HEADERS 帧
headers_frame = Frame(
length=10,
type=FrameType.HEADERS,
flags=0x04, # END_HEADERS
stream_id=1,
payload=b'\x82\x86\x84\x41\x0f\x77\x77\x77\x2e'
)
encoded = headers_frame.encode()
print(f"编码后帧大小: {len(encoded)} 字节")
print(f"十六进制: {encoded.hex()}")
2.3 帧类型详解
HTTP/2 定义了 10 种帧类型,分为两大类:数据帧和控制帧。
2.3.1 帧类型一览表
| 类型值 | 名称 | 用途 | 关键标志位 |
|---|---|---|---|
| 0x0 | DATA | 传输请求/响应负载 | END_STREAM (0x1), PADDED (0x8) |
| 0x1 | HEADERS | 传输头部块 | END_STREAM, END_HEADERS, PADDED, PRIORITY |
| 0x2 | PRIORITY | 设置流优先级 | 无 |
| 0x3 | RST_STREAM | 终止某个流 | 无 |
| 0x4 | SETTINGS | 连接配置参数 | ACK (0x1) |
| 0x5 | PUSH_PROMISE | 服务器推送预告 | END_HEADERS, PADDED |
| 0x6 | PING | 连接活性检测 | ACK (0x1) |
| 0x7 | GOAWAY | 优雅关闭连接 | 无 |
| 0x8 | WINDOW_UPDATE | 流量控制窗口更新 | 无 |
| 0x9 | CONTINUATION | 延续头部块 | END_HEADERS (0x4) |
2.3.2 DATA 帧
DATA 帧用于传输请求或响应的实际内容。
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Pad Length? (8)|
+---------------+-----------------------------------------------+
| Data (*) |
+---------------------------------------------------------------+
| Padding (*) |
+---------------------------------------------------------------+
标志位说明:
| 标志 | 值 | 说明 |
|---|---|---|
| END_STREAM | 0x1 | 表示这是流的最后一帧 |
| PADDED | 0x8 | 包含填充字段,用于混淆帧大小 |
def create_data_frame(stream_id: int, data: bytes, end_stream: bool = False) -> Frame:
"""创建 DATA 帧"""
flags = 0
if end_stream:
flags |= 0x01 # END_STREAM
return Frame(
length=len(data),
type=FrameType.DATA,
flags=flags,
stream_id=stream_id,
payload=data
)
# 示例:发送 JSON 响应
response_body = b'{"users": [{"id": 1, "name": "Alice"}]}'
data_frame = create_data_frame(stream_id=1, data=response_body, end_stream=True)
2.3.3 HEADERS 帧
HEADERS 帧打开一个新流并传输头部块。
package main
import (
"fmt"
"net/http"
)
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// HTTP/2 中,每个请求对应一个流
// HEADERS 帧携带以下头部:
// - :method (GET/POST/...)
// - :path (/path)
// - :scheme (https)
// - :authority (host)
// + 常规头部 (Content-Type, Accept, ...)
fmt.Fprintf(w, "Stream ID 可在服务端日志中查看\n")
fmt.Fprintf(w, "Protocol: %s\n", r.Proto)
// 查看接收到的头部
for name, values := range r.Header {
for _, v := range values {
fmt.Fprintf(w, "%s: %s\n", name, v)
}
}
})
http.ListenAndServeTLS(":8443", "cert.pem", "key.pem", nil)
}
2.3.4 SETTINGS 帧
SETTINGS 帧用于协商连接级参数。
| 设置项 | 标识符 | 默认值 | 说明 |
|---|---|---|---|
| SETTINGS_HEADER_TABLE_SIZE | 0x1 | 4096 | HPACK 动态表最大字节数 |
| SETTINGS_ENABLE_PUSH | 0x2 | 1 | 是否启用服务器推送 |
| SETTINGS_MAX_CONCURRENT_STREAMS | 0x3 | 无限 | 最大并发流数 |
| SETTINGS_INITIAL_WINDOW_SIZE | 0x4 | 65535 | 流级初始窗口大小(字节) |
| SETTINGS_MAX_FRAME_SIZE | 0x5 | 16384 | 最大帧负载大小(字节) |
| SETTINGS_MAX_HEADER_LIST_SIZE | 0x6 | 无限 | 头部列表最大字节数 |
def create_settings_frame(settings: dict) -> Frame:
"""创建 SETTINGS 帧"""
payload = bytearray()
for identifier, value in settings.items():
payload += identifier.to_bytes(2, 'big')
payload += value.to_bytes(4, 'big')
return Frame(
length=len(payload),
type=FrameType.SETTINGS,
flags=0,
stream_id=0, # SETTINGS 帧的 stream_id 必须为 0
payload=bytes(payload)
)
# 发送初始 SETTINGS
initial_settings = {
0x1: 4096, # HEADER_TABLE_SIZE
0x2: 1, # ENABLE_PUSH
0x3: 100, # MAX_CONCURRENT_STREAMS
0x4: 65535, # INITIAL_WINDOW_SIZE
0x5: 16384, # MAX_FRAME_SIZE
}
settings_frame = create_settings_frame(initial_settings)
2.3.5 RST_STREAM 帧
RST_STREAM 帧用于立即终止某个流,不影响连接上的其他流。
| 错误码 | 名称 | 说明 |
|---|---|---|
| 0x0 | NO_ERROR | 正常关闭 |
| 0x1 | PROTOCOL_ERROR | 协议错误 |
| 0x2 | INTERNAL_ERROR | 内部错误 |
| 0x3 | FLOW_CONTROL_ERROR | 流控错误 |
| 0x4 | SETTINGS_TIMEOUT | SETTINGS 超时 |
| 0x5 | STREAM_CLOSED | 流已关闭 |
| 0x6 | FRAME_SIZE_ERROR | 帧大小错误 |
| 0x7 | REFUSED_STREAM | 拒绝流 |
| 0x8 | CANCEL | 取消 |
| 0x9 | COMPRESSION_ERROR | 压缩错误 |
| 0xa | CONNECT_ERROR | 连接错误 |
| 0xb | ENHANCE_YOUR_CALM | 增强限流 |
| 0xc | INADEQUATE_SECURITY | 安全性不足 |
| 0xd | HTTP_1_1_REQUIRED | 需要 HTTP/1.1 |
// Go 中处理 RST_STREAM
package main
import (
"log"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
// 客户端取消请求时(如用户关闭页面),
// 服务器会收到 RST_STREAM
select {
case <-r.Context().Done():
log.Println("客户端取消了请求,收到 RST_STREAM")
return
default:
// 正常处理
w.Write([]byte("Hello, HTTP/2!"))
}
}
2.3.6 GOAWAY 帧
GOAWAY 帧用于优雅关闭连接,告知对端停止创建新流。
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R| Last-Stream-ID (31) |
+-+-------------------------------------------------------------+
| Error Code (32) |
+---------------------------------------------------------------+
| Additional Debug Data (*) |
+---------------------------------------------------------------+
// 服务端优雅关闭示例
package main
import (
"context"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
)
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello"))
})
server := &http.Server{
Addr: ":8443",
Handler: mux,
}
// 监听系统信号
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-quit
log.Println("收到关闭信号,发送 GOAWAY 帧...")
// 优雅关闭:发送 GOAWAY,等待现有请求完成
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := server.Shutdown(ctx); err != nil {
log.Printf("关闭错误: %v", err)
}
}()
log.Println("服务器启动 :8443")
if err := server.ListenAndServeTLS("cert.pem", "key.pem"); err != http.ErrServerClosed {
log.Fatal(err)
}
log.Println("服务器已关闭")
}
2.4 流标识符(Stream Identifier)
2.4.1 流 ID 分配规则
| 规则 | 说明 |
|---|---|
| 客户端发起 | 奇数(1, 3, 5, …) |
| 服务器发起 | 偶数(2, 4, 6, …) |
| 连接控制 | 0(用于 SETTINGS, GOAWAY, WINDOW_UPDATE 等连接级帧) |
| ID 单调递增 | 新流的 ID 必须大于该端已使用的所有 ID |
| 最大值 | 2^31 - 1(约 21 亿),耗尽后需建立新连接 |
2.4.2 流状态机
send PP / recv PP
┌─────────────────┐
│ │
┌────────────▼─────┐ │
│ │ │
idle │ ┌─────────────▼──┐ │
│ │ │ │
└───►│ open │◄───────┘
│ │
┌────┴──────┬─────────┘
│ │
send RST / recv RST /
recv RST send RST
│ │
┌────▼──┐ ┌────▼──┐
│closed │ │closed │
└───────┘ └───────┘
from enum import Enum
class StreamState(Enum):
IDLE = "idle"
OPEN = "open"
RESERVED_LOCAL = "reserved (local)"
RESERVED_REMOTE = "reserved (remote)"
HALF_CLOSED_LOCAL = "half-closed (local)"
HALF_CLOSED_REMOTE = "half-closed (remote)"
CLOSED = "closed"
class Stream:
def __init__(self, stream_id: int):
self.stream_id = stream_id
self.state = StreamState.IDLE
self.headers = {}
self.data = bytearray()
self.window_size = 65535
def on_send_headers(self, end_stream: bool = False):
if self.state == StreamState.IDLE:
self.state = StreamState.OPEN
if end_stream:
self.state = StreamState.HALF_CLOSED_LOCAL
def on_recv_headers(self, end_stream: bool = False):
if self.state == StreamState.IDLE:
self.state = StreamState.OPEN
if end_stream:
self.state = StreamState.HALF_CLOSED_REMOTE
def on_send_end_stream(self):
if self.state == StreamState.OPEN:
self.state = StreamState.HALF_CLOSED_LOCAL
def on_recv_end_stream(self):
if self.state == StreamState.OPEN:
self.state = StreamState.HALF_CLOSED_REMOTE
def on_rst_stream(self):
self.state = StreamState.CLOSED
2.5 帧组装实战
2.5.1 完整的 HTTP/2 请求帧序列
客户端发起 GET /api/users 请求:
帧序列:
1. HEADERS 帧 (stream_id=1, END_HEADERS)
- :method = GET
- :path = /api/users
- :scheme = https
- :authority = api.example.com
- accept = application/json
2. DATA 帧 (stream_id=1, END_STREAM)
- 空负载(GET 请求通常无 body)
服务端响应:
帧序列:
1. HEADERS 帧 (stream_id=1, END_HEADERS)
- :status = 200
- content-type = application/json
2. DATA 帧 (stream_id=1, END_STREAM)
- {"users": [...]}
2.5.2 使用 h2spec 进行合规测试
# 安装 h2spec
# https://github.com/summerwind/h2spec/releases
# 测试 HTTP/2 服务器的协议合规性
h2spec -t -s -p 8443 localhost
# 输出示例:
# 4. Frame Format
# ✓ 4.1. Frame Format
# ✓ Sends a frame with unknown type
# ✓ Sends a frame with invalid length
# ✓ 4.2. Frame Size
# ✓ Sends a DATA frame with 2^14 octets
# ✓ Sends a DATA frame with 2^24-1 octets
2.6 业务场景:API 网关的帧级监控
// 自定义 HTTP/2 Transport,监控帧级别信息
package main
import (
"crypto/tls"
"fmt"
"net/http"
"time"
)
func main() {
transport := &http.Transport{
TLSClientConfig: &tls.Config{
// 启用 HTTP/2
},
MaxIdleConns: 100,
MaxIdleConnsPerHost: 10,
IdleConnTimeout: 90 * time.Second,
}
client := &http.Client{
Transport: transport,
Timeout: 30 * time.Second,
}
// 发送多个并发请求(验证多路复用)
urls := []string{
"https://api.example.com/users",
"https://api.example.com/orders",
"https://api.example.com/products",
}
for _, url := range urls {
go func(u string) {
start := time.Now()
resp, err := client.Get(u)
if err != nil {
fmt.Printf("请求失败 %s: %v\n", u, err)
return
}
defer resp.Body.Close()
fmt.Printf("[%s] %s - %d (%v)\n",
resp.Proto, u, resp.StatusCode, time.Since(start))
}(url)
}
time.Sleep(5 * time.Second)
}
2.7 注意事项
⚠️ 帧大小限制:
- 默认最大帧负载 16KB,可通过 SETTINGS_MAX_FRAME_SIZE 调整至 16MB
- 接收超过限制的帧应返回 FRAME_SIZE_ERROR
- HEADERS 帧过大时需使用 CONTINUATION 帧
⚠️ CONTINUATION 帧:
- 当头部块超过一个帧的容量时,使用 CONTINUATION 帧
- CONTINUATION 帧必须紧跟在 HEADERS 或 PUSH_PROMISE 之后
- 未收到 END_HEADERS 标志前,不得发送其他类型的帧
⚠️ 保留位处理:
- 帧头部的 R 位(第 5 字节最高位)必须为 0
- 接收到 R=1 的帧应视为 PROTOCOL_ERROR
2.8 扩展阅读
- 📖 RFC 7540 Section 4 - Frame Format
- 📖 RFC 7540 Section 6 - Frame Definitions
- 📖 h2spec - HTTP/2 Conformance Testing
- 📖 Wireshark HTTP/2 协议分析