中国的防火长城是如何检测和封锁完全加密流量的

摘要

全加密协议是翻墙生态系统中的一块基石。这类协议对数据包有效载荷的每一个字节都进行了加密，以期让流量"看起来什么都不像"。2021年11月初，中国的防火长城（GFW）部署了一种新的审查技术，这种技术可以实时地被动检测并阻断全加密流量。GFW这一新添的审查能力影响到一大批流行的翻墙协议，包括但不限于Shadowsocks、VMess和Obfs4。虽然中国长期以来一直采用主动探测来识别此类协议，但这次是第一次有关于采用纯被动检测来识别全加密流量的报告。面对这一新的现象，反审查社区不禁要问检测是如何做到的。

在这篇论文中，我们测量并描述了GFW用于审查完全加密流量的新系统。我们发现，审查者并没有直接定义什么是完全加密流量，而是应用粗糙但高效的启发式规则来豁免那些不太可能是完全加密的流量；然后它阻止其余未被豁免的流量。这些启发式规则基于常见协议的指纹、1比特的占比以及可打印的ASCII字符的数量、比例和位置。我们对互联网进行扫描，并揭示了GFW都检查哪些流量和哪些IP地址。我们在一个大学网络的实时流量上模拟我们推断出的GFW的检测算法，以评估其全面性和误报率。结果表明，我们推断出的检测规则很好地覆盖了GFW实际使用的检测规则。 我们估计，如果这一检测算法被广泛地应用，它将有可能误伤大约0.6%的非翻墙互联网流量。

我们对GFW的新审查机制的理解帮助我们得出了几个实用的规避封锁的策略。我们负责任地将我们的发现和建议透露给不同的反审查工具的开发者，从而帮助数以百万计的用户成功绕开了这种新的封锁。

1 引言

完全加密的翻墙协议是翻墙生态系统中的一块基石。不同于像TLS这样的协议以明文握手开始，完全加密（随机化）的协议--如VMess [23]、Shadowsocks [22]和Obfs4 [7]--被设计成连接中的每个字节都与随机数据没有区别。这些 "看起来什么都不像 "的协议的设计理念是，它们应该很难被审查者抓住特征，因此阻断的成本很高。

2021年11月6日，中国的互联网用户报告说他们的Shadowsocks和VMess服务器被封锁了 [10]。 11月8日，一个Outline [42]开发者报告说来自中国的使用量突然下降 [69]。这次封锁的开始时间恰逢2021年11月8日至11日召开的中国共产党第十九届中央委员会第六次全体会议 [4，1]。能够封锁这些翻墙工具代表中国的防火长城（GFW）具备了一种全新的新能力。据我们所知，虽然中国自2019年5月以来一直在采用被动流量分析和主动探测相结合的方式来识别Shadowsocks服务器 [5]，但这是审查者第一次能够仅基于被动流量分析，就实时地大规模封锁完全加密的代理。全加密协议对整个反审查生态系统的重要性以及GFW的未知行为，促使我们去探索和了解检测和封锁的机制原理。

在这项工作中，我们对GFW被动检测和封锁全加密流量的新系统进行了测量和描述。我们发现，审查者没有直接定义什么是完全加密的流量，而是应用了至少五条粗糙但高效的启发式规则来豁免那些不太可能是完全加密的流量；然后，它阻止其余未豁免的流量。这些豁免规则基于常见的协议指纹、基于1比特比例的粗略熵测试、以及第一个TCP数据包的有效载荷中可打印的ASCII字符的比例、位置和最大连续数。

由于GFW的黑箱性质，我们推断出的规则可能并不详尽；但是我们使用科罗拉多大学博尔德分校的实时网络流量，对我们推断出的检测规则进行了评估。有证据表明我们推断出的规则与GFW实际使用的规则间有非常大的重叠。我们还发现，推断出的检测算法会阻断大学网络中所有连接的大约0.6%。 可能是为了减轻假阳性引起的封锁误伤，我们的互联网扫描显示，GFW策略性地只监控26%的连接，而且只监控发往流行数据中心的特定IP范围的连接。

我们还分析了这种新形式的被动封锁与GFW广为人知的主动探测系统 [5]之间的关系。二者是独立运行的。我们发现，主动探测系统也依赖于这种流量分析算法，并额外应用了一条基于数据包长度的豁免规则。因此，能够逃避这种新的封锁的规避策略，也可以帮助防止GFW识别并随后主动探测代理服务器。

我们从对这种新审查技术的理解中得出了各种规避策略。我们负责任且及时地与各种流行的反审查工具的开发者分享了我们的发现和规避建议，包括Shadowsocks [22], V2Ray [59], Outline [42], Lantern [20], Psiphon [21], 和Conjure [33]的开发者。自2022年1月以来，这些规避策略被广泛采用和部署，已帮助数百万用户绕过这一新的封锁技术。据反馈，截至2023年2月，这些工具采用的所有规避策略在中国仍然有效。

2 背景

2.1 流量混淆策略

Tschantz等人将混淆翻墙流量的方法分为两类：隐写(steganography)和多态(polymorphism) [57, § V]。隐写代理的目标是使翻墙流量看起来像应该被允许的流量；多态性的目标是使翻墙流量看起来不像应该被禁止的流量。

实现隐写术的两种最常见的方法是模仿(mimicking)和隧道传输(tunneling)。Houmansadr等人 [39]得出结论，模仿类协议有着根本性的缺陷，并指出将原始流量通过被允许的协议进行隧道传输是一种更抗封锁的方法。Frolov和Wustrow二人 [35]证明，即使使用隧道传输，翻墙软件的设计者仍然需要额外的努力让翻墙协议的指纹与流行的实现方式的指纹保持完全一致，以避免受到基于协议指纹的封锁。例如，在2012年，中国和埃塞俄比亚部署了深度包检测系统，通过Tor使用的不常见的密码套件来检测Tor流量 [44,67,55]。审查设备供应商之前已经根据meek [29]发出的TLS指纹和SNI值来识别并封锁它了 [28]。

为了避免这种复杂性，许多流行的翻墙软件选择了多态的设计。实现多态性的一个常见方法是，从连接中的第一个数据包开始，就对其有效载荷进行完全加密。由于没有任何明文或固定的包头结构指纹，审查者没办法简单地使用正则表达式或通过寻找流量中的特定模式来识别代理流量。这种设计在2009年首次被引入Obfuscated OpenSSH [16]。此后，Obfsproxy [24]、Shadowsocks [22]、Outline [42]、VMess [23]、ScrambleSuit [68]、Obfs4 [7]都采用了这种设计。Geph4 [58]、Lantern [20]、Psiphon3 [21]和Conjure [33]也部分采用这种设计。

完全加密的流量经常被称为“看起来什么都不像”的流量，又或者被误解为“没有特征”；然而，更准确的描述应该是"看起来像随机数据"。事实上，这种流量确实有一个使其与其他流量不同的重要特点：完全加密的流量与随机流量是无法区分的。由于没有可识别的头，整个连接中的流量都是均匀且高熵的，甚至在第一个数据包中就已经如此。相比之下，即使像TLS这样的加密协议也还有相对低熵的握手包，用以传达支持的版本和扩展。

2015年，Wang等人 [61，§5.1]利用连接中第一个数据包有效载荷的长度和高香农熵的特点来识别随机流量，比如Obfs4。同样，在2017年，Zhixin Wang发布了一个概念验证工具，其使用连接中前三个数据包有效载荷的高香农熵来识别Shadowsocks流量 [40]。Madeye扩展了该工具，额外使用有效载荷长度分布来检测ShadowsocksR流量 [47]。He等人 [70，§IV.A]和Liang等人 [46，§II.A]使用单比特频率检测算法，而不是香农熵，来衡量Obfs4流量的随机性。2019年，Alice等人发现，GFW使用每个连接中第一个数据包的长度和熵来怀疑Shadowsocks流量 [5]。

2.2 主动探测攻击及其防御措施

在主动探测攻击中，审查者向被怀疑的服务器发送精心制作的有效载荷，并测量它的反应。如果服务器以与众不同的方式回应这些探测（例如让审查者将其作为代理使用），审查者就可以识别并封锁它。 早在2011年8月，人们观察到GFW向接受过来自中国的SSH登录的外国SSH服务器发送看似随机的有效载荷 [49]。2012年，GFW首先寻找一个独特的TLS密码来怀疑Tor流量；然后向可疑的服务器发送主动探测，以确认其猜测 [67,66,64]。2015年，Ensafi等人对GFW针对各种协议的主动探测攻击进行了详细分析 [27]。自2019年5月起，中国部署了一个审查系统，分两步检测和封锁Shadowsocks服务器：它首先使用每个连接中第一个数据包有效载荷的长度和熵来被动地识别可能的Shadowsocks流量，然后在分阶段地向可疑的服务器发送各种探针，以确认其猜测 [5]。作为回应，研究人员提出了各种针对主动探测攻击的防御措施，包括让服务器对各种连接的反应保持一致 [34，9]和应用前置 [45，36]。 Shadowsocks、Outline和V2Ray都采用了防主动探测的设计 [34,5,19,43,32,71]，使得它们自2020年9月以来在中国就没再被封锁过 [5]，直到最近在2021年11月被再次封锁 [10]。

3 方法

我们在中国境内外的主机之间制作并发送各种测试探针，让它们被GFW观察到。我们在两个端点主机上抓包并比较流量，来观察GFW的反应。这种记录使我们能够识别任何被丢弃或被操纵的数据包，包括主动探测。

实验时间线和实验节点。 我们在表1中总结了所有主要实验的时间线和所使用的实验节点。我们总共使用了腾讯云北京（AS45090）的10台VPS和阿里云北京（AS37963）的1台VPS。 我们没有观察到中国境内的实验节点或任何受影响的国外节点之间的审查行为有任何差异。我们使用了Digital Ocean旧金山（AS14061）的四台VPS：其中三台受到了新审查机制的影响，剩下一台则没有受到影响。我们把这四台VPS变成了水槽服务器(sink server)；也就是说，服务器监听1到65535的所有端口，接受TCP连接，但不向客户端发送任何有效载荷。我们还采用了科罗拉多大学博尔德分校（AS104）中的两台机器进行互联网扫描和实时流量分析。我们根据IP2Location数据库 [3]检查了我们的VPS的IP地址，并确认它们的地理位置与供应商所报告的位置相符。

触发审查制度。因为外界观察者是无法区分完全加密的流量与随机数据的，所以除了使用实际的翻墙工具外，我们还开发了测量工具用来在我们的研究中发送随机数据以触发封锁。这些工具完成一个TCP握手后，会发送一个给定长度的随机有效载荷，然后关闭连接。

使用残留审查(residual censorship)来确认封锁。 与GFW封锁许多其他协议的方式类似 [13,17,63,14]，在一个连接触发审查后，GFW会阻止所有具有相同三元组（客户端IP、服务器IP、服务器端口）的后续连接180秒。这种残留审查允许我们通过从同一客户端发送后续连接到服务器的同一端口来确认封锁。我们逐一进行共计五次的TCP连接，中间有一秒钟的时间间隔。如果五次连接都失败了，我们就得出结论，这个三元组被封锁了。如果一个三元组被封锁，我们在接下来的180秒内不会再使用它进行进一步的测试。

对重复测试的概率阻断进行计算。 我们经常要用相同的有效载荷进行多次连接，才能观察到封锁。在第6.3节中，我们解释了这是因为GFW采用了一种概率阻断策略，大约只有四分之一的概率触发审查。为了减少这种概率行为造成的测量误差，我们在对任何一次阻断（或不阻断）的观察下判断之前，都要发送最多25次有着相同有效载荷的连接。如果我们能够连续成功地用相同的有效载荷进行25次连接，那么我们就得出结论，该有效载荷（或服务器）不受这种新审查的影响。如果在至少发送一次有效载荷后，随后的5次连接尝试都（由于残留审查而）超时了，那么我们就将有效载荷（和服务器）标记为受到了新的审查的影响。在整个研究过程中，我们对所有有效载荷的测试，都采用了这种重复连接的方法。

4 测量并认识新审查技术的特点

我们进行实验以了解GFW如何检测和阻止完全加密的连接。详见表1，在2021年11月6日至2022年5月18日期间，我们使用了三台中国的VPS和三台美国的水槽服务器来进行实验。在同一时期，我们还使用了一台阿里云北京VPS（自治系统号：AS37963）来重复我们所有的实验。我们没有观察到中国境内的实验节点与任何受影响此次审查影响的境外节点之间的审查行为有任何差异。2023年2月16日，我们重新进行了实验，确认所有检测规则仍然有效。这一次，我们使用了一台腾讯云北京的VPS和一台位于Digital Ocean旧金山的水槽服务器。

算法1展示并概述了我们通过测量实验推断出的GFW检测规则。图1则举出了每个推断规则被应用时的例子。虽然我们无法推断这些检测规则被应用的顺序，也无法推断这些规则是否详尽，但我们的实验证实了GFW审查策略的具体组成部分。我们发现，审查者没有直接定义什么是完全加密的流量，而是应用了至少五条粗糙但高效的启发式规则来豁免那些不太可能是完全加密的流量；然后它阻止其余任何未被豁免的流量。这些豁免规则是基于常见的协议指纹、使用1比特比例的粗略熵测试、以及可打印的ASCII字符的占比、位置和最大连续数。

4.1 基于熵的豁免规则（Ex1）

我们观察到，为1的比特数影响了一个连接是否被阻断。为了确定这一点，我们向服务器重复发送连接，并观察哪些连接被阻止。在每个连接中，我们发送256个不同的字节模式中的一个，由1个字节重复100次组成（例如，\x00\x00\x00...，\x01\x01\x01...，...，\xff\xff\xff...）。 我们对每个模式都发送25次包含这一模式的连接到我们的服务器，并观察是否有任何模式导致后续连接被阻断。如果有某个连接被阻断，则表明它的有效载荷触发了审查。我们发现共有40个字节的模式触发了封锁，而其余216个模式没有。 被封锁的模式例子包括\x0f\x0f...，\x17\x17\x17...，和\x1b\x1b\x1b...（以及其他37个）。

所有被阻断的模式的每个字节的八位比特中都恰好有4位是\(1\)比特（例如，二进制的\x1b是00011011）。 我们猜想每个字节的\(1\)比特的数量可能起作用，因为均匀的随机数据将有接近相同数量的二进制的\(1\)比特和\(0\)比特。实质上，这是在测量客户端数据包内的比特的熵。

我们发送这样的字节组合来确认这一猜想：组合中的每种字节单独发送都被允许，但组合起来发送就会被禁止。例如，\xfe\xfe\xfe...和\x01\x01\x01...都没有被单独封锁，但这些字节作为\xfe\x01\xfe\x01...一起发送却被阻止。 我们注意到\xfe\x01的16位比特中有8位被设置为\(1\)（平均每个字节设置4个比特），而\xfe的8位比特中有7位被设置为\(1\)，\x01的8为比特中有1位被设置为\(1\)。这解释了为什么它们单独发送被允许，但组合起来发送就被阻止。

当然，随机或加密的数据不会总是正好有一半的比特被设置为\(1\)。我们通过发送一串50个随机字节（400比特），并设置了越来越多的比特为\(1\)的实验，来测试GFW需要多接近一半的比特才能阻断。 我们产生了401个比特串，其中有0-400个比特被设置为 \(1\)，并对每个字符串的比特位置进行洗牌，以产生一组随机字符串，每个字节设置0-8比特（以0.02比特/字节为增量）。对于每个字符串，我们发送25次连接含有它的连接，以观察它是否会引发后续连接的封锁。我们发现，所有具有3.4比特/字节的字符串都没有被封锁，而3.4至4.6比特/字节的字符串则被封锁了。

这其中有一个例外，那就是有一个4.26比特/字节集的字符串也没有被封锁。这是因为它有超过50%的字节是可打印的ASCII字符；我们接下来会介绍这是另一条豁免规则（Ex2）。我们重复了我们的实验，并确认其他具有相同\(1\)比特数但可打印的ASCII字符较少的字符串，确实被阻止了。

综上所述，我们发现，如果一个连接中，客户端的第一个数据包中\(1\)比特比例偏离一半，GFW就会豁免这个连接。这相当于对熵的粗略测量：随机（加密）数据z总有接近一半的比特被设置为\(1\)，而其他协议由于明文或有零填充的协议头，每字节的\(1\)比特数通常较少。例如，谷歌浏览器105版发送的TLS Client Hello包，由于用零填充，每字节平均只有1.56个\(1\)比特，属于豁免范围。

4.2 基于可打印的ASCII字符的豁免规则(Ex2-4)

我们观察到在第4.1节中发现的比特计数规则有几个例外。例如，模式\x4b\x4b\x4b...没有被封锁，尽管每个字节正好设置了4位。事实上，实际上有70个字节（8选4）正好有4位为\(1\)比特，但是我们的分析发现，其中只有40个触发了审查。那另外30个呢？

这另外30个字节的值都属于可打印的ASCII字符的字节范围，即0x20-0x7e。 我们推测，GFW豁免这些字符可能是为了允许"明文"（人类可读）协议。

我们发现，GFW有三种关于可打印的ASCII字符的豁免方式，都是基于连接中客户端发送的第一个数据包的有效载荷：如果前六个字节是可打印的（Ex2）；如果超过一半的字节是可打印的（Ex3）；或者如果它包含超过20个连续的可打印字节（Ex4），则允许连接。

前六个字节是可打印的（Ex2）。 我们观察到，如果一个连接的前6个字节在可打印的字节范围0x20-0x7e内，那么GFW就会豁免该连接。如果前6个字节中有超出这个范围的字符，那么连接就可能会被阻止，前提是它没有符合其他豁免的规则（例如，每个字节集有少于3.4位的\(1\)比特）。 我们通过生成不同有效载荷进行测试，其中前\(n\)字节来自不同的字符集（如可打印的ASCII字符），而消息的其余分部将是随机的不可打印字符。 我们观察到，对于\(n \lt 6\)，连接被阻断，但对于\(n\ge 6\)，即前\(n\)字节都是可打印的ASCII字符时，没有发生阻断。

第一个数据包有一半的有效载荷是可打印的（Ex3）。 如果第一个数据包的有效载荷中超过一半的字节属于可打印的ASCII范围0x20-0x7e，那么GFW就会豁免该连接。 我们通过构造并发送这样的有效载荷来测试：其前10字节由可打印ASCII范围以外的字符组成（例如0xe8），然后是一个6个字节的重复序列：5个在这个可打印范围内（如0x4b），而最后一个在可打印范围外。我们重复这个6字节的序列5次，然后在字符串的末尾用可打印范围外的\(n\)个字节来填充(用Python符号表示："\xe8"*10 + ("\x4b"*5 + "\xe8")*5 + "\xe8"*n))。 这个实验给我们一个可变长度的模式，随着我们增加\(n\)，可打印的ASCII范围内的字节的比例减少了。 我们发现，对于 \(n < 10\)，连接不会被阻断，而对于 \(n \ge 10\)，连接会被阻断。 这相当于当可打印字符的比例小于或等于一半时被阻断，而当大于一半时不被阻断。

我们设计这样的有效载荷是为了避免触发其他GFW豁免规则，例如\(1\)比特比例（Ex1）、可打印前缀（Ex2）或连续的可打印字符（Ex4）。 例如，我们分别使用 0x4b和0xe8 作为可打印和不可打印的字符，因为它们都正好有4位的设置。 这可以防止GFW因为前面讨论过的\(1\)比特比例规则（Ex1）而豁免封锁我们的连接的情况。 此外，我们避免让可打印字符0x4b连续出现，因为我们观察到这样的模式也能豁免封锁连接，这一点我们接下来会讨论。 我们用其他同样符合这些限制条件的模式（如0x8d和0x2e）重复了我们的实验，并观察到相同的结果。

超过20个连续的字节是可打印的（Ex4）。 一个可打印字符的连续出现也可以免除封锁，即使可打印字符的总比例不到一半。 为了测试这一点，我们发送了一个由可打印范围以外的字符（0xe8）组成的100个字节的模式，以及来自可打印范围的不同数量的连续字节（我们使用0x4b）。 我们的有效载荷从10个字节的0xe8开始，接着是\(n\)字节的0x4b，然后是\(90-n\)字节的0xe8，总长度为100个字节。 我们让变量\(n\)在0-90之间变化，并把每个相同的载荷都向我们的服务器发送25次。 我们发现，对于 \(n\le 20\)，连接被阻断了。当 \(n > 20\)，连接没有被阻断。 这证明当有连续的可打印的ASCII字符出现时，连接会被豁免。 当然，当 \(n > 50\)，连接也会被豁免，因为豁免规则Ex3。

其他编码方式。 我们测试了如果第一个数据包中包含中文字符，是否也可以与可打印的ASCII字符一样，让连接免于阻断。 我们使用了以UTF-8编码的6-36个中文字符串，以及GBK（与我们使用的GB2312字符相同）。 所有这些测试连接都被阻断了，这表明不存在基于汉字的豁免规则。 这可能是因这些编码中出现的汉字的情况很少，或者是因为如果要解析这些编码，会对审查系统增加不合理的复杂性，因为很难知道一个编码字符串的开始或结束位置。

4.3 基于常见协议的豁免规则（Ex5）

为了避免误伤流行的协议，我们观察到GFW明确地豁免了两种流行的协议。 GFW似乎是用客户端数据包的前3-6个字节来推断协议：如果它们与已知协议的字节相匹配，连接就会被免除阻断，即使数据包的其余部分不符合该协议。我们测试了六种常见的协议，发现TLS和HTTP协议被明确地豁免了。这个豁免列表可能并不详尽，因为可能还有其他我们没有测试的豁免协议。

TLS。 TLS连接以TLS ClientHello消息开始，该消息的前三个字节会使GFW豁免连接。我们观察到，GFW豁免了任何前三个字节与以下正则表达式匹配的连接：

[\x16-\x17]\x03[\x00-\x09]

这对应于一个字节的记录类型(record type)，后面是一个两字节的版本(version)。 我们列举了所有256个XX\x03\x03的模式，并在后面加上97个字节的随机数据。我们发现除了那些以0x16（对应TLS中的Handshake包，用于ClientHello）或0x17 （对应TLS中的应用数据类包(Application Data)）开始的模式外，其他所有模式都被封锁。 虽然通常的TLS连接不会以应用数据开头 [53,52]， 但当TLS被用于多路径TCP（MPTCP） [31]时， 常见的情况是，其中一个TCP子流被用于ClientHello，而其他子流在TCP连接建立后立即发送应用数据 [15]。 到目前为止，只有TLS的0x03[0x00-0x03] 版本被定义 [53,52]，但GFW甚至允许更晚的（尚未定义）版本。

HTTP。 审查者用来识别HTTP流量的字节模式很简单，就是在HTTP请求方法的后面跟有一个空格。如果一个信息以GET␣、PUT␣、POST␣或HEAD␣开头，那么这个连接就会被免于阻断。每个请求方法的后面的空格字符（0x20）是让连接免于屏蔽的必要条件。如果不包括这个空格字符，或用任何其他字节代替它，就不能豁免连接。其他的HTTP请求方式（OPTIONS␣, DELETE␣, CONNECT␣, TRACE␣, PATCH␣）均因为前6个字节是可打印字符，而已经满足可打印的ASCII豁免规则（Ex2）。我们发现HTTP请求方法是不区分大小写的：GeT␣、get␣和类似的变体都可以使连接被豁免。请求方式的错误拼写（例如，TEG␣）不属于豁免范围。

不被豁免的协议。 我们测试了其他常见的协议：SSH、SMTP和FTP将被豁免，因为它们都以至少6个字节的可打印的ASCII开头（规则Ex2）。DNS-over-TCP由于包含很大一部分的零，使得它被Ex1规则豁免。然而，如果在DNS-over-TCP消息后附加足够多的随机数据，它将被阻止。

上面观察到的现象让大家提出了一个问题：为什么审查者使用明确的规则来豁免TLS和HTTP，而不是其他协议。 毕竟，审查者不需要明确地豁免这两种协议：HTTP通常会都满足前6个字节为可打印的ASCII的豁免规则（Ex2），而TLS ClientHello包由于有许多零字段，其也会因位数熵相对较低而满足Ex1豁免规则。也许这是因为审查者可以采用这些简单而高效的规则来快速地豁免大部分的网络流量（TLS和HTTP），而不需要进行如计算载荷中\(1\)比特的比例、可打印的ASCII的比例等更深入的分析。

4.4 GFW是如何阻断连接的

一旦GFW使用算法1检测到完全加密的流量，就会按照下面介绍的方式阻断后续流量。

丢弃从客户端到服务器的数据包。 我们先触发GFW的阻断，然后比较在客户端和服务器捕获的数据包。我们观察到，在触发审查后，客户端的数据包被GFW丢弃，并没有到达服务器。然而，服务器发送的数据包没有被阻断，客户端仍然可以收到。

UDP流量不受影响。 新的审查系统只限于TCP。发送一个具有随机有效载荷的UDP数据包不能触发审查。此外，即使某个3元组（客户端IP、服务器IP、服务器端口）由于TCP连接而被封锁，往来于同一（服务器IP、服务器端口）的UDP数据包也不受影响。由于没有UDP拦截，用户在使用Shadowsocks时可能会遇到奇怪的现象：他们仍然可以使用某些依赖UDP的网站或应用程序（如QUIC或FaceTime），但无法访问使用TCP的网站。这是因为Shadowsocks用TCP代理TCP流量，用UDP代理UDP流量。审查者不检测或阻止UDP流量，可能反映了其更糟就是更好(worse is better)的工程思维。从实际情况来看，目前的TCP封锁已经足够有效地让这些流行的翻墙工具瘫痪，而如果增加UDP审查，则需要额外的资源，并给审查系统引入额外的复杂性。

所有端口的流量都可能被阻断。 我们在美国建立了一个监听在所有端口（从1到65535）的水槽服务器。然后，我们让中国的客户端不断地用50字节的随机有效载荷与美国服务器的每个端口进行连接，并在某个端口被封锁后停止反复地连接这一端口。我们发现，从1到65535的所有端口都可能被封锁。因此，在一个不寻常的端口上运行翻墙服务器并不能缓解封锁。我们也没有观察到使用不同端口会导致不同的审查行为。

残留审查(residual censorship)的持续时间受到正在被残留审查的元组的数量的影响。我们发现，这个新的审查系统一旦阻断了一个连接，它就会在后续的120或180秒内继续丢弃所有具有相同3元组（客户端IP、服务器IP、服务器端口）的TCP数据包。这种行为通常被称为“残留审查” [13,17,63,14]。与其他一些残留审查系统不同 [13]，GFW的残留审查定时器不会在观察到更多触发审查的数据包后被重置。

我们还发现，GFW似乎限制了它在任何给定时间内残余审查的连接的数量。我们让中国的客户端重复性地同时连接到一个服务器的500个端口。在每个连接中，客户端发送50字节的随机数据，然后关闭连接。我们记录了每次发生残留审查的持续时长。如图2所示，与只有一个端口被封锁时的180秒持续时间相比，该实验中的残留审查持续时间大幅下降。

4.5 GFW是如何重组流量的

在这一节中，我们将研究GFW的新审查系统是如何重新组合流量，并考虑流量方向的。

一个完整的TCP握手是必要的。 我们观察到，发送一个SYN包，然后再发送一个包含随机数据的PSH+ACK包（在服务器没有完成握手的情况下），并不足以触发阻断。这样的残留审查更难被攻击者利用 [13]。

只有客户端到服务器的数据包可以触发阻断。 我们发现，GFW不仅检查随机数据是否被发送到属于受影响的IP范围内的目标IP地址，而且还检查并只在随机数据从客户端发送到服务器时才进行阻断。这里的服务器被定义为在TCP握手过程中发送SYN+ACK的主机。

我们通过在两台主机之间设置的四个实验来了解这一点。在第一个实验中，我们让在中国的客户端连接并向外国服务器发送随机数据；在第二个实验中，我们仍然让中国的客户端连接到外国服务器，但让外国服务器向客户端发送随机数据；在第三个实验中，我们让美国的客户端连接并向中国服务器发送随机数据；在第四个实验中，我们让美国的客户端连接到中国服务器，但随后让中国服务器向美国客户端发送随机数据。只有第一个实验中的连接被封锁了。

GFW只检查第一个数据包。 GFW似乎只分析TCP连接中的第一个数据包，而不对有多个数据包的流量进行重新组合。我们通过以下实验来测试这一点。在TCP握手后，我们发送第一个数据包，其中只有一个字节的有效载荷 \x21。在等待一秒钟后，我们再发送带有200字节随机有效载荷的第二个数据包。我们重复了25次实验，但连接从未被封锁。 这是因为在看到第一个数据包后，GFW已经通过规则Ex1豁免了连接，因为它的有效载荷中包含100%可打印的ASCII。换句话说，如果GFW在其流量分析过程中把多个数据包重新组合成一个流，它就能够阻止这些连接。

我们发现，GFW不会等到看到服务器的ACK响应时才去阻止一个连接。我们用一个iptables规则将我们的服务器配置为放弃任何传出的ACK数据包。然后我们用200字节的随机有效载荷与服务器建立连接。尽管服务器没有发送任何ACK数据包，GFW仍然阻止了这些连接。

GFW对第一个数据包等待时间超过了5分钟。 我们研究了GFW会在TCP握手之后，但在看到第一个数据包之前，对一个TCP连接进行了多长时间的监控。根据观察，它需要一个完整的TCP握手来触发封锁，我们因此推断GFW可能是有状态的。因此，我们有理由怀疑GFW只在有限的时间内监控一个连接，因为要永久追踪一个连接的状态而不放弃的开销很大。

我们的客户端完成了TCP握手，然后等待了100秒、180秒或300秒，然后发送200字节的随机数据。接着，我们重复了这个实验，但使用iptables规则丢弃了任何RST或TCP keepalive数据包，以防它们帮助GFW保持对连接的追踪。 我们发现这些连接仍然触发了阻断，这表明GFW对连接状态的追踪至少有5分钟。

5 与主动探测系统的关系

正如第2.2节所介绍的，GFW自2019年以来一直在向Shadowsocks服务器发送主动探测探针 [5]。在这一节中，我们研究了这个新发现的实时阻断系统和已知的主动探测系统之间的关系。通过测量实验和对历史数据集的分析，我们发现，虽然这两个审查系统并行工作，但主动探测系统的流量分析模块应用了算法1和图1中总结的所有五条豁免规则，并且还用一条额外的规则，来检查第一个数据包的有效载荷长度。我们还表明，主动探测系统[5]使用的流量分析算法可能自2019年以来有所进化。

主动探测实验。 在部署这个新的实时阻断系统之前，从外界推断主动探测系统的流量分析算法，如果说是有可能的话，也是极具挑战性的。这是因为GFW在看到触发连接和发送主动探测之间设置了一个任意长度的延迟 [5，§3.5]。这就使得我们很难说明GFW的哪些探测是由我们发送的哪些连接触发的。现在我们已经在第4节中推断出了这个新的阻断系统的流量检测规则列表，我们可以测试被算法1豁免的有效载荷是否也不会被主动探测系统所怀疑。

我们在2022年5月19日和6月8日之间进行了实验。如表2所示，我们制作了14种不同类型的有效载荷：其中3种是长度为2、50和200字节的随机数据；其余11种是具有不同长度的数据，这些数据仅能被算法1中的某一个豁免规则豁免。然后，我们从中国北京腾讯云的一个VPS向美国旧金山DigitalOcean的两个不同主机的14个端口，发送了14种有效载荷。其中一台美国主机已知受到当前阻断系统的影响，而另一台美国主机则不受影响。这样，如果我们收到来自GFW的任何探测，我们就知道当前封锁系统使用的某些豁免规则没有被主动探测系统使用。

我们在中国的客户端总共向两台美国服务器的每个端口发送了约17万次连接。然后我们采取措施，将来自GFW的主动探测与其他互联网扫描探测隔离开。我们根据IP2Location数据库 [3]和AbuseIPDB [2]检查每个探测的源IP地址。如果它是一个非中国的IP或者来自一个已知的被用来扫描的IP地址，我们就不认为它是来自GFW的主动探测。我们进一步检查并确认该探针是否属于GFW发送的任何已知类型的探针。

这两个系统独立工作。 新的审查机器纯粹是根据被动流量分析做出封杀决定，而不依赖中国知名的主动探测基础设施 [67,5,66,64,27]。我们之所以知道这一点，是因为虽然GFW仍然向服务器发送主动探测，但在超过99%的测试中，GFW在封锁一个连接之前没有向服务器发送过任何主动探测。举个例子，如表2所总结的，我们进行了33119次连接，但只收到179次主动探测。事实上，与之前的工作 [5, §4.2]的发现相似，主动探测很少被触发。

我们想强调的是，这一发现并不意味着对主动探测的防御没有必要或不再重要 [34,5,9]。恰恰相反，我们认为GFW对纯被动流量分析的依赖，部分原因是Shadowsocks、Outline、VMess和其他许多翻墙软件已经对主动探测采取了有效的防御措施 [34、5、9、19、43、32、71]。GFW仍然向服务器发送主动探测这一事实，意味着审查者仍然试图使用主动探测，尽可能准确地识别翻墙服务器。

主动探测系统对可疑流量应用了五条豁免规则，并增加了一条基于载荷长度的豁免规则。 这个实验表明了两点。首先，与Alice等人 [5, §4.2]的研究结果类似，主动探测系统应用一个额外的规则来检查连接中的有效载荷的长度。在我们的案例中，只有200字节有效载荷的连接曾经触发了主动探测，而2字节或50字节的连接则从来没有。其次，如果流量符合算法1中列出的五条豁免规则中的任何一条，那么该流量也不会触发主动探测系统。

自2019年以来，主动探测系统已经有所发展。 我们想知道算法1中的检测规则是否曾经被用来触发主动探测。为了分析它，我们从Alice等人 [5, §4.1]的低熵实验中获得了282个曾经被重放的有效载荷（这证明这些有效载荷曾经触发了GFW的主动探测）。然后我们写了一个程序来确定一个有效载荷是否会被当前的阻断系统豁免，并将获得的282个有效载荷输入该程序。结果，以前触发主动探测的45个探测被豁免了（根据规则Ex3）。2022年5月19日，我们反复发送这45个有效载荷，让它们被GFW看到，并确认它们确实被当前的阻断系统豁免了。对于每个有效载荷，我们用它从腾讯云北京的VPS到Digital Ocean旧金山的水槽服务器进行了25次连接。 这个结果表明， 自2020年以来，GFW很可能已经更新了其主动探测系统的流量分析模块。此外，目前GFW发送的探针也与2020年观察到的探针不同 [5, §3.2]。 新的探针基本上是随机有效载荷，分别以16、64和256字节为中位数的分布。对于这些长度中的每一个，GFW发送的探针数量大致相同：一台服务器收到了48、46和47个探针，另一台收到了238、228和233个探针。

6 了解阻断策略

在本节中，我们进行了测量实验，以确定审查者的封锁策略。我们发现，可能是为了减少误报和降低运营成本，审查者策略性地将封锁范围限制在热门数据中心的特定IP范围内，并对发往这些IP范围的所有连接采用概率性封堵策略，封锁率大约为26%。

6.1 互联网扫描实验

2022年5月12日，我们从位于科罗拉多大学博尔德分校的服务器上对互联网上10%的IPv4地址的TCP 80端口进行了扫描。按照前人工作中，如何识别在互联网扫描中发现不可靠主机的方法 [41]，我们排除了那些TCP响应窗口为0的服务器（因为我们无法向他们发送数据），以及不接受后续连接的IP地址。这就给我们留下了700万个可扫描的IP地址。然后，我们将这700万个IP地址随机平均分成九个组，并将每组分配到腾讯云北京数据中心的九个实验节点上。然后，我们将一个我们编写的测量程序安装在所有九个实验节点上，并用它进行实验。对于每个IP，该程序连续连接到其80端口，最多25次，每次连接间有一秒钟的间隔。在每个连接中，我们发送相同的50个字节的随机的、可以触发封锁的数据。如果我们在发送数据后看到连续5次连接超时（连接失败），我们就将该IP标记为受影响新审查系统的影响。 反之，如果所有25次连接都成功，我们则将该IP标记为未受到影响。我们将完全无法连接的IP标记为未知（例如，服务器关闭，或者与GFW无关的网络故障使我们无法首先连接）。

我们还重复了这个过程，但发送了50个字节的\x00，这个载荷并会不触发GFW的封锁。如果一个服务器在这个测试中也被标记为受到影响，那这很可能是由于服务器封锁了我们的连接，而不是GFW封锁的。我们从受到影响的IP结果中排除这些IP。这样就只剩下600多万个IP了。

最后，我们排除了可能是由于间歇性网络故障或不可靠的有利条件造成的 "模棱两可 "的结果。 具体来说，我们排除了那些被我们的随机载荷或全零载荷扫描标记为未知（我们从未能够连接），或有间歇性连接超时（例如，几个连接超时，但不是连续的5个）的IP。这就留下了550万个我们可以很容易地将其标记为不受影响（所有25个连接都成功了）或受影响（在某些时候，在我们发送随机数据后，它似乎被封锁了）的IP地址。

6.2 并非所有子网或自治系统都受到同等程度的审查

在经过处理的550万个IP中，98%的IP地址没有受到GFW封锁的影响，这表明中国在采用这种新的审查系统时是相当保守的。我们使用pyasn以及2022年4月的AS数据库 [51]，将这550万个IP地址归入其分配的IP前缀和AS中。 对于大于/20的IP前缀，我们将其分成每/20前缀一组，以保持分配的大小大致相同。我们的550万个IP包括了538个至少有5个测量结果的AS，其中绝大多数基本不受GFW的阻断影响。

图3显示了受影响的自治系统（ASes）和/20IP前缀的分布情况。我们发现，90%以上的AS是以全有或全无的方式受到影响的：要么我们在AS中测试的所有IP地址都受到GFW的阻断影响，要么我们在AS中测试的所有IP地址都没有受到影响。我们还观察到，只有少数AS受到影响：超过95%的AS被观察到只有不到10%的IP地址受到影响，只有7个AS被观察到其中有超过30%的IP地址受到了影响。

图4显示了受影响最大的AS。虽然测量结果偏向于显示较大规模的AS（因为在我们的扫描中占有更多的IP），但它显示了受到严重影响的AS（例如，阿里巴巴美国，Constant）和未受影响的AS（Akamai，Cloudflare）。此外，一些AS既有受影响的IP前缀，也有不受影响的IP前缀（亚马逊、Digital OCean、Linode）。我们看到的所有受影响或部分受影响的AS都是受欢迎的，可用于托管代理服务器的VPS供应商。而未受影响的大型AS通常不向个人客户出售VPS主机（如CDN）。

6.3 概率封堵的特点

正如第3节所介绍的，在得出任何关于封锁的结论之前，我们发送最多25次具有相同有效载荷的连接。这是必要的，因为审查者仅是有概率地实行封锁的。换句话说，仅仅向受影响的服务器发送一次随机的有效载荷，只是有时会触发阻断；但是，如果一个人不断向受影响的服务器发送相同有效载荷的连接，那么阻断终会发生。这就产生了一个疑问：一个连接被封锁的概率是多少？以及为什么审查者只是有概率地实行封锁？

估计封锁概率。 在我们对10%的互联网的扫描中（第6.2节）），有109,489个IP地址被我们标记为受到封锁影响。如第5节所示，在被封锁之前，我们可以与每个IP地址进行成功的随机数据连接的数量分布符合一个几何分布。这个结果表明，对每次连接的阻断是独立的，概率为\(26.3\%\)。

为什么采用概率封锁。 我们猜想，审查者采用概率封锁可能有两个原因：首先，它允许审查者只检查四分之一的连接，减少计算资源。第二，它帮助审查者减少对非翻墙连接的误伤。虽然这种减少也是以降低真阳性率为代价的，但残留审查可能弥补了这一点：一旦一个连接被封锁，其随后的连接也会被封锁数分钟。这使得翻墙流量一旦被发现就很难再成功连接。这也可能进一步支持了之前的说法，即审查者更重视降低检测中的假阳性率，而不执着于极高的真阳性率 [57]。

7 评估GFW的检测规则

在本节中，我们评估了第4节中推断出的GFW检测规则的假阳性率和我们推测的全面性。为了确定这种阻断对常规流量可能产生的影响，我们在不实际阻断任何流量的情况下，在我们大学的网络流量上模拟了推断出的检测规则。与GFW不同，我们对观察到的所有TCP连接模拟了检测规则，而没有将检测仅仅限制在26%的发往流行数据中心的特定IP范围。我们假设在大学网络中很少甚至没有翻墙流量，这样任何根据检测规则被阻断的流量就可能代表了检测中的假阳性。我们发现，我们推断出的检测算法将阻止大学网络上所有连接的大约0.6%。由于GFW的黑箱性质，我们推断出的规则可能只是GFW使用全部规则的一个子集；但是，证据表明，当我们将被判断为会被阻断的连接的前缀加上随机数据，一并从中国发往受到影响的IP地址时，所有被算法1判断为会被阻断的连接真的被GFW阻断了。这表明我们推断的规则很好地覆盖了GFW实际使用的规则。

7.1 流量分析实验

我们拥有对科罗拉多大学博尔德分校40Gbps的实时网络流量的访问权。这一权限允许我们处理校园内所有传入和传出的数据包的副本。利用这一点，我们收集了一个数据集，其中只包含目的地IP和端口号，以及不符合算法1中任何豁免规则的连接的的有效载荷的前6个字节。更确切地说，我们使用PF_RING [50]实现了一个自定义的数据包分析工具。对于每个连接，我们检查了客户端发送的第一个数据包。我们确保该数据包具有正确的TCP校验和，并且其序列号是连接中TCP握手后的第一个预期数据包（确保我们没有错过第一个数据包）。对于那些没有被算法1豁免的连接，即那些我们预计会被封锁的连接，我们记录了目标端口和连接的前六个字节，以帮助识别其协议。

我们在2022年7月至2022年9月期间进行了这种收集。我们总共分析了17亿个连接，并记录了44,2928个不同的将被封锁的连接的前6字节。对于这44,2928个6字节前缀中的每一个，我们将相同的194字节的随机数据附加到它上面，使之成为一个200字节的有效载荷。然后，我们在2022年9月重复发送每个有效载荷，使其经过真正的GFW，以测试它们是否真的被封锁了，或者反而有我们之前没有发现的豁免情况。对于每个有效载荷，我们携带着它，从腾讯云北京的VPS向Digital Ocean旧金山的水槽服务器发送多达25个连接。

7.2 实验结果和分析

估算假阳性率。 在2022年7月至2022年9月期间，我们总共分析了该大学网络上的17亿个连接。对于每个连接，我们确定算法1中的哪些豁免规则可以使其免于被阻断。如图6所示，我们观察到，在我们推断的GFW的检测规则下，该大学平均有0.6%的TCP连接会被阻止。

审查者至少采用了两种策略来减少假阳性（误封）。首先，正如第6节所介绍的，GFW只对一部分IP子网进行审查。这一决定可能是为了缓解审查者所面临的基本比率谬误问题(base rate fallacy) [11]。由于翻墙连接相对占比较小，所以即使算法的的误报率很小（如0.6%），但如果算法被广泛应用于所有流量，也会导致误伤很多的非翻墙流量。通过缩小其适用的IP范围，中国可以减少其检测算法带来的误伤。其次，正如第6.3节所探讨的那样，即使是对这一小部分发往受到审查影响的IP子网上的流量，GFW也只阻止所有流量的四分之一。这就进一步将误伤数量降低到原本的四分之一。

我们发现的0.6%的连接也有可能就是完全加密流量。为了研究这种可能性，我们对每个连接中看到的独特的6字节前缀的数量进行了统计，根据推断出的规则，这些前缀将被GFW阻止。如果这些连接都真的是完全加密的代理，那么我们与其会看到在可能的6字节值上有一个均匀的分布（256^6）。否则，如果有频繁出现的6字节值，那么这些前缀则有可能属于流行协议，这也就代表GFW的流量检测出现了误报。

图7显示了在我们推断的GFW规则下，来自大学网络的实时流量中，被标记会被阻断的所有970万个连接的前6字节的分布情况。此外，表3显示了将被阻断的连接的前6字节值。虽然我们无法识别其中的许多协议，但它们的出现频率和低熵表明，它们不可能是完全加密的代理。

估计推断出的规则的全面性。 在我们制作的442,928个有效载荷中，我们发现只有一个前缀被GFW豁免了，它提醒我们注意TLS应用数据前缀豁免（\x17\x03[\x00-\x09]）。我们把这条豁免规则加入到了我们的推断规则中（Ex5）。这一结果表明，我们推断的豁免规则很好地覆盖了GFW实际使用的检测规则。

8 规避策略

我们对这个新的审查制度的理解帮助我们能够推导出多种规避策略。在第8.1节和第8.2节中，我们介绍了两种被广泛采用的规避检测的方法，它们分别从2022年1月和2022年10月开始帮助中国的用户绕过这次的新审查技术。我们在附录A中讨论了其他规避策略。我们负责任地及时与拥有数百万用户的各种流行的翻墙软件的开发者分享了我们的发现和建议，这一点我们将在第8.3节中详细说明。

8.1 可定制的有效载荷前缀

算法1中的豁免规则Ex2和Ex5只查看连接中的前几个字节。这样做能让GFW高效地豁免不是完全加密的流量；但这样的做法同时也使其可以被用来规避检测。具体来说，我们建议在（翻墙）连接的第一个数据包的有效载荷上预置一个可定制的前缀。

可定制的IV头。 Shadowsocks连接以初始化向量（IV）开始，根据加密方式的不同，其长度为16或32字节 [22]。正如第4.2节所介绍的，将IV的前6个（或更多）字节变成可打印的ASCII，将使这些连接被Ex2规则豁免。同样，将IV的前3、4或5个字节变成普通协议头，将使连接被Ex5规则豁免（例如，将IV的前3个字节变成0x16 0x03 0x03）。这些对策只需对客户端进行很小的改动，而对服务器没有任何改动，因此已经被许多流行的翻墙工具所采用 [72,62,48,56]。将32字节IV的前几个字节限制为可打印的ASCII，不会将其随机性降低到影响加密安全性的程度。例如，即使将前6个字节固定为某个可打印的ASCII，IV中仍然有26个随机字节，这仍然比典型的16字节IV的随机性要大。

局限性。 这是一个权宜之计，有可能被审查者很容易地阻止。审查者可能会跳过前几个字节，将检测规则应用于连接中的其余数据。协议模仿在实践中也很困难 [39]。审查者可以执行更严格的检测规则，或对服务器进行主动探测，以检查它是否真的在运行TLS或HTTP。然而，这一策略在自2022年1月被许多流行的翻墙工具采用后，直到目前的2023年2月仍然有效。这一事实强调了即使是简单的应对方案也能有效对抗资源有限的审查者 [8,57,30]。

8.2 改变Popcount

正如第4.1节所介绍的，如果一个连接的第一个数据包每字节的1比特数量的平均值（popcount）小于等于3.4或大于等于4.6（Ex1），GFW就会豁免该连接。基于这一观察，人们可以通过在数据包中插入额外的1（或者0）来增加（减少）popcount，以绕过封锁。我们设计并分析了一个灵活的方案，它可以将每字节的popcount改变为任何给定的值或范围。我们在Shadowsocks-rust [54]和Shadowsocks-Android [6]上实现了这个方案。自2022年10月以来帮助中国的用户绕过封锁 [8]。2023年1月，面向中国的一个大型翻墙服务提供商（要求不具名），也采用了这个方案的其中一个版本，并取得了类似的成功。

从一个高度概括的层面看，我们把原始的完全加密的数据包作为输入：通过只对密文进行操作，我们无需承担其保密性被破坏的风险。当发送一个数据包时，我们首先计算其每字节的平均popcount；如果该值大于4，那么我们计算我们必须向数据包中添加多少个1比特，以获得一个超过4.6popcount的载荷。反之，如果popcount小于4，那么我们要计算要增加多少个0比特才能使popcount减少到3.4以下。在任何一种情况下，我们在原始密文中添加必要数量的1比特或0比特，然后添加4个字节，表示所添加的比特数，最终给我们一个比特串\(B\)，使其每字节的popcount值会被豁免。

当然，简单地添加1或0会很容易产生协议指纹。为了解决这个问题，我们进行比特级的随机重新排序。特别是，我们利用现有的共享秘密，如密码，作为一个种子，以确定的方式构建一个置换向量。在每个连接中，我们更新这个排列向量，并在发送前用它来对比特串\(B\)中的所有比特重新排序。为了解码，接收方首先更新排列向量，然后用它来还原对比特串的排序；然后读取最后4个字节来确定增加的比特数，删除该比特数，从而能够恢复原来的（完全加密）数据包。

在实践中，我们额外采取了两个步骤来进一步混淆流量。因为如果所有的连接都共享相同的平均字节popcount值，那这是就会成为一个明显的指纹，所以我们将popcount的目标值设置为一个可参数化的范围。其次，由于明文的4字节长度标签可能成为一个指纹，所以我们对其进行加密（与这些翻墙工具对代理流量进行加密的方式相同）。

这个方案有几个优点。首先，该方案支持可参数化地调整平均每字节popcount的值，以防GFW更新其popcount规则来缩小被豁免的popcount范围。其次，由于它的精心设计，没有明显的指纹会向审查者发出信号，表明这是一个经过popcount调整的数据包。最后，它的流量开销很低；它只添加严格需要数量的1（或0）比特。在最坏的情况下，即把popcount从4增加到4.6，这只产生了大约17.6%的额外开销。因此，它不仅可以应用于第一个数据包，还可以应用于连接中的每一个数据包。这样即使审查者在未来检测除第一个数据包以外的数据包，这一策略也仍然有效。

8.3 负责任的披露

2021年11月16日，在GFW采用这种新的封锁方式的十天后 [10]，我们向公众披露了这种新封锁方式的细节 [37,38]。随着我们对这种新封锁的更深入理解，我们得出并评估了不同的规避策略。我们负责任地及时与拥有数百万用户的各种流行的翻墙工具的开发者分享了我们的发现和建议，包括Shadowsocks [22], V2Ray [59], Outline [42], Lantern [20], Psiphon [21], and Conjure [33]。下面我们详细介绍我们的披露和反审查社区的反应。

2022年1月13日，我们与一群开发者分享了我们的第一个规避策略。这个解决方案，详见第8.1节，只需要对客户端进行的很少量的代码修改，而无需对服务器进行任何修改。到2022年1月14日，Shadowsocks-rust开发者zonyitoo、V2Ray开发者Xiaokang Wang和Sagernet开发者nekohasekai已经将这个规避解决方案作为选项添加到他们的客户端 [72,62,48]。2022年10月4日，database64128在Shadowsocks-go上实现了这种策略的用户自定义版本 [18]。2022年10月25日，Outline开发者为他们的客户端采用了一个高度可定制的解决方案 [56]。2022年10月14日，我们发布了修改过的Shadowsocks [8]，其采用了我们在第8.2节中详述的改变popcount的策略。

截至2023年2月14日，这些工具所采用的所有规避策略据报告在中国仍然有效。2023年1月，Outline的开发者报告说，自从他们采用上述缓解措施后，（选择加入匿名数据统计的）Outline服务器的数量增加了一倍。2023年1月，中国的一家大型翻墙服务提供商（目前要求不具名）也采用了我们提出的方案，并且也取得了成功。

虽然我们没有研究中国以外的国家，但根据汇报，我们提出的规避策略在另一个封锁全加密代理的国家——伊朗，也是有效的 [65]。2023年2月13日，Lantern的开发者报告说，自2023年1月以来，所采用的协议"占了我们来自伊朗流量的大部分"。2023年2月13日，另一个翻墙服务提供商报告说，自2022年11月启用Outline的规避功能后，他们的服务从完全被封锁变成了每天为85万名来自伊朗的用户提供服务。

9 伦理

对审查机器的测量研究带有一定的风险和责任，我们对此非常重视。我们的研究涉及处理敏感的网络流量，扫描大量的主机，并在一个敏感国家进行网络测量。由于这项工作的敏感性，我们向我们机构的伦理审查委员会(IRB)提出了详细的研究计划以供审核。虽然IRB决定这项工作不涉及人类主体（因此不需要IRB审核），但我们仍然设计并采取了广泛的防范措施，以尽量减少潜在的风险和伤害。在本节中，我们将讨论这些风险，并详细介绍我们为管理和减轻这些风险而采取的防范措施。

流量分析。 我们在部署我们的网络测量工具时，与我们大学的网络运营商密切合作。他们在管理此类项目方面有着丰富的经验，可以确保我们的测量工具在网络使用政策允许的范围内，并尊重用户隐私。我们设计的实验避免收集潜在的敏感信息，如IP地址，因为这可能会暴露人类的身份信息。我们收集最少的信息，并专注于跟踪汇总统计，以避免可能分析个人信息的风险。具体来说，我们只分析每个连接中的第一个TCP数据包，而忽略任何后续数据包。此外，我们只记录前六个字节的数据，并对其出现次数进行汇总统计；没有任何原始流量被人类检查或记录。我们奉行最小特权原则，只让我们团队中的一部分人访问这些数据。

互联网扫描。 为了最大限度地减少因互联网扫描让某些服务器不堪重负的风险，我们遵循了先前在互联网扫描和大规模审查制度测量方面的最佳做法 [26,60]。我们在科罗拉多大学博尔德分校的扫描主机的80端口上设置了一个专门的网页，以及一个反向DNS。该网页解释了我们的扫描收集了哪些数据，并提供了选择退出未来扫描的方法。在我们的整个实验期间，我们收到并履行了七个删除请求，根据过去扫描互联网的经验，七个请求属正常情况 [25，§5.3] [26，§5.1]。我们对这些服务器的后续扫描只占用很低的带宽：我们为每个请求发送不到100字节，而且对每台服务器每一次只发送一个连接，以避免压垮他们的网络或连接池资源。

实验节点的使用。 从受审查国家内部对审查进行主动测量需要额外的考虑和审慎的评估。我们首先探索了采用远程测量的可能性，但确认这种审查不能从中国境外触发。虽然被审查者观察到敏感连接所产生的风险可能很低，但我们仍然遵循先前工作中讨论的类似标准，限制了我们发送的敏感连接的数量 [5]。特别是，我们只将80端口的查询发送给正在监听该端口的服务器，并且没有对同一服务器进行并发连接，以避免让服务器不堪重负。

我们的研究团队咨询了对中国审查制度的性质和法律问题有着深刻理解的专家，他们帮助我们在使用哪些VPS供应商和如何使用它们方面做出了明智的决定。我们选择了两家由知名商业公司经营的大型VPS供应商，以避免任何潜在的针对个人的法律风险。我们用一位既不是中国公民也没有居住在中国的研究人员的准确身份和联系信息注册了我们的VPS。在整个研究过程中，我们没有收到来自VPS供应商的投诉。 正如在之前的工作中所做的那样 [5]，我们没有提前通知这些大型的VPS供应商。这样做一方面避免了潜在的实验偏差（如对结果的干扰），另一方面也避免了给VPS供应商带来潜在的法律义务或负担。

我们对服务器可能被GFW暂时或长期封锁的风险进行管理。对于在本研究中我们控制的所有主机，我们给它们分配了专用的IP地址，以避免共享的IP地址被封锁。此外，我们从一家允许翻墙，甚至提供自动安装翻墙工具的VPS供应商那里租用了我们位于非审查网络的主机。我们使用自己的服务器进行了测试，与之前关于中国残留审查的研究结果类似 [13,17,63,14]，我们确认GFW从未封锁我们任何机器的IP地址超过180秒，而且封锁只影响我们自己的客户端到服务器的流量，不干扰其他人的客户端到服务器的流量。在我们的服务器已经被使用了五个月后仍未被长期封锁的情况下，我们才开始进行大规模扫描。

10 结论

在这项工作中，我们曝光并研究了中国最新的审查系统，该系统可以实时动态地封锁完全加密流量。这种强大的新审查形式已经部分地或完全地影响了许多主流翻墙工具，包括Shadowsocks、Outline、VMess、Obfs4、Lantern、Phiphon和Conjure。我们进行了广泛的测量，以推断GFW的流量分析算法的各种属性，并用真实世界的流量评估了算法的全面性和误报率。我们利用我们对这个新审查系统的了解，得出有效的规避策略。我们负责任地将我们的发现和建议透露给不同的反审查工具的开发者，从而帮助数百万用户成功地规避了这种新形式的封锁。

鸣谢

我们感谢我们的牧羊人和其他匿名审稿人的宝贵意见和反馈。我们也感谢勇敢的中国用户向我们立即报告封锁事件。我们感谢Benjamin M. Schwartz, zonyitoo, nekohasekai, database64128, AkinoKaede, Max Lv, Mygod, DuckSoft, 以及其他许多来自反审查社区的开发者，感谢他们及时的补丁、协助和讨论。我们对来自Jigsaw的Outline开发者Vinicius Fortuna表示诚挚的感谢，感谢他提供有见地的建议并协助我们接触社区。我们感谢Lantern开发者Adam Fisk和Ox Cart分享了他们的工具在伊朗的部署情况。我们也感谢Milad Nasr提供的丰富信息。 我们感谢klzgrad针对论文初稿分享的独到思考和评论。我们也非常感谢David Fifield，他提供了一个针对obfs4的概念验证补丁，为研究讨论做出了贡献，对论文的早期草稿提供了建设性的反馈和建议，并在整个研究过程中提供了指导和支持。

这项工作得到了美国国家科学基金会CNS-1943240、CNS-1953786、CNS-1954063和CNS-2145783拨款的部分支持，也得到了美国国防部高级研究计划局（DARPA）的青年教师奖计划（DARPA-RA-21-03-09-YFA9-FP-003）和DARPA的HR00112190125号协议支持。所表达的观点、意见和/或发现属于作者本人，不应解读为代表美国国防部或美国政府的官方观点或政策。批准公开发布；分发无限制。

可用性

为了保持可复现性并鼓励后续的研究，我们发布了我们的源代码和数据： https://gfw.report/publications/usenixsecurity23/en。

参考文献

A 其他暂时性的规避策略

使用非TCP传输协议。 正如第4.4节所介绍的，UDP流量不会触发阻断。目前，人们可以通过简单地切换到（或隧道传输）UDP或QUIC来翻墙。这只是一种权宜之计，因为审查者可以对UDP进行审查。

对第一个数据包进行Base64编码。 回顾一下，如果第一个数据包的50%以上的字节是可打印的ASCII码，GFW就不会对连接进行审查。满足这一特性的一个直接方法是对所有的加密流量进行简单的base64编码。这也只是一个权宜之计；base64编码的数据很容易被发现，审查者可以简单地进行base64解码，然后应用其规则。虽然它今天对GFW很有效，但我们不认为它是一个长期的解决方案。

超过20个连续的可打印的ASCII字节。 如果第一个数据包有超过20个连续的可打印的ASCII字节，GFW就会豁免连接。满足这一要求的方法之一是只对完全加密的数据包的一小部分进行base64编码，或者甚至只是在密码文本中插入至少21个可打印的ASCII字符。虽然我们认为这比对整个数据包进行base64编码更难发现，但它也让我们觉得仅仅是一种短期的权宜之计。

所有上述对策都可以只在客户端实现，而不需要代理服务器的支持。 这可以通过应用先前工作 [12]中的一个想法来实现：发送一个诸如上述的数据包，由审查者处理，而不是由代理服务器处理。例如，在发送连接的实际第一个数据包之前，客户端可以发送一个满足上述规则之一的数据包，但它有一个错误的校验值（审查者不会检查，但代理会）或一个有限的TTL（大到足以到达审查者，但到达不了目标服务器）。 虽然这些技术首先是针对伊朗的协议过滤器验证的，但我们已经验证了相同的方法对GFW针对完全加密流量的封锁也起作用。虽然这为部署提供了一个令人鼓舞的简单途径，但它本身并没有将这些权宜之计提升为长期的解决方案。

How the Great Firewall of China Detects and Blocks Fully Encrypted Traffic

Abstract

One of the cornerstones in censorship circumvention is fully encrypted protocols, which encrypt every byte of the payload in an attempt to “look like nothing”. In early November 2021, the Great Firewall of China (GFW) deployed a new censorship technique that passively detects—and subsequently blocks—fully encrypted traffic in real time. The GFW’s new censorship capability affects a large set of popular censorship circumvention protocols, including but not limited to Shadowsocks, VMess, and Obfs4. Although China had long actively probed such protocols, this was the first report of purely passive detection, leading the anti-censorship community to ask how detection was possible.

In this paper, we measure and characterize the GFW’s new system for censoring fully encrypted traffic. We find that, instead of directly defining what fully encrypted traffic is, the censor applies crude but efficient heuristics to exempt traffic that is unlikely to be fully encrypted traffic; it then blocks the remaining non-exempted traffic. These heuristics are based on the fingerprints of common protocols, the fraction of set bits, and the number, fraction, and position of printable ASCII characters. Our Internet scans reveal what traffic and which IP addresses the GFW inspects. We simulate the inferred GFW’s detection algorithm on live traffic at a university network tap to evaluate its comprehensiveness and false positives. We show evidence that the rules we inferred have good coverage of what the GFW actually uses. We estimate that, if applied broadly, it could potentially block about 0.6% of normal Internet traffic as collateral damage.

Our understanding of the GFW’s new censorship mechanism helps us derive several practical circumvention strategies. We responsibly disclosed our findings and suggestions to the developers of different anti-censorship tools, helping millions of users successfully evade this new form of blocking.

1 Introduction

Fully encrypted circumvention protocols are a cornerstone of censorship circumvention solutions. Whereas protocols like TLS begin with a handshake that comprises plaintext bytes, fully encrypted (randomized) protocols—such as VMess [23], Shadowsocks [22], and Obfs4 [7]—are designed such that every byte in the connection is functionally indistinguishable from random. The idea behind these “looks like nothing” protocols is that they should be difficult for censors to fingerprint and therefore costly to block.

On November 6, 2021, Internet users in China reported blockings of their Shadowsocks and VMess servers [10]. On November 8, an Outline [42] developer reported a sudden drop in use from China [69]. The start of this blocking coincided with the sixth plenary session of the 19th Chinese communist party central committee [4, 1], which was held on November 8–11, 2021. Blocking these circumvention tools represents a new capability in China’s Great Firewall (GFW). To our knowledge, although China has been using passive traffic analysis and active probing together to identify Shadowsocks servers since May 2019 [5], it is the first time the censor has been able to block fully encrypted proxies en masse in real time, completely based on passive traffic analysis. The importance of fully encrypted protocols to the entire anti-censorship ecosystem and the mysterious behaviors of the GFW motivate us to explore and understand the underlying mechanisms of detection and blocking.

In this work, we measure and characterize the GFW’s new system for passively detecting and censoring fully encrypted traffic. We find that, instead of directly defining what fully encrypted traffic is, the censor applies at least five sets of crude but efficient heuristics to exempt traffic that is unlikely to be fully encrypted traffic; it then blocks the remaining non-exempted traffic. These exemption rules are based on common protocol fingerprints, a crude entropy test using the fraction of set bits, and the fraction, position, and maximum contiguous count of ASCII characters in the first TCP payload.

Due to the black-box nature of the GFW, our inferred rules may not be exhaustive; however, we evaluate our inferred rules on real-world traffic from a network tap at CU Boulder, and provide evidence that our rules have significant overlap with the GFW’s. We also find that the inferred detection algorithm would block roughly 0.6% of all connections on our network tap. Possibly to mitigate over-blocking caused by false positives, our Internet scans show that the GFW strategically only monitors 26% of connections and only to specific IP ranges of popular data centers.

We also analyze the relationship between this new form of passive censorship and the GFW’s well-known active probing system [5], which operate in parallel. We find that the active probing system also relies on this traffic analysis algorithm but has additional packet length-based rules applied. Consequently, the circumvention strategies that can evade this new blocking will also prevent the GFW from identifying and subsequently active-probing the proxy servers.

We derive various circumvention strategies from our understanding of this new censorship system. We responsibly and promptly shared our findings and circumvention suggestions with the developers of various popular anti-censorship tools, including Shadowsocks [22], V2Ray [59], Outline [42], Lantern [20], Psiphon [21], and Conjure [33]. These circumvention strategies have been widely adopted and deployed since January 2022, helping millions of users bypass this new censorship. As of February 2023, all circumvention strategies these tools adopted are reportedly still effective in China.

2 Background

2.1 Traffic Obfuscation Strategies

Tschantz et al. divide approaches to obfuscating censorship circumvention traffic into two types: steganograpic and polymorphic  [57, § V]. The goal of steganographic proxies is to make circumvention traffic look like allowed traffic; the goal of polymorphism is to make circumvention traffic not look like forbidden traffic.

The two most common approaches to achieving steganography are mimicking and tunneling. Houmansadr et al. [39] conclude that mimicking a protocol is fundamentally flawed and suggest that tunneling through allowed protocols be a more censorship-resistant approach. Frolov and Wustrow [35] demonstrate that even when a tunneling approach is used, it still requires effort to perfectly align protocol fingerprints with popular implementations, in order to avoid blocking by protocol fingerprints. For instance, in 2012, China and Ethiopia deployed deep packet inspection to detect Tor traffic by its uncommon ciphersuits [44, 67, 55]. Censorship middlebox vendors have previously identified and blocked meek [29] traffic based on on its TLS fingerprint and SNI value [28].

To avoid this complexity, many popular proxies opt for polymorphic designs. A common way to achieve polymorphism is to fully encrypt the traffic payload, starting from the first packet in a connection. Without any plaintext or fixed header structure to fingerprint, the censor cannot easily identify proxy traffic with regular expressions or by looking for specific patterns in traffic. This design was first introduced in Obfuscated OpenSSH in 2009 [16]. Since then, it has been employed by Obfsproxy [24], Shadowsocks [22], Outline [42], VMess [23], ScrambleSuit [68], Obfs4 [7], and partially used in Geph4 [58], Lantern [20], Psiphon3 [21], and Conjure [33].

Fully encrypted traffic is often referred to as “looks like nothing” traffic, or misunderstood as “having no characteristics”; however, a more accurate description would be “looks like random”. In fact, such traffic does have an important characteristic that sets it apart from other traffic: Fully encrypted traffic is indistinguishable from random. Since there are no identifiable headers, traffic will have high entropy homogeneously throughout the entire connection, even in the first data packet. By contrast, even encrypted protocols like TLS have relatively low-entropy handshake headers that convey supported versions and extensions.

In 2015, Wang et al. [61, §5.1] used the length and high Shannon entropy of the first packet payload in a connection to identify randomized traffic, like Obfs4. Similarly, in 2017, Zhixin Wang released a proof-of-concept tool that used the high Shannon entropy of the first three packets’ payloads in a connection to identify Shadowsocks traffic [40]. Madeye extended the tool to additionally use the payload length distribution to detect ShadowsocksR traffic [47]. He et al. [70, §IV.A] and Liang et al. [46, §II.A] used a single-bit frequency detection algorithm, rather than the Shannon entropy, to measure the randomness of Obfs4 traffic. In 2019, Alice et al. found that the GFW was using the length and entropy of the first data packet in each connection to suspect Shadowsocks traffic [5].

2.2 Active Probing Attacks and Defenses

In active probing attacks, the censor sends well-crafted payloads to a suspected server and measures how it responds. If the server responds to these probes in an identifiable way (e.g. lets the censor use it as a proxy), the censor can block it. As early as August 2011, the GFW was observed to send seemingly random payloads to foreign SSH servers that accepted SSH logins from China [49]. In 2012, the GFW first looked for a unique TLS ciphersuit to identify Tor traffic; it then sent active probes to the suspected servers to confirm its guess [67, 66, 64]. In 2015, Ensafi et al. conducted a detailed analysis of the GFW’s active probing attacks against various protocols [27]. Since May 2019, China has deployed a censorship system to detect and block Shadowsocks servers in two steps: It first uses the length and entropy of the first packet payload in each connection to passively identify possible Shadowsocks traffic, and then sends various probes, in different stages, to the suspected servers to confirm its guess [5]. In response, researchers proposed various defenses against active probing attacks, including consistent server reactions [34, 9] and application fronting  [45, 36]. Shadowsocks, Outline, and V2Ray have incorporated probe-resistant designs [34, 5, 19, 43, 32, 71], making them unblocked in China since September 2020 [5], until the recent blocking in November 2021 [10].

3 Methodology

We crafted and sent various test probes between hosts inside and outside of China, letting them be observed the GFW. We observed the GFW’s reactions by capturing and comparing traffic on both endpoints. This logging allows us to identify any dropped or manipulated packets, as well as active probes.

Experiment timeline and Vantage points. We summarized the timeline and vantage point usage of all major experiments in Table 1. In total, we used ten VPSes in TencentCloud Beijing (AS45090) and one VPS in AlibabaCloud Beijing (AS37963). We did not observe any differences in the censoring behavior between our vantage points within China or any affected external vantage points. We used four VPSes in DigitalOcean San Francisco (AS14061): three of them were affected by the new censorship, the other one was not. We turned these four VPSes into sink servers; that is, the servers listen on all ports from 1 to 65535 to accept TCP connections, but do not send any data back to the client. We also employed two machines in the CU Boulder (AS104) for Internet scanning and live traffic analysis. We checked the IP addresses of our VPSes against IP2Location database [3], confirming their geo-locations are as reported by their providers.

Triggering censorship. Because fully encrypted traffic is indistinguishable from random data , beyond using actual circumvention tools, we developed measurement tools that send random data to trigger blocking in our study. The tools initiate a TCP handshake, send a random payload of a given length, and then close the connection.

Using residual censorship to confirm blockings. Similar to how the GFW blocks many other protocols [13, 17, 63, 14], after a connection triggers the censorship, the GFW blocks all subsequent connections having the same 3-tuple (client IP, server IP, server port) for 180 seconds. This residual censorship allows us to confirm blocking by sending follow-up connections from the same client to the same port of the server. We make five TCP connections one by one with a one-second interval in between. If all five connections failed, we conclude that the 3-tuple is blocked. Once a 3-tuple is blocked, we do not use it for further tests in the next 180 seconds.

Accouting for probabilistic blocking with repeated tests. We often had to make multiple connections with the same payload before we observed blocking. In Section 6.3, we explain that this is because the GFW employs a probabilistic blocking strategy, where censorship is only triggered approximately a quarter of the time. To account for this probabilistic behavior, we send the same payload in up to 25 connections before drawing any blocking (or not blocking) conclusion. If we can successfully make 25 connections with the same payload in a row, then we conclude that the payload (or server) is not affected by this censorship. If after sending the payload at least once, a sequence of 5 subsequent connection attempts timeout (due to residual censorship), we label the payload (and server) as affected by censorship. We use this method of repeated connections to measure blocked payloads in all the tests throughout our study.

4 Characterizing the New Censorship System

We conduct experiments to understand how the GFW detects and blocks fully encrypted connections. Detailed in Table 1, between Nov 6, 2021 and May 18, 2022, we used three VPSes in China and three sink servers in the US to conduct our experiments. During the same period, we also used one VPS in AlibabaCloud Beijing (AS37963) to repeat all our experiments. We did not observe any differences in the censoring behavior among our vantage points within China or any affected external vantage point. On February 16, 2023, we reran our experiments and confirmed all detection rules still held. This time, we used one VPS in TencentCloud BJ and one sink server in DigitalOcean SFO.

Algorithm 1 presents a high-level overview of the GFW’s detection rules we inferred, and Figure 1 illustrates examples of these inferred rules in action. While we cannot infer the order in which these rules get applied or if they are exhaustive, our experiments confirm specific components of the GFW’s censorship strategy. We find that, instead of directly defining what fully encrypted traffic is, the censor applies at least five sets crude but efficient heuristic rules to exempt traffic that is unlikely to be fully encrypted traffic; it then blocks the remaining non-exempted traffic. These exemption rules are based on common protocol fingerprints, a crude entropy test using the fraction of bits set, and the fraction, position, and maximum contiguous count of ASCII characters.

4.1 Entropy Exemption (Ex1)

We observed that the fraction of bits set influences whether a connection is blocked. To determine this, we sent repeated connections to our server and observed which were blocked. In each connection, we sent one of 256 different byte patterns, consisting of 1 byte repeated 100 times (e.g., \x00\x00\x00..., \x01\x01\x01..., ..., \xff\xff\xff...). We sent each pattern in 25 connections to our server, and observed if any patterns resulted in blocking subsequent connections, indicating the payload triggers blocking. We found 40 byte patterns triggered blocking , while the remaining 216 patterns did not. Example patterns that were blocked include \x0f\x0f\x0f..., \x17\x17\x17..., and \x1b\x1b\x1b...(and 37 others).

All of the blocked patterns consist of bytes with exactly 4 (out of 8) bits that were \(1\) (for instance, \x1b in binary is 00011011). We hypothesized that the number of set bits (\(1\) bits) per byte may play a role, as uniformly random data will have close to the same number of total \(1\)s and \(0\)s in binary. In effect, this is essentially measuring the entropy of the bits within the client’s packet.

We confirmed this by sending combinations of bytes that were individually allowed, but together resulted in being blocked. For example, both \xfe\xfe\xfe... and \x01\x01\x01... were not blocked individually, but these bytes sent together as \xfe\x01\xfe\x01... resulted in blocking. We note \xfe\x01 has 8 (out of 16) bits set to \(1\) (an average of 4 bits per byte set), while \xfe has 7 out of 8, and \x01 has 1 of 8 set, explaining why individually they are allowed, but together they are blocked.

Of course, random or encrypted data will not always have exactly half of the bits set to \(1\). We tested how close to half the GFW needed in order to block, by sending a sequence of 50 random bytes (400 bits) with an increasing number of bits set. We produced 401 bitstrings with 0–400 bits set to \(1\), and shuffled each string, yielding a set of random strings with 0–8 bits set per byte (in increments of 0.02 bits/byte). For each string, we made 25 connections and sent the string to observe if it triggered subsequent connections to be blocked. We found that all strings with \(\le 3.4\) or \(\ge 4.6\) bits/byte set were not blocked, while strings with between 3.4 and 4.6 bits/byte set were blocked.

There was a single exception to this for a string with 4.26 bits/byte set, which we determined was not blocked due to having over 50% of its bytes be printable ASCII characters; we show next this is an exemption rule (Ex2). We repeated our experiment and confirmed that other strings with the same number of bits set with less printable ASCII are indeed blocked.

In summary, we find that the GFW exempts a connection if the fraction of bits set in the client’s first data packet deviates from half. This corresponds to a crude measure of entropy: random (encrypted) data will have close to half of the bits set to \(1\), while other protocols usually have fewer \(1\) bits per byte due to plaintext or zero-padded protocol headers. For instance, Google Chrome version 105 sends a TLS client hello with an average of only 1.56 bits set per byte, falling outside the censorship range, owing to padding with zeros.

4.2 ASCII Characters Exemption (Ex2-4)

We observed several exceptions to the bit counting rule we discovered in Section 4.1. For instance, the pattern \x4b\x4b\x4b... was not blocked, despite having exactly 4 bits set per byte. Indeed, there are actually 70 characters (8 choose 4) that have exactly 4 bits set, but our analysis found that only 40 of those triggered censorship. What about the other 30?

These other 30 byte values all fall within the byte range that comprises the printable ASCII characters , 0x20–0x7e. We conjecture that the GFW exempts characters presumably to allow “plaintext” (human-readable) protocols.

We found three ways in which the GFW exempts connections based on printable ASCII characters in the first packet payload from the client: if the first six bytes are printable (Ex2); if more than half of the bytes are printable (Ex3); or if it contains more than 20 contiguous printable bytes (Ex4).

First six bytes are printable (Ex2). We observe that the GFW exempts blocking if the first 6 bytes of a connection fall within the printable byte range 0x20–0x7e. If there are characters outside this range in the first 6 bytes, then a connection may be blocked, assuming it does not have other exempting properties (for example, fewer than 3.4 bits per byte set). We tested this by generating messages where the first \(n\) bytes were sourced from different character sets (such as ASCII printable characters) and the rest of the message would be random unprintable characters. We find that for \(n < 6\), we observe censorship, but for \(n \ge 6\) where the first \(n\) bytes are ASCII printable characters, no blocking occurs.

Half of the first packet are printable (Ex3). If more than half of all bytes in the first packet fall into the printable ASCII range 0x20–0x7e, the GFW exempts the connection. We tested this by sending packets consisting of 10 bytes of characters outside this range (e.g. 0xe8), followed by a repeating sequence of 6 bytes: 5 within the range (e.g., 0x4b), and one outside. We repeat this 6 byte sequence 5 times, and then pad the end of the string with \(n\) bytes outside the range (in Python notation: "\xe8"*10 + ("\x4b"*5 + "\xe8")*5 + "\xe8"*n). This experiment gives us a variable-length pattern that decreases the fraction of bytes in the printable ASCII range as we increase \(n\). We find that for \(n < 10\), connections are not blocked, while for \(n \ge 10\) they are. This corresponds to blocking when the fraction of printable characters is less than or equal to half, and not blocking when greater than half.

We design our probes to avoid triggering other GFW exemptions, such as bit counts (Ex1), printable prefixes (Ex2), or runs of printable characters (Ex4). For example, we use 0x4b and 0xe8 as our printable and non-printable characters respectively, since they both have exactly 4 bits set. This prevents the GFW from exempting our connection from blocking due to the bit count rule (Ex1) discussed previously. In addition, we avoid having contiguous runs of printable 0x4b characters, as we observed that such runs can also exempt a connection from blocking, which we discuss next. We repeated our experiments with other patterns that also met these constraints (e.g. 0x8d and 0x2e), and observed the same results.

More than 20 contiguous bytes are printable (Ex4). A contiguous run of printable characters can also exempt blocking, even if the total fraction of printable characters is less than half. To test this, we sent a pattern of 100 bytes of a character outside the printable range (0xe8) with a varying number of contiguous bytes from the printable range (we used 0x4b). Our payload started with 10 bytes of 0xe8, followed by \(n\) bytes of 0x4b, and then \(90-n\) bytes of 0xe8, for a total length of 100 bytes. We varied \(n\) from 0–90, and sent each of the 91 payloads in 25 connections to our server. We found that with \(n \le 20\), the connection was blocked. For \(n > 20\), the connection was not blocked, indicating the presence of a run of printable characters exempts blocking. Of course, past \(n > 50\), the connection will also be exempt, because of Ex3.

Other encodings. We tested whether Chinese characters in the first packet were exempted from blocking in the same way as printable ASCII characters did. We used strings of 6–36 Chinese characters encoded in UTF-8, as well as GBK (identical to GB2312 for the character we used). All of these tests were blocked, suggesting that there is no exemption for Chinese characters. It is possible that the presence of Chinese characters in these encodings is rare, or that parsing these encodings adds unjustified complexity since it is hard to know where an encoded string starts or ends.

4.3 Common Protocols Exemption (Ex5)

To avoid blocking popular protocols by mistake, we observe that the GFW explicitly exempts two popular protocols. The GFW appears to infer protocols from the first 3–6 bytes of the client’s packet : If they match the bytes of a known protocol, the connection is exempted from blocking, even if the rest of the packets do not conform to the protocol. We tested six common protocols and found that the TLS and HTTP protocols are explicitly exempted. This list may not be exhaustive, as there may be other exempted protocols we did not test.

TLS. TLS connections start with a TLS Client Hello message, and the first three bytes of this message cause the GFW to exempt the connection from blocking. We observe that the GFW exempts any connection whose first three bytes match the following regular expression:

[\x16-\x17]\x03[\x00-\x09]

This corresponds to the one-byte record type, followed by a two-byte version. We enumerated all 256 patterns of XX\x03\x03 followed by 97 bytes of random data, and found all patterns were blocked except those that start with either 0x16 (corresponding to the Handshake TLS record type, used in the Client Hello) or 0x17 (corresponding to the Application Data record type). While normal TLS connections do not begin with Application Data [53, 52], when TLS is used over Multipath-TCP (MPTCP) [31], it is common for one of the TCP subflows to be used for the Client Hello and for other subflows to send Application Data immediately after the TCP connection is established [15]. As of today, only TLS versions 0x03[0x00-0x03] have been defined [53, 52], but the GFW allows even later (not yet defined) versions.

HTTP. The byte pattern used by the censor to identify HTTP traffic is simply the method followed by a space. If a message starts with GET␣, PUT␣, POST␣, or HEAD␣, the connection will be exempt from blocking. The space character (0x20) after each verb is necessary to exempt connections from blocking. Not including this space character, or replacing it with any other byte will not exempt the connection. The other HTTP methods (OPTIONS␣, DELETE␣, CONNECT␣, TRACE␣, PATCH␣) fall into the ASCII printable exemption (Ex2), as the first 6 bytes are printable characters. We find that the method is case-insensitive: GeT␣, get␣, and similar variations are exempt. Typos in the verb (e.g., TEG␣) are not exempt.

Non-exempted protocols. We tested other common protocols: SSH, SMTP, and FTP would be exempt as they all start with at least 6 bytes of printable ASCII (rule Ex2). DNS-over-TCP is exempt due to containing a large fraction of zeros, making it exempt by the Ex1 rule. However, if a large enough amount of random data was appended after a DNS-over-TCP message, it would be blocked.

This observation raises the question of why the censor has explicit rules to exempt TLS and HTTP, but not other protocols. After all, the censor does not need to exempt these two protcols explicitly: HTTP will commonly be exempt by printable ASCII for the first 6 bytes (rule Ex2), and TLS Client Hello messages have relatively low bit-wise entropy (rule Ex1), owing to many zero fields. Nonetheless, the censor may employ these simple but efficient rules to quickly exempt the bulk of traffic (TLS and HTTP) from the more in-depth analysis of calculating the popcount, fraction of ASCII, etc.

4.4 How the GFW Disrupts Connections

Once the GFW detects fully encrypted traffic using Algorithm 1, it blocks the subsequent traffic as introduced below.

Packets are dropped from client to server. We triggered the GFW’s blocking and compared the captured packets from both the sending client and receiving server. We observe that after triggering blocking, the client’s packets are dropped by the GFW, and do not reach the server. However, packets sent by the server are not blocked and are still received at the client.

UDP traffic is not affected. The new censorship system is limited to TCP. Sending a UDP datagram with a random payload cannot trigger the blocking. Additionally, once a 3-tuple (client IP, server IP, server Port) is blocked due to a triggering TCP connection, UDP datagrams to or from the same (server IP, server Port) are not affected. Because of the absence of UDP blocking, users may experience odd behavior while using Shadowsocks: they can still access websites or use apps that rely on UDP (e.g. QUIC or FaceTime), but cannot access websites that use TCP. This is because Shadowsocks proxies TCP traffic with TCP and proxies UDP traffic with UDP. Not detecting or blocking UDP traffic may reflect the censor’s worse is better engineering mindset. From a practical view, the current TCP blocking can already effectively paralyze these popular circumvention tools, while employing UDP censorship requires additional resources and invites extra complexity to the censorship system.

Traffic on all ports can get blocked. We set up a sink server listening on all ports from 1 to 65535 in US. We then let our client in China continuously make connections with 50-byte random payloads to each port of the US server and stop when a port got blocked. We find that blocking can happen on all ports from 1 to 65535. Therefore, running circumvention servers on an unusual port cannot mitigate the blocking. We also do not observe any difference in censor’s behaviors among ports.

The duration of residual censorship is affected by the number of on-going residual blocking. We find that once this new censorship system blocks a connection, it continues to drop all subsequent TCP packets having the same 3-tuple (client IP, server IP, server port) for 120 or 180 seconds. This behavior is often referred to as “residual censorship” [13, 17, 63, 14]. Unlike some other residual censorship systems [13], the GFW’s residual censorship timer does not reset when additional packets are sent.

We also find that the GFW seems to limit the number of connections it residually blocks at any given time. We let our clients in China repetitively make connections to 500 ports of a single server simultaneously. In each connection, the client sent 50 bytes of random data and then closed the connection. We recorded the duration of each occurrence of residual censorship. As shown in Figure 2, in comparison to the 180 s duration when only one port is blocked, the residual censorship duration in this experiment decreased dramatically.

4.5 How the GFW Reassembles Flows

In this section, we examine how the GFW’s new censorship system reassembles flows and considers flow directions.

A complete TCP handshake is necessary. We observe that sending a SYN packet followed by a PSH+ACK packet containing random data (without the server completing its end of the handshake) is not sufficient to trigger blocking. The blocking is thus harder to exploit for residual censorship attacks [13].

Only client-to-server packets can trigger the blocking. We find that the GFW not only checks if the random data is sent to a destination IP address that falls in an affected IP range, it also examines and will only block if the random data is sent from client to server. The server here is defined as the host that sends a SYN+ACK during the TCP handshake.

We learned this by setting up four experiments between the same two hosts. In the first experiment, we let the Chinese client connect and send random data to the foreign server; in the second experiment, we still let the Chinese client connect to the foreign server, but let the foreign server send random data to client; in the third experiment, we let the US client connect and send random data to Chinese server; in the forth experiment, we let the US client connect to the Chinese server, but then let the Chinese server send random data to the US client. Only connections in the first experiments were blocked.

The GFW only examines the first data packets. The GFW appears to only analyze the first data packet in a TCP connection, without reassembling the flows with multiple data packets. We tested this with the following experiment. After a TCP handshake, we send the first data packet with only one byte of payload \x21. After waiting for one second, we then send the second data packet with a 200-byte random payload. We repeated the experiment 25 times, but the connections never got blocked. This is because after seeing the first data packet, the GFW had already exempted the connections by rule Ex1 as it contained 100% printable ASCII in the payload. In other words, if the GFW reassembled multiple packets into a flow during its traffic analysis, it would have been able to block these connections.

We found that the GFW does not wait until seeing an ACK response from the server to block a connection. We configured our server to drop any outgoing ACK packets with an iptables rule. We then made connections with 200-byte random payloads to the server. The GFW still blocked these connections though the server never sent any ACK packets.

The GFW waits more than 5 minutes for the first data packets. We examine how long the GFW monitors a TCP connection after the TCP handshake, but before it sees the first data packet. From the observation that it requires a complete TCP handshake to trigger the blocking, we infer the GFW may be stateful. It is thus reasonable to suspect the GFW only monitors a connection for a limited amount of time, as it can be expensive to maintain a state forever without expiring it.

Our client completed TCP handshakes and then waited for 100, 180, or 300 seconds, before sending 200 bytes of random data. We then repeated the experiment but used iptables rules to drop any RST or TCP keepalive packets in case they helped the GFW keep the connection state active. We found that these connections still triggered blocking, suggesting the GFW maintained connection states for at least five minutes.

5 Relation with the Active Probing System

As introduced in Section 2.2, the GFW has been sending active probes to Shadowsocks servers since 2019 [5]. In this section, we study the relationship between this newly discovered real-time blocking system and the existing active probing system. By conducting designed measurement experiments and analyzing historical datasets, we show that while these two censorship systems work in parallel, the current traffic analysis module of the active probing system applies all five sets of exemption rules summarized in Algorithm 1 and Figure 1, with one additional rule that examines the payload length of the first data packet. We also show evidence that the traffic analysis algorithm used by the active probing system [5] may have evolved since 2019.

Active probing experiment. Prior to the deployment of this new real-time blocking system, inferring the traffic analysis algorithm of the active probing system was extremely challenging, if possible at all. This is because the GFW employs an arbitrary delay between seeing a triggering connection and sending active probes [5, §3.5], making it difficult to account for which probes by the GFW are triggered by which connections we send. Now that we have inferred a list of traffic detection rules of this new blocking system in Section 4, we can test if a payload exempted by Algorithm 1 will also not get suspected by the active probing system.

We conducted the experiments between May 19, 2022 and June 8, 2022. As shown in Table 2, we crafted 14 different types of payloads: three of them are random data with lengths of 2, 50, and 200 bytes; the remaining 11 were data with various lengths that will only be exempted by exactly one of the exemption rules in Algorithm 1. We then sent the same 14 payloads from a VPS in Tencent Cloud Beijing China, to 14 ports of two different hosts in DigitalOcean San Francisco US. One US host is known to be affected by the current blocking system, while the other US host is unaffected. This way, if we received any probes from the GFW, we know certain exemption rules used by the current blocking system are not used by the active probing system.

In total, our client in China sent around 170k connections to each port of the two US servers. We then took steps to isolate the GFW’s probes from other Internet scanners’. We check the source IP address of each probe against IP2Location database  [3] and AbuseIPDB [2]. We do not consider it as a probe from the GFW if it was a non-Chinese IP or from a known spammer IP address. We further check if the probe belongs to any known types of probes sent by the GFW.

The two systems work independently. The new censorship machine makes its blocking decisions purely based on passive traffic analysis, without relying on China’s well-known active probing infrastructure [67, 5, 66, 64, 27]. We know this because, while the GFW still sends active probes to the servers, in more than 99% of the tests, the GFW did not send any active probes to the server before blocking a connection. For example, as summarized in Table 2, we made 33,119 connections but only received 179 active probes. Indeed, similar to the findings by prior work [5, §4.2], active probes are rarely triggered.

We want to emphasize that this finding does not mean that defenses against active probing are not necessary or not important anymore [34, 5, 9]. On the contrary, we believe that the GFW’s reliance on purely passive traffic analysis is partially because Shadowsocks, Outline, VMess, and many other censorship circumvention implementations have adopted effective defenses against active probing [34, 5, 9, 19, 43, 32, 71]. The fact that the GFW still sends active probes to servers implies that the censor still attempts to use active probing to accurately identify circumvention servers whenever possible.

The active probing system applies the five exemption rules, with one additional length rule, to suspect traffic. This experiment suggests two points. First, similar to the findings by Alice et al. [5, §4.2], the active probing system applies an additional rule to examine the length of the connection. In our case, only connections with 200-byte payloads ever triggered the active probing, not ones with 2 bytes or 50 bytes. Second, the traffic exempted by any of the five rules discovered in  Algorithm 1 will also not trigger the active probing system.

The active probing system has evolved since 2019. We want to know if the same detection rules in Algorithm 1 were historically used to trigger active probing. To analyze it, we obtained 282 payloads that got replayed (and thus once triggered the GFW) in the low-entropy experiment from Alice et al. [5, §4.1]. We then wrote a program to determine if a payload would be exempted by the current blocking system, and fed the program with the obtained 282 payloads. As a result, 45 probes that previously triggered active probing were exempted (by rule Ex3). On May 19, 2022, we repeatedly sent these 45 payloads through the GFW, confirming that they were indeed exempted from the current blocking. For each payload, we made 25 connections with it from a VPS in TencentCloud Beijing to a sink server in DigitalOcean SFO. This result suggests that the GFW has likely updated the traffic analysis module of its active probing system since 2020. In addition, the probes sent by the current GFW are also different from those observed in 2020 [5, §3.2]. The new probes are essentially random payloads that are distributed in trios of 16, 64, and 256 bytes. For each of these lengths, the GFW sent about the same number of probes: 48, 46, and 47 to one server, and 238, 228, and 233 to the other.

6 Understanding the Blocking Strategies

In this section, we conduct measurement experiments to characterize the censor’s blocking strategies. We find that, possibly to mitigate false positives and reduce operation costs, the censor strategically limits the scope of blocking to specific IP ranges of popular data centers, and it applies a probabilistic blocking strategy to 26% of all connections to these IP ranges.

6.1 Internet Scanning Experiment

On May 12, 2022, we performed a 10% IPv4 Internet scan on TCP port 80, from a server located at CU Boulder. Following prior work that identifies unreliable hosts in Internet scans [41], we remove IPs that respond with a TCP window of 0 (as we cannot send them data), or do not accept a subsequent connection. This leaves us with 7 million scannable IPs. We then randomly and equally split these 7 million IP addresses into nine subsets, and assigned each to our nine vantage points in TencenCloud Beijing datacenter. We then used a measurement program we wrote and installed in all nine vantage points for the experiment. For each IP, the program connects to its port 80 sequentially up to 25 times, with a one-second interval in between. In each connection, we send the same 50 bytes of random data that can trigger the blocking. If we see 5 consecutive connections time out (fail to connect) after we have sent data, we label the IP as affected. Otherwise, if all 25 connections succeed, we label the IP as unaffected. We label IPs that we cannot connect to at all as unknown (e.g., the server is down, or a network failure unrelated to the GFW prevents us from connecting in the first place).

We also repeated this process but sent 50 bytes of \x00, which does not trigger blocking by the GFW. If a server is marked as affected in this test, it is likely due to the server blocking us, and not the GFW, and we remove these IPs from our results. This leaves just over 6 million IPs.

Finally, we remove “ambiguous” results that may be due to intermittent network failures or unreliable vantage points. Specifically, we remove IPs that either of our random or zero scans labelled unknown (we were never able to connect), or had intermittent connection timeouts (e.g., several connections timed out, but not 5 consecutively). This leaves 5.5 million IPs that we can easily label as unaffected (all 25 connections succeeded) or affected (at some point it appeared blocked after we sent random data).

6.2 Not All Subnets/ASes are Affected Equally

Of the 5.5 million processed IPs, 98% of them are unaffected by the GFW’s blocking, suggesting that China is fairly conservative in employing this new censorship. We group these 5.5 million IP addresses into their allocated IP prefixes and ASes, using pyasn with an AS database from April 2022 [51]. For IP prefixes larger than /20, we break the allocation into a set of /20 prefixes to keep allocations roughly the same size. Our 5.5 million IPs comprise 538 unique ASes that have at least 5 results, and the vast majority of these are largely unaffected by the GFW’s blocking.

 Figure 3 shows the distributions of the fraction of affected ASes and /20 prefixes. We found that more than 90% ASes are affected in an all-or-nothing way: either all IP addresses we tested in the AS are affected by the GFW’s blocking, or no IP addresses we tested in the AS are unaffected. We also observe that only a few ASes are affected: over 95% of ASes see less than 10% of their IPs affected, and only 7 ASes see more than 30% of their IPs affected.

 Figure 4 shows the top affected ASes. While this is skewed toward larger ASes (which have more IPs in our scan), it shows both ASes that are heavily affected (e.g., Alibaba US, Constant) and ones that are not (Akamai, Cloudflare). In addition, some ASes have a mix of affected and not affected prefixes (Amazon, Digital Ocean, Linode). All of the affected or partly-affected ASes we see are popular VPS providers that could be used to host proxy servers, while large unaffected ASes do not typically sell VPS hosting to individual customers (e.g. CDNs).

6.3 Characterizing Probabilistic Blocking

As introduced in Section 3, we send up to 25 connections with the same payload before drawing any conclusions about blocking. This is necessary because the censor implements blocking probabilistically. In other words, just sending a random payload to an affected server once would only sometimes trigger blocking; however, if one keeps making connections with the same payload to the affected server, blocking will occur eventually. This raises the question on what the probability is for a connection to get blocked, and why the censor implements blocking only probabilistically.

Estimating the blocking rate. From our 10% Internet scan (Section 6.2), there were 109,489 IP addresses that we label as blocked. As shown in Section 5, the distribution of the number of successful random data connections we can make to each IP address before getting blocked fits a geometric distribution. This result suggests that the blocking of each connection is independent, with a probability of \(26.3\%\).

Why probabilistic blocking is used. We conjecture that the censor employs probabilistic blocking possibly for two reasons: First, it allows the censor to only examine one-fourth of connections, reducing computation resources. Second, it helps the censor reduce the collateral damage to non-circumvention connections. While this reduction also comes at the expense of lower true positives, the residual censorship may make up for it: once a connection is determined to be blocked, subsequent connections are also blocked for several minutes after, making it difficult for proxy users to successfully connect once detected. This may also further support prior claims that censors put more emphasis on reducing their false positive rate than in achieving a high true positive rate [57].

7 Evaluating the GFW’s Detection Rules

In this section, we evaluate the false positive rate and comprehensiveness of the GFW’s detection rules we inferred in Section 4. To determine the impact this blocking may have on regular traffic, we simulate the inferred detection rules to traffic on our university network without actually blocking any traffic. Different from the GFW, we simulate the detection rules against all TCP connections observed without limiting the detection to 26% of connections to specific IP ranges of popular data centers. We expect to see little to no circumvention traffic in this network, and any traffic that would be blocked under detection rules likely represents false positive blocking. We find that the inferred detection algorithm would block roughly 0.6% of all connections on our network. Due to the black-box nature of the GFW, our inferred rules may only be a subset of what the GFW uses; however, we show that all connections that Algorithm 1 would block were indeed blocked when we sent their prefixes along with random data through the GFW, suggesting our inferred rules have good coverage of what the GFW uses.

7.1 Traffic Analysis Experiment

We have access to a 40 Gbps network tap at CU Boulder that allows us to process copies of all incoming and outgoing packets on our campus. Using this, we collected a dataset comprising only destination port numbers and the first 6 bytes of payload data for connections that do not already satisfy the other exemption rules in Algorithm 1. More precisely, we implemented a custom packet analysis tool using PF_RING [50]. For each connection, we inspected the first data packet sent by the client. We ensured that the packet has a correct TCP checksum, and that its sequence number is the first expected data packet after the TCP handshake in the connection (making sure we have not missed the first data packet). For connections that are not exempted by Algorithm 1—i.e., those we expect to be blocked—we logged the destination port and the first six bytes of the connection to help identify its protocol.

We performed this collection between July 2022 and September 2022. In total, we analyzed 1.7 billion connections and logged 442,928 unique 6-byte prefixes of would-be-blocked connections. For each of these 442,928 6-byte prefixes, we append the same 194-byte random data to it to make a 200-byte payload. We then repetitively sent each payload past the real GFW in September 2022, to test whether they were indeed blocked, or if instead there were exemptions we had not previously identified. For each payload, we made up to 25 connections with it from a VPS in TencentCloud Beijing to a sink server in DigitalOcean SFO.

7.2 Experiment Results and Analysis

Estimating the false positive rate.   In total, we analyzed 1.7 billion connections on our network between July 2022 and September 2022. For each connection, we determine which rules in Algorithm 1 would exempt it from being blocked. As shown in Figure 6, we observe on average that 0.6% of TCP connections from our tap would be blocked under the GFW’s detection rules we inferred.

There are at least two strategies the censor employs to reduce the false positive rate. First, as introduced in Section 6, the GFW only applies this censorship to a fraction of IP subnets. This decision may be an attempt to mitigate the base-rate problem faced by the censor [11]. Since relatively few connections in total are proxy connections, even a small false positive rate (such as 0.6%) would result in blocking mostly benign traffic, if applied broadly. By narrowing the scope of IPs it is applied to, China can reduce the collateral damage of its censorship. Second, as explored in Section 6.3, even for traffic towards this subset of IP subnets, the GFW is observed to block only about one-quarter of all traffic, reducing the false positive rate to one-fourth.

It is possible that the 0.6% of connections we identified may be fully encrypted proxies. To investigate this possibility, we keep a count of the number of unique 6-byte prefixes we see in each connection that would be blocked under the GFW’s rules. If these connections are all truly fully encrypted proxies, we would expect to see a uniform distribution over the \(256^6\) possible 6-byte values. Otherwise, if there are 6-byte values that occur frequently, it could be headers of popular protocols, indicating false positives in the GFW’s blocking.

Figure 7 shows the distribution of the first 6 bytes of all 9.7 million connections from our tap that would be blocked under the GFW rules we inferred. In addition, Table 3 shows the top 6-byte values from would-be blocked connections. While we are not able to identify many of these protocols, their frequency along with the low entropy indicates that they are not likely to be fully encrypted proxies.

Estimating the comprehensiveness of the inferred rules. Among the 442,928 payloads we crafted and sent past the real GFW, we found only one prefix got exempted by the GFW, which alerted us to the TLS Application Data prefix exemption (\x17\x03[\x00-\x09]). We added this exemption to our inferred rules (Ex5). This result suggests our inferred rules have good coverage of what the GFW uses.

8 Circumvention Strategies

Our understanding of this new censorship system allows us to derive multiple circumvention strategies. In Section 8.1 and Section 8.2, we introduce two widely adopted countermeasures that have been helping users in China bypass censorship since January 2022 and October 2022, respectively. We discuss other circumvention strategies in Section A. We responsibly and promptly shared our findings and suggestions with the developers of various popular anti-censorship tools that have millions of users, which we detail in Section 8.3.

8.1 Customizable Payload Prefixes

The exemption rules Ex2 and Ex5 from Algorithm 1 only look at the first several bytes in a connection, allowing the GFW to efficiently exempt non-fully encrypted traffic; however, this lends itself to a potential countermeasure. Specifically, we propose prepending a customizable prefix to the payload of the first packet in a (circumvention) connection.

Customizable IV header. Shadowsocks connections begin with an Initialization Vector (IV), which is of length 16 or 32 bytes depending on the encryption ciphers [22]. As introduced in Section 4.2, turning the first six (or more) bytes of the IVs into printable ASCII will exempt connections by the rule Ex2. Similarly, turning the first three, four, or five bytes of the IVs into common protocol headers will exempt connections by the rule Ex5 (e.g., turning the first three bytes of an IV into 0x16 0x03 0x03). These countermeasures require minimal changes to the client and no changes to the server, and therefore has been adopted by many popular circumvention tools [72, 62, 48, 56]. Restricting the first few bytes of a 32-byte IV to be printable ASCII will not reduce the randomness to the point that affects the security of encryption. For example, even fixing the first six bytes to printable ASCII still leaves the IVs with 26 random bytes, which is still more than a typical 16-byte IV.

Limitations. This is a stopgap solution and could potentially be blocked by the censor fairly easily. The censor may skip the first several bytes and apply the detection rules to the rest data in a connection. Protocol mimicry is also difficult in practice [39]. The censor can enforce stricter detection rules, or actively probe a server to check if it is genuinely running TLS or HTTP. Nevertheless, the fact that this strategy still works as of February 2023, more than one year since its adoption by many popular circumvention tools in January 2022, underscores that even simple solutions can be effective against finite-resourced censors [8, 57, 30].

8.2 Altering Popcount

As introduced in Section 4.1, the GFW exempts a connection if its first data packet has an average popcount-per-byte \(\le 3.4\) or \(\ge 4.6\) (Ex1). Based on this observation, one can increase (decrease) the popcount by inserting additional ones (zeroes) into the packet to bypass censorship. We introduce and analyze a flexible scheme that alters the popcount-per-byte to any given value or range. We implemented this scheme on Shadowsocks-rust [54] and Shadowsocks-android [6], helping users in China bypass censorship since October 2022 [8]. In January 2023, a large-scale circumvention service in China (that asked not to be named), also implemented a version of this scheme and found similar success.

At a high level, we take original fully-encrypted packets as input: By operating only on the ciphertexts, we do not risk violating confidentiality. When sending a packet, we first compute its average popcount-per-byte; if the value is greater than 4, then we determine how many one-bits we would have to add to the packet in order to obtain a popcount over 4.6. Conversely, if the popcount is less than 4, then we determine how many zero-bits we would have to add to decrease the popcount to less than 3.4. In either case, we append the necessary number of one- or zero-bits to the original ciphertext and then append 4 bytes denoting the number of bits added, ultimately giving us a bit-string \(B\) that has a popcount-per-byte that would not subject it to censorship.

Of course, simply appending ones or zeroes would be easy to fingerprint. To address this, we do bit-level random shuffling. In particular, we leverage the existing shared secrets, such as password, as a seed to deterministically construct a permutation vector. In each connection, we update this permutation vector and use it to shuffle all the bits in the bit-string \(B\) before sending it. To decode, the receiver first updates the permutation vector and then uses it to un-shuffle the bit-string; then it reads the last 4 bytes to determine the number of bits added, removes that number of bits, and is thus able to recover the original (fully encrypted) packet.

In practice, we take two additional steps to further obfuscate the traffic. Since it is an obvious fingerprint if all connections share the same popcount-per-byte value, we set the goal value to a parameterizable range. Second, since the 4-byte length tag in plaintext may be a fingerprint, we encrypt it (the same way these circumvention tools encrypt proxy traffic).

This scheme has several advantages. First , the scheme supports parameterizable popcount-per-byte in case the GFW updates its popcount rule to block an even larger range. Second , because of its careful design, there are no obvious fingerprints that would signal to the censor that this is a popcount-adjusted packet. Finally , it incurs low overhead; it adds only as many ones (or zeroes) strictly necessary (padded to the nearest byte). In the worst case—increasing the popcount from 4 to 4.6—this incurs only about 17.6% overhead. As a result, it could feasibly be applied not just to the first packet, but to every packet in the connection, thereby insulating it against future updates to the censor that might look past the first packet.

8.3 Responsible Disclosure

On November 16, 2021, ten days after the GFW employed this new blocking [10], we revealed details of this new blocking to the public [37, 38]. With the development of our understanding of this new blocking, we derived and evaluated different circumvention strategies. We responsibly and promptly shared our findings and suggestions with the developers of various popular anti-censorship tools that have millions of users , including Shadowsocks [22], V2Ray [59], Outline [42], Lantern [20], Psiphon [21], and Conjure [33]. Below we introduce our disclosure and the responses from the anti-censorship community in detail.

On January 13, 2022, we shared our first circumvention strategy with a group of developers. This solution, detailed in Section 8.1, requires minimal code changes to the clients and no changes to the servers. By January 14, 2022, Shadowsocks-rust developer zonyitoo, V2Ray developer Xiaokang Wang and Sagernet developer nekohasekai had already added this circumvention solution as an option to their clients [72, 62, 48]. On October 4, 2022, database64128 implemented a user-customizable version of this strategy on Shadowsocks-go [18]. On October 25, 2022, Outline developers adopted a highly customizable solution for their client [56]. On October 14, 2022, we released a modified Shadowsocks [8] that employed the popcount-altering strategy we detailed in Section 8.2.

As of February 14, 2023, all circumvention strategies adopted by these tools are reportedly still effective in China. In January 2023, Outline developers reported that the number of Outline servers (that opted-in for anonymous metrics) had doubled since they adopted the mitigation above. In January 2023, a large circumvention service provider in China (that asked not to be named at this time) also implemented our proposed scheme and has also found success.

While we did not study countries other than China, our proposed circumvention strategies are reported to be also working in Iran, another country that reportedly blocks and throttles fully encrypted proxies [65]. On February 13, 2023, Lantern developers reported that the adopted protocol “accounted for the majority of our Iran traffic” since January 2023. On February 13, 2023, a different circumvention service provider reported that, after enabling Outline’s mitigation feature in November 2022, their services turned from being completely blocked to serving 850k daily users from Iran.

9 Ethics

Censorship measurement research carries an element of risk and responsibility which we take seriously. Our research involves handling sensitive network traffic, scanning large numbers of hosts, and performing network measurements in a sensitive country. Due to the sensitive nature of this work, we approached our institution’s IRB with our detailed research plan for review. While the IRB determined that the work does not involve human subjects (and thus does not require IRB review), we have designed and implemented extensive precautionary efforts to minimize potential risks and harms. In this section, we discuss these risks and detail the precautionary measures we adopted to manage and mitigate them.

Traffic analysis. We worked closely with our university’s network operators, who have extensive experience in managing such projects, to deploy our network measurement tool to ensure it is within the network use policy and respects user privacy. We design our experiments to avoid collecting potentially sensitive information, such as IP addresses, which could reveal human identifiable information. We collect minimal information and focus on tracking aggregate statistics to avoid potentially identifying individuals. Specifically, we only analyzed the very first TCP data packet in each connection and ignored any subsequent packets. In addition, we only logged the first six bytes of data and keep an aggregate count of their occurrences; no raw traffic was ever inspected by a human nor logged. We practiced the least privilege principle, giving only a subset of our team access to this data.

Internet scanning. To minimize the risk of overwhelming servers when performing Internet-wide scans, we followed the best practices outlined in prior work in Internet scanning and widescale censorship measurement [26, 60]. We set up a dedicated webpage, along with a reverse DNS to it, on port 80 of our scanning host at CU Boulder. The webpage explains what data our scanning collects, and offers ways to opt out of future scans. During our entire experiment period, we received and honored seven removal requests, which is typical based on past experiences scanning the Internet [25, §5.3] [26, §5.1]. Our follow-up scans to these servers were low-bandwidth: we sent less than 100 bytes for each request, and each server only performed one connection at a time to avoid overwhelming their network or connection pool resources.

The use of vantage points. Active censorship measurement from within censored countries requires additional considerations and prudent evaluation. We first explored the possibility of performing the measurement remotely but confirmed that this censorship could not be triggered from outside of China. While it may be low risk to have sensitive queries observed by the censor, we follow similar standards discussed in prior work to limit the number of these sensitive queries we send [5]. In particular, we only send queries on port 80 to servers that are listening on that port, and made no concurrent connections to the same server to avoid overwhelming server operations.

Our research team consulted experts with a deep understanding of the nature and legal concerns of Chinese censorship, who helped us make informed decisions on which VPS providers to use and how to use them. We selected two large-scale VPS providers run by well-known commercial companies in order to avoid any potential legal risks to individuals. We registered our VPSes with the accurate identity and contact information of one of our researchers who is neither a citizen of nor resides in China. We received no complaints from the providers throughout our research. As done in prior work [5], we do not inform these large VPS providers of the experiments ahead of time, to avoid potential experiment bias (e.g. interference in results) or placing potential legal obligations or burdens on the VPS providers.

We manage the risk of potentially getting any server blocked by the GFW temporarily or in the long term. For all hosts we controlled in this study, we assigned dedicated IP addresses to them to avoid blocking shared IP addresses. In addition, we rented our non-censoring network hosts from a VPS provider that permits censorship circumvention usage and even offers automatic installation of circumvention tools. Similar to the findings in prior work on residual censorship in China [13, 17, 63, 14], we tested using our own servers and confirmed that the GFW never blocked any of our machines’ IP addresses for more than 180 seconds, and the blocking only affected traffic from our clients to the servers, without interfering with traffic from others’. Knowing that our servers were used for five months but never experienced any long-term blocking, we proceeded to perform our large-scale scans.

10 Conclusion

In this work, we exposed and studied China’s latest censorship system that dynamically blocks fully encrypted traffic in real time. This powerful new form of censorship has affected many mainstream circumvention tools partially or in full, including Shadowsocks, Outline, VMess, Obfs4, Lantern, Phiphon, and Conjure. We conducted extensive measurements to infer various properties about the GFW’s traffic analysis algorithm and evaluated its comprehensiveness and false positives against real-world traffic. We use our knowledge of this new censorship system to derive effective circumvention strategies. We responsibly disclosed our findings and suggestions to the developers of different anti-censorship tools, helping millions of users successfully evade this new form of blocking.

Acknowledgments

We thank our shepherd and other anonymous reviewers for their valuable comments and feedback. We also thank the brave users in China for immediately reporting the blocking incidents to us. We are grateful to Benjamin M. Schwartz, zonyitoo, nekohasekai, database64128, AkinoKaede, Max Lv, Mygod, DuckSoft, and many other developers from the anti-censorship community for their prompt patches, assistance, and discussions. We express our sincere appreciation to Outline developer, Vinicius Fortuna, at Jigsaw for offering insightful suggestions and assisting us in reaching out to the community. We thank Lantern developers Adam Fisk and Ox Cart for sharing the deployment experience of their tool in Iran. We also thank Milad Nasr for his informative input. We appreciate klzgrad sharing thoughtful comments on an earlier draft of the paper. We are also deeply grateful to David Fifield for providing a proof-of-concept patch against obfs4, contributing to the discussions, providing constructive feedback and suggestions on an earlier draft of the paper, and offering guidance and support throughout the entire study.

This work was supported in part by NSF grants CNS-1943240, CNS-1953786, CNS-1954063 and CNS-2145783, by the Young Faculty Award program of the Defense Advanced Research Projects Agency (DARPA) under the grant DARPA-RA-21-03-09-YFA9-FP-003, and by DARPA under Agreement No. HR00112190125. The views, opinions, and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government. Approved for public release; distribution is unlimited.

Availablity

To maintain reproducibility and encourage future research, we released our source code and data: https://gfw.report/publications/usenixsecurity23/en.

References

A Other Stopgap Circumvention Strategies

Use a non-TCP transport protocol. As introduced in Section 4.4, UDP traffic does not trigger blocking. Currently, one can circumvent censorship by simply switching to (or tunneling over) UDP or QUIC. This is merely a stopgap measure, as the censor can enable their censorship for UDP.

Base64-encode the first packet. Recall that the GFW does not censor connections if more than 50% of the first packet’s bytes are printable ASCII. One straightforward way to satisfy this property would be to simply base64-encode all of the encrypted traffic. This, too, is only a stopgap solution; base64-encoded data is easy to detect, and the censor could simply base64-decode and then apply its rules. Although it is effective against the GFW today, we do not consider it as a long-term solution.

More than 20 contiguous bytes of printable ASCII. The GFW exempts connections if the first packet has more than 20 contiguous bytes of printable ASCII. One way to satisfy this is to base64-encode only a small portion of the fully-encrypted packet—or even just insert at least 21 printable ASCII characters into the ciphertext. While we believe this would be more difficult to detect than base64-encoded the entire packet, it also strikes us as a short-term stopgap.

All of the above countermeasures can be implemented on the client-side only, without requiring support from the proxy server. This is possible by applying an idea from prior work [12]: sending a packet such as the ones described above that gets processed by the censor but not by the proxy. For instance, prior to sending the actual first packet of the connection, the client could send a packet that satisfies one of the above rules but that has a broken checksum (which the censor will not check, but the proxy will) or a limited TTL (large enough to reach the censor but not the destination). While these techniques were first verified against Iran’s Protocol Filter, we have verified that these same approaches work against the GFW’s blocking of fully encrypted traffic. Although this provides an encouragingly easy path for deployment, it alone does not elevate these stopgap solutions to longer-term ones.

On September 20, 2021, Apple released iCloud Private Relay (archive) as a new service on iOS 15, iPadOS 15, and macOS Monterey.

Although Apple does not introduce Private Relay as a censorship circumvention tool, in this post, we attempt to understand the potential value of iCloud Private Relay for censorship circumvention. We first introduce how private relay works based on Apple’s documents and our measurement. We then present our empirical observation on its censorship resilience, supported by our measurements in China. As of September 23, 2021, we haven’t found any evidence of censorship against it in China. We also discuss its blocking resistance against common censorship methods, including DNS hijacking, SNI filtering, IP blocking, active probing, as well as self-censorship. Finally, we present some important but unanswered questions about the Private Relay.

We do not intend to make this a comprehensive report. Instead, we hope to start off discussions by presenting our thoughts, observations and measurement methods, encouraging more censorship measurement and circumvention enthusiasts to study it.

Main Findings

• As of September 23, 2021, we haven’t found any evidence of censorship against Private Relay in China.
• The Private Relay can be easily block by common censorship methods, including DNS hijacking, (QUIC) SNI filtering, IP blocking. It may be possible to block by active probing as well.
• The service has been self-censored in many countries, though it is reportedly still usable with a foreign iCloud account.

Introduction

Below is an introduction based on our measurement and our understanding of Apple’s documents. In summary, the Private Relay has a two-hop structure, consisting of an ingress relay and an egress relay:

  ------------
 |DNS resolver|
  ------------
       ^
       |
    A mask.icloud.com?
HTTPS mask.icloud.com?
       |
       0
       |
    ------           -------------           ------------           -------
   |client| <==1==> |ingress relay| <==2==> |egress relay| <==3==> |website|
    ------           -------------           ------------           -------

• Step 0: The client sends two plaintext queries of type A and HTTPS for mask.icloud.com or mask-api.icloud.com to a DNS resolver, asking for the IP addresses of ingress relays.
• Step 1: The client then selects one IP address from the answers and initiates an encrypted QUIC connection to its port 443.
• Step 2: According to the document, “[t]he second relay, which is operated by a third-party content provider, generates a temporary IP address, decrypts the name of the website you requested and connects you to the site”.
• Step 3: The traffic between the egress relays and websites is exactly the same as traffic between clients and websites when no Private Relay is used.

Capture traffic between an iPhone and relays

To capture and analyze the traffic from a mobile device, one intuitive way is to set up a VPN that works at the network layer, tunneling all the traffic at the transport layer and above to a (local) server, where tcpdump or wireshark can be run. However, the iCloud Private Relay feature appears to be disabled when a VPN is used.

As an alternative, we set up a WiFi hotspot from the desktop and let the iPhone connect to it. We then captured and analyzed the traffic from the laptop. Below is the script we used to setup the hotspot, which was borrowed from this tutorial.

#!/bin/bash

set -x
set -e

## Source: https://computingforgeeks.com/create-wi-fi-hotspot-on-ubuntu-debian-fedora-centos-arch/

## Change the IFNAME to your Wi-Fi network interface: `ip link show`
IFNAME="wlp4s0"
CON_NAME="MyHotSpot"
PASSWORD="77fdda98a6feaf6cc9"

nmcli con add type wifi ifname "$IFNAME" con-name "$CON_NAME" autoconnect yes ssid "$CON_NAME"

nmcli con modify "$CON_NAME" 802-11-wireless.mode ap 802-11-wireless.band bg ipv4.method shared

nmcli con modify "$CON_NAME" wifi-sec.key-mgmt wpa-psk
nmcli con modify myhotspot wifi-sec.psk "$PASSWORD"

nmcli connection show "$CON_NAME"

nmcli con up "$CON_NAME"

nmcli connection show "$CON_NAME"

When observing the DNS and initial QUIC traffic, we find the following Wireshark filter helpful:

quic.long.packet_type == 0 or udp.port == 53

Measure current censorship and evaluate potential censorship cost

In this section, we measure current censorship in China, and discuss the cost for a censor to detect and block Private Relay using commonly used censorship methods.

DNS hijacking

As introduced above, the client needs to get an IP address of the ingress relay before initiating QUIC connections to it. Since these DNS queries are (possibly intentionally) sent in plaintext, it is vulnerable to the DNS poisoning attack. Actually, Apple suggests DNS hijacking as “[t]he fastest and most reliable way” to block Private Relay:

The fastest and most reliable way to alert users is to return a negative answer from your network’s DNS resolver, preventing DNS resolution for the following hostnames used by Private Relay traffic. Avoid causing DNS resolution timeouts or silently dropping IP packets sent to the Private Relay server, as this can lead to delays on client devices.

mask.icloud.com

mask-h2.icloud.com

In practice, we observed two ways for the client to get an IP address of the resolvers. The first way is:

1. The client first sends two DNS queries of type A and HTTPS for mask.icloud.com. The responses include a type CNAME answer mask.apple-dns.net, along with many type A answers.
2. The client appears to select the first answer in the reponses, which is the CNAME one. The client thus has to send another two DNS queries of type A and HTTPS for mask.apple-dns.net.
3. The client will then select the first answer in the responses.

The second way is:

1. The client first sent two DNS queries of type A and HTTPS for mask-api.icloud.com. The responses include a type CNAME answer mask-api.fe.apple-dns.net, along with many type A answers.
2. The client appears to select the first answer in the reponses, which is the CNAME one. The client thus has to send another two DNS queries of type A and HTTPS for mask-api.fe.apple-dns.net.
3. The client will then select the first answer in the responses.

Note that we did not observe any query of the documented mask-h2.icloud.com. This observation aligns with the finding in this post.

Measuring current DNS censorship in China

Although it is trivial for the GFW to poison the domains mentioned above, we have not been able to observe any DNS poisoning against these domains yet. Specifically, we tested by sending DNS queries from China to the outside, and also from outside to China. You can also test it yourself from the outside of China, exploiting the bi-directional characterisitc of the GFW. It is worth noting that dig does not support type HTTPS queries yet; be careful that dig will fall back to type A queries without a blocking warning for a query like this: dig @1.1.1.1 mask.icloud.com -t HTTPS +timeout=2.

We thus used the follwing script with Scapy. Since 104.193.82.0 is a Chinese IP address without a DNS service running, we would have received responses injected by the GFW if any of the queries were censored.

#!/usr/bin/env python3

# This script is only reponsible for sending DNS queries, but not for receiving responses.
# To observe DNS responses, use tcpdump or wireshark. eg. :
# sudo tcpdump host 104.193.82.0

from scapy.all import *

# https://www.ietf.org/archive/id/draft-ietf-dnsop-svcb-https-07.html#name-the-svcb-record-type
TYPE_HTTPS=65

CHINESE_IP="104.193.82.0"

for qname in ["mask.icloud.com",
              "mask-api.icloud.com",
              "mask.apple-dns.net",
              "mask-api.fe.apple-dns.net",
              "mask-h2.icloud.com"]:
    for qtype in [TYPE_HTTPS, "A", "AAAA"]:
        send(IP(dst=CHINESE_IP)/UDP(dport=53)/DNS(rd=1, qd=DNSQR(qname=qname, qtype=qtype)))

SNI filtering

As explained in this answer, although clienthello messages are encryped in QUIC, the secrets are derived from a fixed salt and the Destination Connection ID field. The Initial packets can thus be easily decrypted. Actually, a newer version of the Wireshark can automate the decryption process for you.

It is therefore possible for censor to decrypt the Inital packets and check the SNI field against mask.icloud.com.

Measuring current (QUIC-)SNI censorship in China

We tested by capturing the initial packets from client and replaying them on a Chinese server. The server responded with QUIC handshake messages and we observed no disruption.

For example, we first saved the following hex stream as quic.hex:

c80000000108bec8eac6d55fe88a08e87fe5dfa21d247700452c6a2a855275bb191ffe213c2ad1e07467f9ed24956172c4bee69e8446049a94fbae38973cf11ce80cc1379237e4a0f610ae2408ac096635b3978dcf21b4c81d96a2e53d9a9b04dc234341869f7ea85dc99e2ea028827257c4b6993a29ae07e9368c22426d1780abcf8c4b5ab8b2e3ba4de878306ecf4a4e5851c2168b8412f9a55fa5971520914f13c4a86106161e19bfa1eff9c08a9b566e1656ebceeb7184d60a0328203e5e66fc16ad8452343dee5e2ad22ba0ef80b978ba62c64ac75826b79d119c5a7bf9859655d79116e3f4069e87269bab7f9d0373d8e98e4c4891eaf621ce073c61f59eeddd828c96d785ff3155083f5ac93263e5496a6a38a2a2e0bbf64041e76a500bf4748143f2b8705c3732dc12b218f428eeadd02c50e71c5ffaa1ca14c483ac44c75d10e98d38ddaa38f38c0ba7af20108967541586c51bccd2765781b123b4a91fed0f32f0b11bfc4ff5beaccb023f7d977787a0d09942f5159da772b9ca5a7c512a8644bb399858cc6ee2a5d5c099be6780a619cbadc6407db320d34179bf1ff94401ef0e134d0d8ae705a468b5dcd7b9c078e72ddba146e947dca7d4968b3fc892e425ef60bec05df120b20f26f340ed134b064b4fddd194fee666ff49c943b82f812c6f57daaa70ef5aa7b511e8d9501a447783a4e7eee709499161c4055941d516f16bc4ed114d90f6d49c1a297484749fa99f84eb309c2743018eeebb71c6050061e4899b94ecc746fb98174ce383a9d250f61d3aca4db9249122763ca03c41a67b616e722f5171e34a610aeb9cfb6c74f8bdd549d1b0fbd4a766dd66dc355de7f55d55e029d495687c149d9bac0eba89276a0c8048a97545c08597ea836917ceaebe2e334d9376f3c3dc29ee6df84508558d2c77a1139907aba7735945846e3a8c4675845e01c7e2cb1370b31221f95e1bd0c8e5ecc9e86bf5658859379e3c752e34d6bf9e0e9481cf9ed5df79b9c756cb904603eaab2478b6aaa5740b28213f2716b2b4769e21d9c7e2d62e9708de9644a3de048745f079717e0a565475d0684be9cb13c261f55832953c37078cde29894b2176eab5157e4262dbba7919ef2c66d0cf86d7de93059e9f013e2e82544dc803dea878e184d248065454c65c26d8c67b7778e229390de7560815e6cdc53cce1fb11d62d9b0ecea890b4310ffcf7cd544ca8d6a1b9eed7b92a93fd6c00d0a2338f66ad77c9220c69437b3651b18899c68a8e59f12dc2f014d70a6ca5b4aa419516fde01079a1f76c3198db4f6229641e5e89b1b6aa9797c27b55f439e98858f9d3eef1ffff6f5e52e9e94468d21e8ab965abb864836be07016fdbb63a24e954b863a98d590033bd163df6a7740d256a0bbaa910e45a8f40877b6b84fd2d8f57604d236e4351bd228dc707fe3538440b2796dbab58183f306912c6104d13ea96c649fd338994b4a2d5ecbfdd66b69b5245763371cc38c92774723f546a27519db4660f5dc6312f5f56edad2dcb77bd3034c8a4a084ee7e57016fea8a5fcb114ee5ae97d55b177dd8b1ccd0508fb6baee6244ccedf2705ba35a760b944acb4b3e0394b5add44e851d18e0400d99b4910cd4cb63311727f4a98289ce4ee960c506b72243fde14cf5d3185cc4b598f080faf9ebc75847dc7126bb90c47368c5408898e7bdaf9cde4f04299043600dcdf850c306c737d4be37c316eed63718804e9972f6c95d79771ada173293b06037f1282f4e79f8116e3d4c5fed2ec6db335faf2b0481b3aa3a0192f9ff3fea35ae1bafe71bbcd07a301fe11638a180b6b202c29dea331ac6a2527587a82175cd7b96033b165b88580e83df7759ebe6586d68d4efe6028403d5d0d700e967ca4908bbd8e4

We then send it to the ingress relay and received a response:

xxd -r -p quic.hex | nc -u mask.icloud.com 443 -v

Note that the payload in the example will not get a reponse from an ingress relay anymore, approximately two days after it was first generated. One can probably still trigger reponses from the ingress relays with freshly generated Initial packets.

(Quic-)TLS fingerprint filtering

As explained above, the clienthello messages in the Initial packets can be easily decrypted. It is thus possible for the censor to conduct censorship based on the TLS fingerprint.

Our observation of the TLS fingerprint of the Private Relay aligns with the findings in this report:

The connection to the relay uses QUIC to port 443/UDP and TLS 1.3. The clienthello includes the server name extension and the server name “mask.icloud.com.” Only 3 cipher suites are offered (TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256). The server ends up selecting the AES128 suite. Application Layer Protocol Negotiation (ALPN) is also used, with unsurprisingly HTTP/3 being the only option.

Note that, apart from the 3 cipher suites, we saw a forth Grease ciphersuit (0x2a2a).

As a side note, we also observed two GREASE extensions 0xAAAA and 0X3A3A. They probably do not have anything to do with authentication (if they do, it’s something very non-standard Apple is doing). The GREASE extensions turned out to be not uncommon; web browsers send them as well. As expalined in this doc, they were used to “reserve a set of TLS protocol values that may be advertised to ensure peers correctly handle unknown values”. In other words, GREASE is meant to provide automated diversity to protocol fields, to prevent the protocol from “rusting shut” by assuming only certain values may appear.

We are also curious, by any chance, tlsfingerprint.io can tell us how unique these (or any) (QUIC) ClientHello fingerprints are? (@sergeyfrolov, @ewust)

Active probing to ingress relay

We find that the ingress relay will respond to a replay of the Initial packets within approximately two days since the packets were generated. We also tried to use quic-go and curl --http3 to send a typical Quic with SNI=mask.apple.com to the ingress relay; however, the ingress relay did not respond anything in this case. We suspected that has something to do with the ALPN extension included in the clienthello sent by the legitimate clients. It may also be possible that the client traffic contains some other authenticators.

IP blocking to ingress relays

As introduced above, we could still receive responses from ingress relays by sending Initial packets from China. This indicates that, at least for the IP addresses we tested, China has not blocked it yet.

However, it could be fairly easy to block the ingress relay IP in serveral ways:

1. block all IP addresses to which mask.icloud.com, mask-api.icloud.com and mask-h2.icloud.com resolve.
2. observe QUIC connections with SNI=mask.apple.com and confirm it is indeed an ingress relay using the active probing approach mentioned above. Then block the corresponding IP addresses.

IP-based discrimination against egress relays

Similar to Tor exit relays, which are available publicly, Apple provides an up-to-date lists of egress IP ranges (archive). This list could be easily used by websites to discriminate against Private Relay users, like what Tor users have been suffering from.

Questions not answered yet

How Apple implements the self censorship

Apart from all the possible external censorship methods discussed above, Apple has been conducting self-censorship to prevent users in heavily censored areas from using the Private Relays. It is thus important to understand how Apple implements the self censorship in order to circumvent it.

Specifically, Apple admitted that:

Private Relay isn’t available in all countries and regions. If you travel somewhere Private Relay isn’t available, it will automatically turn off and will turn on again when you re-enter a country or region that supports it. Private Relay will notify you when it’s unavailable and when it’s active again.

According to many news sources, these countries include China, Belarus, Colombia, Egypt, Kazakhstan, Saudi Arabia, South Africa, Turkmenistan, Uganda, the Philippines, and Russia.

It remains an unanswered questions on how and what self-censorship has been implemented by Apple. From our testing, it seems that the ingress server does not refuse service based on the geolocation of client IP addresses. However, it is still unclear to us how Apples determines the location of the user and thus refuse to be activated.

One report claimed (archive) that Apple learned users’ geo-location from users’ IP addresses connected to its certain servers; proxying traffic to these certain servers will activate the Private Relay service.

Another user report claimed (archive) that it is sufficient to activate Private Relay by changing the iCloud region to ones where Private Relay is not self-censored. However, another user in the post failed to activate the service with the same settings.

We note that it is not uncommon for a Chinese iOS circumventor to have a non-Chinese iCloud account. This is because, due to the heavy censorship against circumvention tools in Chinese app stores, it is almost essential to have a non-Chinese iCloud account to install other circumvention tools.

How does the user authentication work?

Apple claims that:

Private Relay validates that the client connecting is an iPhone, iPad, or Mac, so you can be assured that connections are coming from an Apple device.

All connections that use Private Relay validate that the client is an iPhone, iPad, or Mac and that the customer has a valid iCloud+ subscription. Private Relay enforces several anti-abuse and anti-fraud techniques, such as single-use authentication tokens and rate-limiting.

We are curious how (or if) Apple authenticates Priavet Relay users on the ingress and egress relays.

How does the underlying decryption work?

In the introduction section, we mentioned that the Private Relay “has a two-hop structure”. However, we do not have know anything more about the underlying structure. For example, is an onion-routing structure used? Amir Houmansadr expressed concerns on the intransparency of the underlying protocol. Further investigation is required to better understand the underlying encryption/decryption mechanism.

Acknowledgement

We thank a person who prefers to stay anonymous for lending us an iPhone for testing.

Contacts

This report first appeared on Net4People. We also maintain an up-to-date copy of the report on GFW Report and ntc.party.

We encourage you to share your comments publicly or privately. Our private contact information can be found at the footer of GFW Report.

On September 20, 2021, Apple released iCloud Private Relay (archive), a new capability embedded into iOS 15, iPadOS 15, and macOS Monterey. Its objective is to enhance the privacy and security of Apple users who surf the web in Apple’s Safari browser. This comes as an exciting news to privacy advocates, especially given Apple’s controversial plans for inspecting iCloud photos, a decision that has caused outrage in the privacy community.

In this report, we present an early analysis of iCloud Private Relay, aiming to validate the claims made by Apple on how this new feature can enhance user privacy. The report will get updated as we expand our findings.

How iCloud Private Relay Works: Apple’s Explanation

Apple has offered some high-level insights on the architecture of iCloud Private Relay (archive), but, unfortunately, many details are missing on its technical design. The following excerpts are the only technical details we could find officially from Apple (we have highlighted important pieces):

iCloud Private Relay is a new internet privacy service offered as a part of an iCloud+ subscription that allows users on iOS 15, iPadOS 15, and macOS Monterey to connect to and browse the web more privately and securely. Private Relay protects users’ web browsing in Safari, DNS resolution queries, and insecure http app traffic. Internet connections set up through Private Relay use anonymous IP addresses that map to the region a user is in, without divulging the user’s exact location or identity.

The iCloud Private Relay service uses an innovative multi-hop architecture in which users’ requests are sent through two separate internet relays operated by different entities. This way, no single party — including Apple — can view or collect the details of users’ browsing activity. Private Relay validates that the client connecting is an iPhone, iPad, or Mac, so you can be assured that connections are coming from an Apple device. Private Relay replaces the user’s original IP address with one assigned from the range of IP addresses used by the service. The assigned relay IP address may be shared among more than one Private Relay user in the same area. The relay IP address presented to networks and web servers accurately represents the client’s coarse city-level location by default, allowing your network to receive relevant location information when attempting to enforce geo-based restrictions based on IP address.

When Private Relay is enabled, your requests are sent through two separate, secure internet relays. Your IP address is visible to your network provider and to the first relay, which is operated by Apple. Your DNS records are encrypted, so neither party can see the address of the website you’re trying to visit. The second relay, which is operated by a third-party content provider, generates a temporary IP address, decrypts the name of the website you requested and connects you to the site. All of this is done using the latest internet standards to maintain a high-performance browsing experience while protecting your privacy.

Based on the above excerpts from Apple, and also through experiments in a lab setting using Apple devices, it appears that a Private Relay connection has the following format:

   |Safari (an apple user)| <==1==> |ingress relay (a proxy controlled by Apple)| <==2==> |egress relay (a third-party content provider, e.g., Akamai)| <==3==> |website|

Our Immediate Questions about Private Relays

Apple’s concise explanation of Private Relays leaves many questions lingering about the technical design of Private Relays, that can directly impact its privacy promises. In particular:

1. The protocol is mostly unknown! What is the “innovative multi-hop architecture” that Apple claims to be using? Is it borrowing ideas from the classic onion routing protocol (which is used by Tor), or has Apple come up with another architecture?

2. Does it really anonymize your IPs, as claimed by Apple? Apple claims that Private Relays anonymize the IP addresses of the users. How are they defining anonymity? By the metrics long-established in the privacy community, or perhaps based on their own definition of anonymity? Apple explains that “your requests are sent through two separate, secure internet relays.” So, if Apple’s “innovative multi-hop architecture” is just a 2-hop onion routing system, then by all means this is completely broken when it comes to anonymizing IP addresses. This is because, a 2-hop onion encryption can not provide by-design anonymity, as the two hops can decide to disclose your identity once needed. That is, at any time the third-party content provider (e.g., Akamai) can work with Apple to de-anonymize a misbehaving Safari client who is connecting through Private Relays. So, it appears that Apple is only promising you to not look into your connections.

3. What about other non-IP means of identification? Seems like Private Relays aim to anonymize IP addresses only. Ask a first year graduate student working on privacy, and they will enumerate a plethora of techniques other than IP addresses that can give away your online identity, e.g., cookies, various forms of tracking, and side channels. So, one wonders, is Apple doing anything about those other features? Protecting just the IP address and ignoring other forms of identification will only give users an illusion of privacy! (Actually, seems like people have already found some vulnerability that leaks the IP addresses of Private Relay connections, using a known WebRTC side channel!)

Concluding Thoughts

• Apple has only offered some very high-level information on the architecture of iCloud Private Relay, but critical technical details are left undisclosed, in particular, the crypto techniques used to authenticate relays, whether and how traffic is encrypted between the relays, how exactly are the ingress and egress relays selected for each connection, and so on. Apple seems to be ignoring the most fundamental principle of information assurance, Kerckhoffs’s principle, that states that security by obscurity is doomed to fail! We hope Apple will release extensive technical details of its iCloud Private Relay as well as open source its code so it can be analyzed and inspected by security and privacy researchers and practitionors.
• Despite the missing details from Apple on the technical design of Private Relays, the provided description by Apple leads us to conclude that private relays do not provide privacy by-design, but privacy by-decision. That is, there’s nothing technical preventing Apple and the third-party content providers (like Akamai) to work together to de-anonymize a misbehaving “anonymized” connection, instead they have only decided to respect your privacy (perhaps only untill you start misbehaving).
• It appears that Apple is only targetting a single aspect of anonymity, namely IP addresses. Your network traffic can leak your identity/privacy in many ways other than your IP address (e.g. cookies, various trackers, side channels), but it seems like all other facets are swept under the rug. In fact, this is the very reason that the Tor Browser exists: a specialized browser designed by Tor to complement its (3-hop) onion routing technique by taking care of the non-IP means of privacy leakage. Therefore, we are worried that Private Relays are likely to give average users only a falsified perception of privacy, leading them towards radical actions (like browsing unsafe websites) that they would not take if it was not for such privacy illusion.
• Don’t get me wrong! I truly admire Apple for taking steps towards improving their users’ privacy. But, they might as well want to do it right!

Acknowledgements

We thank David Fifield and many other people for giving us feedback.

Related Resources

• About iCloud Private Relay
• Prepare Your Network or Web Server for iCloud Private Relay
• The list of egress relays (archive)
• A First Look at Apple’s iOS 15 “Private Relay” feature
• iOS 15 iCloud Private Relay Vulnerability Identified

苹果公司于2021年9月20日，发布了一项名为iCloud Private Relay (archive)的新服务，包含在iOS 15， iPadOS 15和macOS Monterey中。

尽管苹果公司没有将它的翻墙功能作为卖点，在这篇报告中，我们试图理解iCloud Private Relay的翻墙价值。首先，基于我们的测量和对苹果文档的理解，我们介绍Private Relay的工作原理。接着我们通过在中国进行的测量实验实证性地评估Private Relay的抗封锁能力。截止2021年9月23日，我们尚未发现Private Relay被防火长城审查的迹象。我们还将讨论Private Relay面对常见的审查方式（如DNS劫持，SNI过滤，IP封锁，主动探测，和自我审查）时的抗封锁能力。最后，我们将提出一些关于Private Relay的重要但还未解决的问题。

我们无意将这篇报告作得面面俱到。而仅想抛砖引玉地介绍我们的测量方法、观察及想法。我们鼓励更多的互联网审查爱好者做更深入的研究。

主要发现

• 截止2021年9月23日，我们尚未发现Private Relay被防火长城审查的迹象
• Private Relay可以很容易地被常见的审查方式封锁，包括DNS劫持，SNI过滤，IP封锁，主动探测，和自我审查。主动探测Priavte Relay服务器也许也是可行的。
• Private Relay这项服务已被苹果在包括中国在内的许多国家和地区自我审查。但用户报告只需使用相应的国外iCloud账户即可绕过禁用。

介绍

以下介绍基于我们的测量和对苹果的文档的理解。简而言之，Private Relay采用两跳结构，由一个入口节点和一个出口节点组成：

  ------------
 | DNS 服务器  |
  ------------
       ^
       |
    A mask.icloud.com?
HTTPS mask.icloud.com?
       |
       0
       |
    ------           -------------           ------------           ------
   |客户端 | <==1==> |   入口节点   | <==2==> |   出口节点   | <==3==> |目标网站|
    ------           -------------           ------------           ------

• 第0步： 客户端发送两个明文的DNS请求到DNS服务器。请求的类型为A和HTTPS，请求的域名为mask.icloud.com或mask-api.icloud.com。目的是得到入口节点的IP地址。
• 第1步： 客户端选取其中一个DNS服务器返回的IP地址，并发送QUIC初始包到入口节点的443端口。
• 第2步： 根据文档，“出口节点由第三方运行，会生成一个临时IP地址，解密得到请求的目标网站地址，并与网站进行连接”。
• 第3步： 出口节点和目标网站间的流量与，不启用Piravte Relay时，客户端和目标网站间的流量完全相同。

捕获iPhone与入口节点间的流量

一种自然想到的捕获和分析移动设备流量的方式是，在手机上建立起工作在网络层的VPN，把所有传输层及更上层的流量都转发到一个（本地的）服务器上，再在服务器上运行tcpdump或者wireshark。然而我们发现在VPN打开的状态下，iCloud Private Relay是无法启用的。

作为替代方式，我们在电脑上建立起无线热点，并让iPhone连上去。我们这样就可以在电脑上捕获并分析流量了。我们用了以下脚本搭建无线热点，脚本借用了这个教程里的知识。

#!/bin/bash

set -x
set -e

## Source: https://computingforgeeks.com/create-wi-fi-hotspot-on-ubuntu-debian-fedora-centos-arch/

## 记得将IFNAME替换成你的Wi-Fi network interface的名字: `ip link show`
IFNAME="wlp4s0"
CON_NAME="MyHotSpot"
PASSWORD="77fdda98a6feaf6cc9"

nmcli con add type wifi ifname "$IFNAME" con-name "$CON_NAME" autoconnect yes ssid "$CON_NAME"

nmcli con modify "$CON_NAME" 802-11-wireless.mode ap 802-11-wireless.band bg ipv4.method shared

nmcli con modify "$CON_NAME" wifi-sec.key-mgmt wpa-psk
nmcli con modify myhotspot wifi-sec.psk "$PASSWORD"

nmcli connection show "$CON_NAME"

nmcli con up "$CON_NAME"

nmcli connection show "$CON_NAME"

在观察DNS和初始的QUIC流量时，我们发现以下过虑条件很好用：

quic.long.packet_type == 0 or udp.port == 53

测量当前的审查并评估潜在的审查成本

在这一节中，我们将测量中国当前对于Pirvate Relay的审查，并讨论审查者采用常用审查方法检测并封锁Private Relay的成本。

DNS劫持

前面提到客户端会通过DNS查询得到一个入口节点的IP地址，然后用QUIC协议与其建立连接。因为这些DNS请求（很有可能被故意设计成）是明文的，因此容易受到DNS劫持攻击。事实上，苹果公司自己就建议使用DNS劫持来“最快速和稳定”地封锁Private Relay：

The fastest and most reliable way to alert users is to return a negative answer from your network’s DNS resolver, preventing DNS resolution for the following hostnames used by Private Relay traffic. Avoid causing DNS resolution timeouts or silently dropping IP packets sent to the Private Relay server, as this can lead to delays on client devices.

mask.icloud.com

mask-h2.icloud.com

我们观察到客户端有两种获得入口节点地址的方式。第一种方式是：

1. 客户端首先发送两个DNS查询，查询类型为A和HTTPS，查询的域名为mask.icloud.com。DNS应答包含一个CNAME答案mask.apple-dns.net，以及多个A答案。
2. 客户端选择DNS应答包中的第一个答案，即那个CNAME答案。客户端因此需要再次发送两个DNS查询，查询类型还是A和HTTPS，查询的域名为mask.apple-dns.net。
3. 客户端接着还会选取DNS应答包中的第一个答案，这次是A答案。

第二种方式是：

1. 客户端首先发送两个DNS请求，查询类型为A和HTTPS，查询的域名为mask-api.icloud.com。DNS应答包含一个CNAME答案mask-api.fe.apple-dns.net，以及多个A答案。
2. 客户端选择DNS应答包中的第一个答案，即那个CNAME答案。客户端因此需要再次发送两个DNS查询，查询类型还是A和HTTPS，查询的域名为mask-api.fe.apple-dns.net。
3. 客户端接着还会选取DNS应答包中的第一个答案，这次是A答案。

我们没有观察到客户端会查询文档中记录的mask-h2.icloud.com。这篇报告也提到没有观察到包含mask-h2.icloud.com的DNS查询。

测量中国当下的DNS审查

虽然污染上述提到的域名对GFW来说小菜一碟，但是我们还并未观察到GFW真的采取审查。具体而言，我们从中国发送上述DNS请求到国外，也从国外发送请求回中国，来让GFW的设备看到我们的请求。你即使不在中国，也同样可以利用GFW不区分DNS来源于国内或国外的特性，来测量DNS污染。需要注意的是，dig命令尚不支持发送HTTPS类型的DNS请求。如果使用dig @1.1.1.1 mask.icloud.com -t HTTPS +timeout=2这样的请求，它会自动回落到发送A类型请求。因为回落警告不足够明显，大家要当心回落带来的测量失误。

我们因此在从国外发DNS请求往国内时，使用了以下脚本，调用Scapy发送DNS请求。这里我们用到的104.193.82.0是一个中国的IP地址，而且这个中国的IP地址没有运行DNS服务，这样如果我们收到了任何DNS答复，那么一定是GFW或其他中间人伪造的。我们因此也就可以判断GFW是否审查了相应的域名。

#!/usr/bin/env python3

# 这个脚本只负责发送DNS请求，不负责接收DNS应答。
# 如果想观察DNS应答，请使用tcpdump或者wireshark。比如：
# sudo tcpdump host 104.193.82.0

from scapy.all import *

# https://www.ietf.org/archive/id/draft-ietf-dnsop-svcb-https-07.html#name-the-svcb-record-type
TYPE_HTTPS=65

CHINESE_IP="104.193.82.0"

for qname in ["mask.icloud.com",
              "mask-api.icloud.com",
              "mask.apple-dns.net",
              "mask-api.fe.apple-dns.net",
              "mask-h2.icloud.com"]:
    for qtype in [TYPE_HTTPS, "A", "AAAA"]:
        send(IP(dst=CHINESE_IP)/UDP(dport=53)/DNS(rd=1, qd=DNSQR(qname=qname, qtype=qtype)))

SNI过滤

正如这个答案所介绍的，虽然QUIC中的Clienthello消息是加密的，但密钥是由固定的salt和明文的Destination Connection ID导出的。QUIC的初始包因此也就可以被很容易的解密。事实上，新版的Wireshark就可以自动地解密客户端发出的QUIC初始包。

审查者也因此可以加密QUIC初始包并检查其中的SNI值是否为mask.icloud.com。

测量当下中国的(QUIC-)SNI审查

我们测试的方法是抓取客户端发送的初始包，然后在中国的服务器上重放。我们观察到服务器会回以QUIC握手包。我们还没有观察到审查者阻断这一过程。

举例来讲我们首先将以下十六进制流存入名为quic.hex的文件:

c80000000108bec8eac6d55fe88a08e87fe5dfa21d247700452c6a2a855275bb191ffe213c2ad1e07467f9ed24956172c4bee69e8446049a94fbae38973cf11ce80cc1379237e4a0f610ae2408ac096635b3978dcf21b4c81d96a2e53d9a9b04dc234341869f7ea85dc99e2ea028827257c4b6993a29ae07e9368c22426d1780abcf8c4b5ab8b2e3ba4de878306ecf4a4e5851c2168b8412f9a55fa5971520914f13c4a86106161e19bfa1eff9c08a9b566e1656ebceeb7184d60a0328203e5e66fc16ad8452343dee5e2ad22ba0ef80b978ba62c64ac75826b79d119c5a7bf9859655d79116e3f4069e87269bab7f9d0373d8e98e4c4891eaf621ce073c61f59eeddd828c96d785ff3155083f5ac93263e5496a6a38a2a2e0bbf64041e76a500bf4748143f2b8705c3732dc12b218f428eeadd02c50e71c5ffaa1ca14c483ac44c75d10e98d38ddaa38f38c0ba7af20108967541586c51bccd2765781b123b4a91fed0f32f0b11bfc4ff5beaccb023f7d977787a0d09942f5159da772b9ca5a7c512a8644bb399858cc6ee2a5d5c099be6780a619cbadc6407db320d34179bf1ff94401ef0e134d0d8ae705a468b5dcd7b9c078e72ddba146e947dca7d4968b3fc892e425ef60bec05df120b20f26f340ed134b064b4fddd194fee666ff49c943b82f812c6f57daaa70ef5aa7b511e8d9501a447783a4e7eee709499161c4055941d516f16bc4ed114d90f6d49c1a297484749fa99f84eb309c2743018eeebb71c6050061e4899b94ecc746fb98174ce383a9d250f61d3aca4db9249122763ca03c41a67b616e722f5171e34a610aeb9cfb6c74f8bdd549d1b0fbd4a766dd66dc355de7f55d55e029d495687c149d9bac0eba89276a0c8048a97545c08597ea836917ceaebe2e334d9376f3c3dc29ee6df84508558d2c77a1139907aba7735945846e3a8c4675845e01c7e2cb1370b31221f95e1bd0c8e5ecc9e86bf5658859379e3c752e34d6bf9e0e9481cf9ed5df79b9c756cb904603eaab2478b6aaa5740b28213f2716b2b4769e21d9c7e2d62e9708de9644a3de048745f079717e0a565475d0684be9cb13c261f55832953c37078cde29894b2176eab5157e4262dbba7919ef2c66d0cf86d7de93059e9f013e2e82544dc803dea878e184d248065454c65c26d8c67b7778e229390de7560815e6cdc53cce1fb11d62d9b0ecea890b4310ffcf7cd544ca8d6a1b9eed7b92a93fd6c00d0a2338f66ad77c9220c69437b3651b18899c68a8e59f12dc2f014d70a6ca5b4aa419516fde01079a1f76c3198db4f6229641e5e89b1b6aa9797c27b55f439e98858f9d3eef1ffff6f5e52e9e94468d21e8ab965abb864836be07016fdbb63a24e954b863a98d590033bd163df6a7740d256a0bbaa910e45a8f40877b6b84fd2d8f57604d236e4351bd228dc707fe3538440b2796dbab58183f306912c6104d13ea96c649fd338994b4a2d5ecbfdd66b69b5245763371cc38c92774723f546a27519db4660f5dc6312f5f56edad2dcb77bd3034c8a4a084ee7e57016fea8a5fcb114ee5ae97d55b177dd8b1ccd0508fb6baee6244ccedf2705ba35a760b944acb4b3e0394b5add44e851d18e0400d99b4910cd4cb63311727f4a98289ce4ee960c506b72243fde14cf5d3185cc4b598f080faf9ebc75847dc7126bb90c47368c5408898e7bdaf9cde4f04299043600dcdf850c306c737d4be37c316eed63718804e9972f6c95d79771ada173293b06037f1282f4e79f8116e3d4c5fed2ec6db335faf2b0481b3aa3a0192f9ff3fea35ae1bafe71bbcd07a301fe11638a180b6b202c29dea331ac6a2527587a82175cd7b96033b165b88580e83df7759ebe6586d68d4efe6028403d5d0d700e967ca4908bbd8e4

我们再将其发送到入口节点，并得到了回应:

xxd -r -p quic.hex | nc -u mask.icloud.com 443 -v

注意从生成这个十六进制荷载的大约两天后，发送它已经不能再能得到入口节点的回应了。如果重放刚生成的初始包，应该还可以触发入口节点的回应。

(Quic-)TLS指纹过滤

如上所述，QUIC初始包可以被很容易的解密得到ClientHello。这就给了审查者采用基于TLS指纹的审查的可乘之机。

我们对Private Relay的TLS指纹的观察于这篇报告基本一致：

The connection to the relay uses QUIC to port 443/UDP and TLS 1.3. The clienthello includes the server name extension and the server name “mask.icloud.com.” Only 3 cipher suites are offered (TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256). The server ends up selecting the AES128 suite. Application Layer Protocol Negotiation (ALPN) is also used, with unsurprisingly HTTP/3 being the only option.

除了引文中提到的3种密码套件外，我们还观察到了第四种Grease ciphersuit (0x2a2a)。

作为旁注，我们还观察到了ClientHello中包含了两个GREASE extensions：0xAAAA和0X3A3A。他们不太可能是被用于验证相关的目的（如果真的是被用作验证目的，那就是很不服合标准的做法）。GREASE extensions其实并不罕见；浏览器也会发送它们。正如这篇文档所解释的，它们被用来“保证TLS实现可以正确的处理不认识的值”。换而言之，因为GREASE的存在，TLS实现就不能假设只会收到和处理某些特定的值了。

我们很好奇tlsfingerprint.io是否能告诉我们这些（或者任何）(QUIC) ClientHello的指纹有多特殊？(@sergeyfrolov, @ewust)

主动探测入口节点

我们发现在大约两天之内重放QUIC初始包到入口节点，入口节点都会回以发送握手包。我们还尝试使用quic-go和curl --http3对入口节点进行典型的Quic握手，SNI为mask.apple.com。但入口节点并不回应。我们怀疑这与合法客户端发送的ClientHello的ALPN extension有关。但入口节点是否回应也可能还与其他的验证信息有关。

封锁入口节点的IP地址

如前所述，从中国发送的QUIC握手包可以得到入口节点的回应。这说明，至少我们测量的IP地址还没有被封锁。

但是仍有很多方法封锁入口节点的IP地址，比如审查者可以：

1. 封锁所有解析mask.icloud.com，mask-api.icloud.com或 mask-h2.icloud.com时返回的IP地址。
2. 观察QUIC链接中SNI为mask.apple.com的服务器IP地址，然后用主动探测确认入口节点。确认后封锁相应IP地址。

基于出口节点的IP地址歧视用户

如Tor出口节点一样，苹果也有一个实时更新的出口节点IP段列表 (存档)。这个列表可以方便网站基于出口节点的IP地址来歧视Private Relay用户，正如Tor用户所遭受的歧视一样。

还未解决的问题

苹果是如何实现自我审查的

除了上述的种种审查方式外，苹果公司还通过自我审查的方式阻止在本就身处受到严格审查地区的用户使用Private Relay。因此了解并绕过苹果公司的自我审查尤为重要。

具体来讲，苹果公司承认：

Private Relay isn’t available in all countries and regions. If you travel somewhere Private Relay isn’t available, it will automatically turn off and will turn on again when you re-enter a country or region that supports it. Private Relay will notify you when it’s unavailable and when it’s active again.

根据相关的新闻报道， 苹果在以下地区禁用了Private Relay：中国、白俄罗斯、哥伦比亚、埃及、哈萨克斯坦、沙特阿拉伯、南非、土库曼斯坦、乌干达、菲律宾和俄罗斯。

苹果自我审查的实现机制还有待研究。我们的测试显示，入口节点似乎并不基于用户IP地址来拒绝服务。但是我们仍不清楚苹果公司是如何判断用户是否身处被禁止使用Private Relay的国家。

一项报告(存档)声称苹果公司是根据用户访问特定的一组苹果服务器时的IP地址，来判断用户位置的；使用境外代理访问这组苹果服务器即可激活Private Relay功能。

而另一个用户报告(存档)只需将iCloud的地区设置为非禁用地区，就可以启用Private Relay了。但同一个帖子中的另一用户声称使用了非禁用地区iCloud但仍然无法开启Private Relay。我们欢迎中国的用户分享你的经验。

另外作为一点背景介绍，对于中国的iOS翻墙用户，拥有一个非中国iCloud账户并不罕见。这是由中国的App Store对翻墙软件的严格审查导致的。

苹果是如何验证Private Relay用户的？

苹果公司声称：

Private Relay validates that the client connecting is an iPhone, iPad, or Mac, so you can be assured that connections are coming from an Apple device.

All connections that use Private Relay validate that the client is an iPhone, iPad, or Mac and that the customer has a valid iCloud+ subscription. Private Relay enforces several anti-abuse and anti-fraud techniques, such as single-use authentication tokens and rate-limiting.

我们好奇苹果是如何验证Private Relay用户的。

苹果的Private Relay是如何加密解密的？

前面我们介绍Private Relay采用两跳结构。除此之外我们并不了解更多的技术细节。比如说，这两跳是否采用了如onion-routing的结构？Amir Houmansadr表达了对Private Relay协议不透明的关切。Private Relay的协议和工作原理因此有待更多的调查。

致谢

我们感谢一位把iPhone手机借我们测试的人。

联系我们

这篇报告首发于Net4People。我们还在GFW Report和ntc.party同步更新了这篇报告。

我们鼓励您公开地或私下地分享与报告中的发现和假设相关的问题、评论或证据。我们私下的联系方式可见GFW Report的页脚。

在近期的IMC'20的工作中(论文, 演讲)，我们揭示了中国的防火长城采用流量分析与主动探测相结合的手段来检测和封锁Shadowsocks服务器。

在这篇短文中，我们将分别向技术小白和翻墙软件开发者提供防御GFW主动探测的实用建议。 我们还将介绍Len et al.展示的partitioning oracle攻击的缓解办法。 如果在遵循了本文的建议后，你的Shadowsocks服务器仍被封锁，请将封锁情况汇报给GFW Report以及相应的开发者。

给用户的建议

根据我们的测试和来自开发者的报告，在采用了适当的配置后，以下两个Shadowsocks实现的最新版本已经可以抵御来自GFW的主动探测：Shadowsocks-libev和OutlineVPN。

针对Shadowsocks-libev的使用建议

如果你决定使用Shadowsocks-libev，我们强烈建议你根据这篇教程来部署一台抗封锁的Shadowsocks-libev服务器。我们会时刻更新那篇教程的，以应对之后新出现的针对Shadowsocks的识别和攻击。

如果你已经拥有了一台Shadowsocks-libev服务器，你可以根据以下规则来确认你的服务器是否配置得可以对抗GFW的主动检测和封锁。

截止2021年1月，你需要做到以下几点来防止你的Shadowsocks-libev服务器被封锁：

1. 确保你的服务器版本为v3.3.1及以上。你可以通过以下命令查看服务器的版本ss-server -h。
2. 使用AEAD ciphers， 而不用 stream ciphers。换句话说，仅在以下几种加密方式中进行选择：chacha20-ietf-poly1305 (推荐), aes-256-gcm, aes-192-gcm或者aes-128-gcm。

为了缓解针对Shadowsocks的partitioning oracle攻击，你需要:

1. 使用一个长的随机密码。这样的密码可以用以下命令在终端生成： openssl rand -base64 16;
2. 禁用UDP模式。

注意：针对客户端没有特殊的要求，任何与Shadowsocks-libev服务器兼容的客户端均可。

针对OutlineVPN的使用建议

为了防止你的OutlineVPN服务器被GFW封锁，你需要做到以下几点：

1. 使用最新版的从官网下载的服务端。
2. 使用最新版的从官网下载的客户端。

注意：

1. Outline会自动生成一个长的，随机的密码，因此你不必像为Shadowsocks-libev那样手动配置密码。
2. Outline服务端会自动更新，因此你不必手动升级服务端。
3. Outline只采用chacha20-ietf-poly1305这一种AEAD cipher作为加密方式，因此你不必手动选择加密方式。

给翻墙软件开发者的建议

下面我们介绍我们发现的GFW的最新审查能力，并附上我们给翻墙软件开发者的相应建议。这些建议不仅对Shadowsocks，而且对其他许多翻墙软件都有用。 我们欢迎你加入我们的讨论，分享你的想法，评论，疑惑和关切。

正确的校验

首先，我们强烈建议翻墙软件的开发者们彻底废除不具备校验的加密构造。仅仅有保密性是不够的。

• 对于新开发的翻墙软件来说，这意味着根本不应考虑支持不具备校验的加密构造。
• 对于现存的翻墙软件来说，这意味着开发者应该勇敢地移除所有与不具备教研的加密构造相关的代码，即使以不向下兼容为代价。

我们这看似大胆的建议实际上出于合理的原因。如我们在论文中所介绍的， 一些类型的GFW主动探测会利用Shadowsocks的stream ciphers的malleability。这已经不是第一次不具备校验的加密结构造成漏洞了。事实上，不具备校验的加密结构是许多Shadowsocks和其他翻墙软漏洞的根本来源。

早在2015年8月，BreakWa11发现了一个关于Shadowsocks的stream ciphers的漏洞。这个漏洞是由于缺乏数据完整性保护而造成的（英文总结）。 在2020年，类似的漏洞又被发现存在于V2Ray中（总结）。

当Shadowsocks开发者试图引入one time auth模式来缓解2015年的那个漏洞时，另一个因数据长度缺乏完整性保护的主动探测又被引入了（英文总计）。

2020年2月，Zhejiang Peng发现了一个关于Shadowsocks stream ciphers的灾难性的漏洞，(英文总结）。 利用使用了stream cipher的Shadowsocks服务器作为decryption oracle，攻击者可以在不需要密码的情况下，完全解密Shadowsocks会话。

其实早在2017年2月，AEAD ciphers就已经成为了Shadowsocks协议的一部分。而校验问题也理应在那时就被解决了。但实际情况是，截止2021年，大量的服务器仍然因为使用被废弃的stream ciphers而存在着安全隐私漏洞，以及被准确识别的风险。

这样的现象表明，在实际操作中，许多的用户不能够正确的选择加密方式。这可能与使用过时的教程或一键脚本有关。 我们因此鼓励开发者从Shadowsocks各实现中彻底移除stream ciphers，帮助用户做出正确的选择。

使用同时基于nonces和时间的重放过滤器

我们建议翻墙软件的开发者们采用同时基于nonces和时间的重放过滤器。 因为采用基于时间的重放过滤器需要对Shadowsocks协议进行根本性地变动，我们建议开发者至少要此采用基于nonces的过滤器，并且做到：

1. 要么建议用户在每次过滤器重置后修改密码；
2. 要么开发一个机制，可以让重放过滤器在软件重启后依然记得之前的nonces。

这些建议是基于以下的研究发现和推论。 如论文的section 3.5所介绍的， GFW既可以在观察到一个合法连接的瞬间对其进行重放；也可以等待三周甚至更长时间后才重放。 因此，一个更加合理的主动探测模型应该允许审查者在任意时长后对合法连接进行重放。

这样的一个主动探测模型揭示了纯粹基于nonces的重放过滤所须要面对的不对称性。GFW仅用少量资源就可以记录一些合法的连接，并且在经过任意的时长后再重放它们；但与此同时，Shadowsocks需要大量的资源和相对复杂的实现来永久性地记住所有的合法链接，直至密码被更换。 注意，Shadowsocks服务器必须在重启后还记住这些nonces；否则重放过滤器不会过滤基于重启前的合法连接的重放。

幸运的是，这个不公平的局面可以通过同时引入基于时间的重放过滤机制来扭转：服务器只需要处理并验证时间戳未过期的连接，就像VMess服务器那样。 如此一来服务器就不需要永久性地记住所有合法连接中的nonces。

我们还想强调，对于那些仅仅短暂暴露变化的监听端口的翻墙服务器，重放过滤仍是必要的。因为GFW可以瞬时重放合法连接中的第一个数据包，而这时暴露的监听端口因为未完成数据传输，依然是开启的。

让服务器的反应保持一致

我们建议开发者们确保翻墙服务器在正常运行时，和遇到不合法的连接时都反应一致。理想情况下，可以按照Frolov et al.的建议，让服务器遇到错误连接时“read forever”。 这是因为，审查者会故意触发协议的边边角角等特殊情况，来识别服务器指纹。

除了我们在Shadowsocks-libev和OutlineVPN中发现的服务器反应指纹，Frolov et al.还展示了包括Shadowsocks-python和OutlineVPN在内的多种翻墙软件可以通过关闭连接时的TCP flags和连接时长来识别。studentmain报告，直至2020年12月， 许多的Shadowsocks实现仍存在着我们在Shadowsocks-libev和Outline中发现的问题。

Frolov et al.建议代理服务器在遇到错误连接时不要立即关闭连接，而是“read forever”。这样做不但避免泄漏超时值，而且使得服务器发送与正常连接关闭时相同的TCP flags来关闭错误连接。

进一步说，“reading forever”本身不会带来更加独特的指纹。因为Frolov et al.发现互联网上大量的服务器都会有无限超时值（“infinite timeout”）的特征。David Fifield调查显示，许多流行的翻墙软件已经采取了“read forever”策略。这些软件包括OSSH，obfs4，Outline和Lampshade。

强制采用强密码

Len et al. 于2020年展示了针对Shadowsocks服务器的partitioning oracle攻击。利用在Shadowsocks中使用的non-committing AEAD，攻击者可以更高效地猜出密码。我们因此建议开发者强制采用强密码。一种可能的实现方式是要求用户密码的熵高于一定值。

主动探测你的实现

如果你是一个不同于Shadowsocks-libev和Outline的翻墙软件开发者或贡献者，我们鼓励你检查同样的漏洞是否也存在于你的Shadowsocks实现中。我们开源了我们在论文Section 5.1中用到的prober 模拟器。

鸣谢

我们想要感谢来自Jigsaw的Vinicius Fortuna，来自APNIC的Robert Mitchell和Dan Fidler，以及来自Qv2ray的DuckSoft和Student Main对本文提供的反馈。

联系

这篇报告首发于GFW Report。我们还在APNIC blog，net4people以及ntc.party同步更新了博文。

我们鼓励您或公开地或私下地分享您的评论或疑问。我们私下的联系方式可见GFW Report的页脚。

这篇教程记录了如何安装，配置并维护一台Shadowsocks-libev服务器。 这篇教程的亮点在于， 按照这里的配置建议，你的Shadowsocks-libev服务器可以抵御各种已知的攻击， 包括来自GFW的主动探测和封锁以及partitioning oracle攻击。 我们还在教程的最后加入了有关Shadowsocks-libev部署的常见问题。 截止2021年11月7日，我们收到零星的用户报告按此教程配置的服务器仍遭到了端口封锁，我们因此在文中分享一个配置备用端口来缓解端口封锁的方法。

我们致力于更新和维护这篇教程。如果今后发现了新的针对Shadowsocks-libev的攻击，我们将在第一时间在这篇教程中加入缓解攻击的办法。 因此请考虑将这个页面加入到你的收藏夹中。 另外，我们希望这篇教程对技术小白同样友好，因此如果你在任何步骤卡住了，请联系我们，或在下方评论区留言。我们会对教程作相应改进。

安装

安装Snap应用商店

通过Snap应用商店安装Shadowsocks-libev是官方推荐的方式。

• 如果你的服务器运行Ubuntu 16.04 LTS及以上的版本，Snap已经默认安装好了。
• 如果你的服务器运行了其他的Linux发行版，你只需跟着对应的发行版安装Snap core。

现在来检测一下你的服务器已经安装了需要的snapd和Snap core:

sudo snap install core

安装Shadowsocks-libev

现在我们安装最新的Shadowsocks-libev:

sudo snap install shadowsocks-libev --edge

配置

下面是我们推荐的Shadowsocks-libev服务器配置：

{
    "server":["::0","0.0.0.0"],
    "server_port":8388,
    "method":"chacha20-ietf-poly1305",
    "password":"ExamplePassword",
    "mode":"tcp_and_udp",
    "fast_open":false
}

注意，你需要把里面的ExamplePassword替换成一个更强的密码。 强密码有助缓解最新发现的针对Shadowsocks服务器的Partitioning Oracle攻击。 你可以用以下命令在终端生成一个强密码：openssl rand -base64 16。

你还可以考虑将server_port的值从8388改为1024到65535之间的任意整数。

现在打开通过Snap安装的Shadowsocks-libev默认的配置文件：

sudo nano /var/snap/shadowsocks-libev/common/etc/shadowsocks-libev/config.json

将上方替换过密码的配置信息复制粘贴到配置文件后， 按Ctrl + x退出。 退出时，文本编辑器将问你"Save modified buffer?"，请输入y然后按回车键。

可以看到，通过Snap安装的Shadowsocks-libev默认的配置文件路径太长了，不便于记忆。同时默认配置路径又没有在官方文档中标出。 我们因此建议你收藏此页面，以备今后查找。

防火墙

我们使用ufw来管理Shadowsocks服务器的防火墙。

在基于Debian的服务器上，可以通过如下命令安装ufw：

sudo apt update && sudo apt install -y ufw

然后开放有关ssh和Shadowsocks-libev的端口。 请注意，以下命令假设你在/var/snap/shadowsocks-libev/common/etc/shadowsocks-libev/config.json中的server_port的值为8388。 如果你的server_port用了其他的值，请对以下命令作相应的修改：

sudo ufw allow ssh
sudo ufw allow 8388

现在我们启动ufw:

sudo ufw enable

启动时如果弹出Command may disrupt existing ssh connections. Proceed with operation (y|n)?，请输入y并按回车键。

最后，请用sudo ufw status检查一下你的配置是否和下面的一样：

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
8388                       ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
8388 (v6)                  ALLOW       Anywhere (v6)

运行Shadowsocks-libev

现在我们启动Shadowsocks-libev：

sudo systemctl start snap.shadowsocks-libev.ss-server-daemon.service

记得设置Shadowsocks-libev开机自启动：

sudo systemctl enable snap.shadowsocks-libev.ss-server-daemon.service

维护

检查运行状态和日志

以下命令可以查看Shadowsocks-libev的运行状态：

sudo systemctl status snap.shadowsocks-libev.ss-server-daemon.service

如果你看到绿色的Active: active (running)，那么你的Shadowsocks-libev服务器就在正常的运行； 如果你看到红色的Active: failed，请用跳至如下命令journalctl -u snap.shadowsocks-libev.ss-server-daemon.service的尾部查看问题出在哪里了。

重新加载配置文件

每当你修改过配置文件后，请用如下命令重启Shadowsocks-libev以加载修改后的文件：

sudo systemctl restart snap.shadowsocks-libev.ss-server-daemon.service

配置备用端口来缓解端口封锁

截止2021年11月7日，我们收到零星的用户报告按此教程配置的服务器仍遭到了端口封锁。

因为报告的封锁方式均为端口封锁，而非IP封锁，我们在此分享一个用备用端口来缓解端口括封锁的方法。

你可以在服务器上使用以下命令来将服务器从12000到12010端口接收到的TCP和UDP流量全部转发到8388端口：

sudo iptables -t nat -A PREROUTING -p tcp --dport 12000:12010 -j REDIRECT --to-port 8388
sudo iptables -t nat -A PREROUTING -p udp --dport 12000:12010 -j REDIRECT --to-port 8388

记得：

1. 将12000:12010替换成一个只有你自己知道的端口号，或者端口区间（我们建议从1024到65535之间任选几个端口或一个区间）。
2. 将8388端口替换成你的Shadowsocks服务端实际使用的端口。

这样一来，如果你使用的12000端口遭到了封锁，那么你无须更换IP，或者登录服务器修改配置文件。而是只需要在客户端（电脑或者手机上）将端口从12000改为12001就可以继续使用了。

如果你配置正确，那么以下命令的输出应该类似于：

sudo iptables -t nat -L PREROUTING -nv --line-number

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpts:12000:12010 redir ports 8388
2        0     0 REDIRECT   udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpts:12000:12010 redir ports 8388

注意任何1024到65535的端口都可以作为备用端口。即使使用ephermeral端口(/proc/sys/net/ipv4/ip_local_port_range)作为配用端口也不会干扰服务器正常的向外连接。

常见问题

Q:为什么我用了教程里的配置，服务器还是被封了?

A: 截止2021年11月7日，我们收到零星的用户报告服务器的端口被封。因为通过这篇教程配置的Shadowsocks-libev服务器已经可以抵御已知的所有来自GFW的主动探测，所以有可能审查者使用了未知的攻击手段。如果你也遇到了类似问题，请考虑使用上述的备用端口方法来缓解封锁。我们鼓励你将封锁情况汇报给我们，我们会认真地调查。

Q: 我应不应该从发行版的仓库下载安装Shadowsocks-libev?

A: 发行版仓库里的Shadowsocks-libev不一定是最新版的。比如，截止2021年1月，Debian buster仓库的Shadowsocks-libev的版本为v3.2.5。而这个版本的Shadowsocks-libev是不够防御来自GFW的主动探测的（详见Figure 10）。

Q: 我应该怎样更新用Snap安装的Shadowsocks-libev?

A: 因为Snap会每天自动更新通过其安装的软件，因此通常情况下你不需要手动更新。如若需要手动更新，请用： sudo snap refresh。

Q: 为什么用chacha20-ietf-poly1305作为加密方式?

A: 因为它是其中一种AEAD ciphers。而AEAD ciphers可以抵御来自GFW的主动探测。它同时也是Shadowsocks-libev及OutlineVPN的默认加密方式。

Q: 我应该用Shadowsocks的stream cipher吗?

A: 完全不应该。因为Shadowsocks的stream cipher有着不可接受的安全隐私漏洞，并且可以被准确的主动探测。如Figure 10所示，即使是最新版的Shadowsocks-libev，在使用stream cipher时同样可以被准确识别。更具灾难性的是，在不需要密码的情况下，攻击者可以完全解密被记录下来的Shadowsocks会话。

Q: 但为什么我用的机场仍在使用stream cipher?

A: 这清楚地说明你的机场缺乏安全意识和安全措施。请把这篇教程，这个演讲，和这篇总结，分享给你的机场主。

Q: 我应该把配置中的server_port改为像443这样的常见端口吗?

A: 不应该。因为不论你使用哪个端口，GFW都会检测并怀疑你的Shadowsocks流量。

Q: 为什么配置文件使用tcp_and_udp模式?

A: 我们之前使用tcp_only模式是为了缓解Partitioning Oracle攻击。但正如Vinicius所指出的，如果你使用了长的随机密码，那么partitioning oracle攻击就不能成功。因此也就不需要禁用UDP代理模式。开启UDP代理模式可能会让经过Shadowsocks代理的视频通话质量更佳。

Q: 为什么配置文件禁用了fast_open?

A: 我们推荐你阅读这里的讨论。

联系

这篇报告首发于GFW Report。我们鼓励您或公开地或私下地分享您的评论或疑问。我们私下的联系方式可见GFW Report的页脚。

This tutorial documents how to install, configure and maintain a Shadowsocks-libev server. One cool thing about this tutorial is, by following this tutorial, your Shadowsocks-libev servers should be able to defend against various attacks, including active probing from the GFW and the partitioning oracle attack. Additionally, we compile a list of commonly asked questions, debunking common myths of Shadowsocks-libev. As of November 7, 2021, we received a few reports on the blocking of Shadowsocks ports. We thus share a way to setup backup ports to mitigate the inconvenience caused by port blocking.

Please consider bookmark this page because we commit to make this tutorial up-to-date and provide latest best practices to defend against emerging attacks. This tutorial is intended to be friendly to non-technical users. If you get lost at any step of this tutorial, please let us know and we will improve the documentation.

Installation

Install Snap

Snap is the officially recommended way to install Shadowsocks-libev.

• If your server is running Ubuntu 16.04 LTS or later, Snap is already installed.
• If your server is running some other Linux distributions, simply follow the corresponding installation instructions.

Now double check you have both snapd and Snap core installed:

sudo snap install core

Install Shadowsocks-libev

Now we install the latest Shadowsocks-libev from the edge channel:

sudo snap install shadowsocks-libev --edge

Configuration

Below is the recommended configuration for Shadowsocks-libev server:

{
    "server":["::0","0.0.0.0"],
    "server_port":8388,
    "method":"chacha20-ietf-poly1305",
    "password":"ExamplePassword",
    "mode":"tcp_and_udp",
    "fast_open":false
}

You need to replace the password ExamplePassword with a much stronger one. A strong password is recommended to mitigate the latest Partitioning Oracle Attacks against Shadowsocks servers. A handy way to generate a strong password from your terminal is: openssl rand -base64 16.

You may also want to change the server_port from 8388 to a different value between 1024 and 65535.

Now, open the default configuration file:

sudo nano /var/snap/shadowsocks-libev/common/etc/shadowsocks-libev/config.json

After copying and pasting your settings to the file, type Ctrl + x to exit. The text editor will ask "Save modified buffer?", and you can type y and then hit Enter.

The default configuration file path is too long to remember, and it is not well documented elsewhere yet. We thus encourage you to bookmark this page so that you can go back to check it whenever you want.

Firewall

We use ufw to open ports for the Shadowsocks server.

To install ufw on a Debian-based server:

sudo apt update && sudo apt install -y ufw

Then open ports for ssh and Shadowsocks-libev. Note that if you set the server_port to a value different than 8388 in /var/snap/shadowsocks-libev/common/etc/shadowsocks-libev/config.json, you need to change the value 8388 below accordingly:

sudo ufw allow ssh
sudo ufw allow 8388

Now enable ufw:

sudo ufw enable

If it prompts Command may disrupt existing ssh connections. Proceed with operation (y|n)?, type y and hit Enter.

Finally, run sudo ufw status, and the output should look like this:

Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
8388                       ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
8388 (v6)                  ALLOW       Anywhere (v6)

Run Shadowsocks-libev

Now you can start the Shadowsocks-libev:

sudo systemctl start snap.shadowsocks-libev.ss-server-daemon.service

Remember to let Shadowsocks-libev auto-start after a server reboot:

sudo systemctl enable snap.shadowsocks-libev.ss-server-daemon.service

Maintenance

Check status and log

To check the status of the service:

sudo systemctl status snap.shadowsocks-libev.ss-server-daemon.service

If you see Active: active (running) in green, your Shadowsocks-libev server is running properly; if you see Active: failed in red, jump to the end of journalctl -u snap.shadowsocks-libev.ss-server-daemon.service to see what is wrong.

Reload configuration file

Whenever you change the configuration file, remember to restart Shadowsocks-libev to load the latest settings:

sudo systemctl restart snap.shadowsocks-libev.ss-server-daemon.service

Mitigate port blocking by having backup ports

As of November 7, 2021, we received a few reports on the blocking of Shadowsocks ports.

In particular, since the GFW has been reportedly doing port blocking, rather than IP blocking, having a few backup ports may mitigate the incovenience when port blocking happens.

A user can run the following command on server to redirect both TCP and UDP traffic ranging from port 12000 to 12010, to port 8388:

sudo iptables -t nat -A PREROUTING -p tcp --dport 12000:12010 -j REDIRECT --to-port 8388
sudo iptables -t nat -A PREROUTING -p udp --dport 12000:12010 -j REDIRECT --to-port 8388

Remember to:

1. replace 12000:12010 with a different range that only you know (we suggest any port from 1024 to 65535);
2. replace 8388 with the Shadowsocks port you used.

This way, whenever a port on server got blocked, a user only needs to change the configurations of server_port on the clients, without having to login to the server, change configuration, and then restart Shadowsocks. For example, if you had been using port 12000 but it got blocked, all you need to do is changing the server_port from 12000 to another backup port 12001 on your client.

When properly configured, the output should look like this:

sudo iptables -t nat -L PREROUTING -nv --line-number

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpts:12000:12010 redir ports 8388
2        0     0 REDIRECT   udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpts:12000:12010 redir ports 8388

Note that setting a PREROUTING rule on ephermeral ports (/proc/sys/net/ipv4/ip_local_port_range) will not disrupt normal outgoing connections that use those ephermeral ports as source ports.

Q&A

Q: Why did my server still get blocked when I followed your tutorial?

A: As of November 7 2021, we indeed received a few report on the blocking of Shadowsocks. Since this tutorial can defend all known active probing attacks by the GFW, it is likely that the censor has employed some unknown attacks against Shadowsocks-libev. If your server got blocked, too, please consider using the backup ports to mitigate the blocking. We also encourage you report the block to us and we will carefully investigate it.

Q: Should I install Shadowsocks-libev from a distribution repo?

A: A distribution repo may not always include the latest version of Shadowsocks-libev. For example, as of January 2021, the version included in Debian buster repo was v3.2.5, which was not sufficient to defend active probings from the GFW (see Figure 10).

Q: How can I update Shadowsocks-libev via snap?

A: Usually you don’t have to update it manually because snap automatically updates all apps once per day. To manually update immediately: sudo snap refresh.

Q: Why do you use chacha20-ietf-poly1305?

A: Because it is one of the AEAD ciphers, which can defend the active probings by the GFW. It is also the default encryption method for both Shadowsocks-libev and OutlineVPN.

Q: Should I use any stream cipher in Shadowsocks?

A: No. It is unacceptably insecure. Even the latest version of Shadowsocks-libev operating in stream cipher mode is vulnerable to active probing (see Figure 10). More devastatingly, an attacker can get full decryption of recorded Shadowsocks sessions, without knowing the password.

Q: But my “airport” is still using stream cipher?

A: Then it is clear sign that your “airport” has very poor security awareness. Point the owners to this tutorial, as well as this post and this summary.

Q: Should I change the server_port to some common ports like 443?

A: No. The GFW can still suspect your Shadowsocks traffic, regardless of the server port you use.

Q: Why do you operate Shadowsocks-libev in tcp_and_udp mode?

A: Previously, we used tcp_only mode to mitigate the Partitioning Oracle Attacks against Shadowsocks servers; however, as Vinicius pointed out, the partitioning oracle attack is not feasible when you have long random passwords, and it is recommended to “enable UDP support in order to enjoy better video calls”.

Q: Why do you disable fast_open?

A: We recommend that you read this rationale.

Contacts

This report appeared first on GFW Report.

We encourage you to share your comments publicly or privately. Our private contact information can be found at the footer of GFW Report.

In our recent IMC'20 work (paper, talk) we provided insight into the combination of traffic analysis and active probing techniques that the Great Firewall uses to detect and block Shadowsocks servers.

In this short post, we provide practical suggestions for non-technical users and circumvention tool developers to prevent their circumvention servers from being detected and blocked. We also introduce the mitigation to partitioning oracle attacks newly demonstrated by Len et al.. If your Shadowsocks servers still get blocked, even when you follow the instructions below, please report the block to GFW Report and/or the corresponding developers.

For users

According to our own testing, and reports from developers, with proper configurations, there are at least two Shadowsocks implementations whose latest releases are resistant to the GFW’s active probes: Shadowsocks-libev and OutlineVPN.

Advice on using Shadowsocks-libev

If you decide to use Shadowsocks-libev, we strongly encourage you to follow this tutorial on how to install, configure and maintain a Shadowsocks-libev server. We commit to make that tutorial up-to-date and provide latest best practices to defend against emerging attacks against Shadowsocks.

If you already have a Shadowsocks-libev server installed, check the rules below to see if your servers have been properly configured.

As of January 2021, to prevent your Shadowsocks-libev server from being blocked, you have to:

1. Ensure your server version is v3.3.1 or above. You can check the server version with the command ss-server -h.
2. Use AEAD ciphers, not stream ciphers. In other words, choose one of the following encryption methods: chacha20-ietf-poly1305 (recommended), aes-256-gcm, aes-192-gcm or aes-128-gcm.

To mitigate the partitioning oracle attacks against Shadowsocks servers, you need to:

1. use a long random password, which can be generated from terminal using: openssl rand -base64 16;
2. and disable UDP mode.

Note: there is no special requirement for the client. Any client compatible with the server is okay.

Advice on using OutlineVPN

To prevent your OutlineVPN server from being blocked, you have to:

1. Use the latest client from its official website.
2. Use the latest server from its official website.

Note that:

1. The Outline already generates a long, random secret for you, so you don’t have to do it manually like what you have to do for a Shadowsocks-libev server.
2. The Outline server can automatically update itself, so you don’t have to manually upgrade it.
3. The Outline exclusively uses chacha20-ietf-poly1305 (an AEAD cipher) as the encryption method, so you don’t have to select encryption methods manually.

For circumvention tool developers

Below, we introduce our findings on the latest capabilities of the GFW; along with our suggestions for making circumvention tools more resistant to censorship. These suggestions are useful not only for Shadowsocks, but for other circumvention tools. We encourage you to participate in this discussion and we welcome any thoughts, comments, questions or concerns.

Proper authentication

First of all, we strongly encourage circumvention tool developers to completely deprecate unauthenticated cryptographic constructions. Confidentiality alone is not enough.

• For newly developed circumvention tools, unauthenticated cryptographic constructions are unacceptable.
• For existing circumvention tools, developers should bravely remove all code related to unauthenticated cryptographic constructions, even at the cost of incompatibility.

Our seemingly bold suggestion comes for a good reason. As introduced in our paper, certain types of the GFW’s probes exploit the ciphertext malleability of Shadowsocks’s stream ciphers. This is not the first vulnerability caused by unauthenticated cryptographic constructions. In fact, the lack of proper authentication has been the root cause of many vulnerabilities in Shadowsocks and other circumvention tools.

In August 2015, BreakWa11 discovered an active probing vulnerability in Shadowsocks stream ciphers, resulting from their lack of integrity protection (see English summary). In 2020, similar vulnerabilities has also been discovered in other circumvention tools like V2Ray (see the summary). Shadowsocks developers mitigated the vulnerability by having the server not immediately terminate a connection when a target specification contains an unknown address type.

Shadowsocks developers attempted to further mitigate the problem by introducing a one time auth mode, in which each chunk of data would carry its own authenticator. But a lack of integrity protection in chunk length prefixes led to another active probing vulnerability (see English summary).

In February 2020, Zhejiang Peng disclosed a devastating vulnerability in Shadowsocks stream ciphers (see English summary). Using the Shadowsocks server as a decryption oracle, an attacker, without knowledge of the shared master password, can get full decryption of recorded Shadowsocks connections.

As early as February 2017, AEAD ciphers had became part of the Shadowsocks protocol specification. While the authentication problem should have been fixed by that time, many servers are still using the vulnerable and deprecated unauthenticated stream ciphers in 2021. This phenomenon indicates that, in practice, many users cannot properly choose the right ciphers to use, possibly misguided by out-dated tutorials. We therefore encourage developers to help users make the right choice by completely removing stream ciphers from Shadowsocks.

Replay filtering based on both nonces and timing

We suggest that circumvention tool developers deploy a replay filter that takes into account both nonces and timing. Since a timing-based replay filter may require a fundamental changes to existing protocols, developers should at least deploy a nonce-based replay filter and:

1. Either advise users to change passwords after each replay filter initialization; or
2. Develop a mechanism to let the replay filter remember nonces even after a restart.

These suggestions are based on the following research findings and rationale. As introduced in section 3.5 of the paper, the GFW can replay a legitimate connection immediately; or it may remember a legitimate connection but not replay it until three weeks later. Therefore, a realistic adversary model of active probing should permit the censor to perform replay attacks after an arbitrarily long delay.

Such a model reveals an asymmetry between attack and defense for purely nonce-based replay defense mechanisms. While it does not cost much in terms of resources for the GFW to record a few legitimate payloads and replay them after a fairly long delay, it is costly and complicated for Shadowsocks servers to remember the nonces of all authenticated connections forever, or until the master password is changed. The Shadowsocks server must remember those nonces even after being restarted; otherwise, the replay filter will be ineffective against replays that span a restart.

Fortunately, this unfair game can be inverted by the addition of a timing-based defense mechanism: the server only responds to authenticated connections that are not replays and whose timestamp is within an expiration time, similar to what VMess servers do. This way, the server does not need to remember nonces forever, but only for a limited time.

We also want to emphasize that a replay filter is required even in circumvention tools that use a random emphermeral open port per session. This is because the GFW may send a replay immediately after seeing the first data packet of a legitimate connection, before the session and the port are closed.

Being consistent in servers’ reactions

We suggest that developers make sure their circumvention tools react consistently not only in normal operation, but also when an error occurs. Ideally, as suggested by Frolov et al., a server should “read forever” on error. The reason for these suggestions is that censors may intentionally trigger protocol edge cases in an attempt to fingerprint servers.

In addition to the inconsistencies we found in Shadowsocks-libev and OutlineVPN, Frolov et al. demonstrated that various proxy servers, including Shadowsocks-python and OutlineVPN, can be identified using TCP flags and timing metadata after the servers close a connection. studentmain reported that, as of December 2020, various Shadowsocks implementations still have similar problems.

Frolov et al. suggest that proxy servers should read forever when errors occur, rather than terminating the connection. Doing so not only avoids revealing a specific timeout value, but also lets the server close the connection with consistent TCP flags in the non-error case.

Furthermore, reading forever will not make fingerprints of your circumvention tools more unique, as Frolov et al. found that a large number of endpoints exhibits such “infinite timeout” behavior. As surveyed by David Fifield, various popular circumvention tools have employed the “read forever” strategies, including OSSH, obfs4, Outline and Lampshade.

Enforce using strong passwords

In 2020, Len et al. demonstrated partitioning oracle attacks against Shadowsocks servers. Exploiting the non-committing AEAD schemes used in Shadowsocks, an attacker can recover the password of Shadowsocks servers in a more efficient manner. We thus encourage the developers to enforce users to use strong passwords, possibly by setting entropy requirements for users’ passwords.

Probe your implementations

If you are a developer or contributor of a Shadowsocks implementation other than Shadowsocks-libev or Outline, we encourage you to check if similar vulnerabilities exist in your implementation of Shadowsocks. We have open-sourced the prober simulator we used in Section 5.1 of the paper.

Acknowledgement

We want to thank Vinicius Fortuna at Jigsaw, Robert Mitchell and Dan Fidler at APNIC, as well as DuckSoft and Student Main at Qv2ray for offering feedback to a draft of this post.

Contacts

This report appeared first on GFW Report. We also maintain an up-to-date copy of the report on APNIC blog, net4people and ntc.party.

We encourage you to share your comments publicly or privately. Our private contact information can be found at the footer of GFW Report.

A typical DNS poisoning event consists of three steps:

1. A client sends a DNS query to a DNS server, asking for the IP address of a sensitive domain;
2. the GFW observes the query and injects a forged DNS response, telling the client a wrong IP address;
3. the client receives the forged response, and attempts to connect to the wrong IP address.

But what will happen after step 3?

While packets sent to these wrong IP addresses are often believed to be dropped or null-routed; in this report, we document an interesting phenomena: the GFW pretends to be some of the injected IP addresses to accept (or deny) TCP handshakes from the clients. Such behavior tricks censored clients to send data that would never be sent if the handshakes were not accepted. The censor can thus learn more about the effect of its censorship and also the intention of clients.

We characterize this behavior and fingerprint the censoring machine. Our findings suggest that the censoring machine is likely to be stateless; it also employs some load-balancing that accepts around 75% of the traffic. We locate the injecting point of the censoring machine, and find that DNS queries or TLS connections to these IP addresses are not affected by the DNS or SNI censorship. Finally, we suggest that users encrypt their DNS queries and block all outgoing traffic to these injected IP addresses.

Main findings

• The GFW pretends to be four injected IP addresses to accept (or deny) TCP handshakes at all 65535 ports, potentially seducing censored clients to send more data.
• The implementation of the censoring machine appears to be crude and stateless.
• The censoring machine employs some load-balancing to accept around 75% of the traffic. The load-balancing algorithm is associated with (srcIP, dstIP, srcPort, dstPort) tuple.
• The GFW has a 31-second residual censorship after denying a TCP handshake to some of the injected IP addresses.
• The censoring machine is likely to share the same location as the DNS injection points.

Reachability of Injected IP addresses

Our investigation started from something we could not explain. As show in Figure 3 of Anonymous et al. 2020, when testing the reachability of the injected IP addresses from China and the US, around 40% of the results are different. For example, 0.4% of the IP-port pairs are open when tested from China, but are filtered when tested from the US. (See definitions of port state open, closed and filtered in the Nmap manual.)

To answer why such inconsistency exists, we re-tested their reachability. We first obtained the injected IP addresses from the open dataset. We then used Nmap to SYN ping all 65535 ports of the 215 IP addresses from China:

nmap -iL ips_after_drop.txt -p1-65535 -Pn --min-rate=500 -oX all_ips_65535.xml

In total, we discovered 14 IP addresses that has at least one non-filtered port; the rest 201 IP addresses appeared to not respond at any port. Based on the port states, we categorized these 14 IP addresses into three groups:

• Group 1 (G1): 4 IP addresses whose 1-65535 ports are open when tested from China; but are all filtered when tested from the US:

  8.7.198.46
  46.82.174.69
  59.24.3.174
  93.46.8.90
  

• Group 2 (G2): 7 IP addresses have a few closed ports; and all other ports are filtered when tested from China. All ports are filtered when tested from the US:

  8.7.198.45
  67.228.126.62
  93.46.8.89
  118.5.49.6
  188.5.4.96
  203.98.7.65
  208.101.48.171
  

• Group 3 (G3): 3 IP addresses have a few open ports; and a few closed ports; the rest are filtered when tested from China. Similar results with slight differences were discovered when tested from the US.

  31.13.64.49
  31.13.72.54
  31.13.85.1
  

G1 IPs

We first discuss the four IP addresses in group 1:

8.7.198.46
46.82.174.69
59.24.3.174
93.46.8.90

We call them listening IPs because all their ports from 1 to 65535 were observed to accept TCP handshakes, when tested from China.

G1 IPs are exclusively used by Injector 1

Prior work shows that the GFW uses multiple DNS injectors, and each maintain a separate blacklist (see Figure 5) and injects a different set of IP addresses (see Table 3). Therefore, it is intuitive to ask:

1. Which injectors are using these listening IPs?
2. What domains do the GFW poison with these IP addresses?

These questions can be answered easily using the open dataset:

grep --max-count 1 "59\.24\.3\.174" injector*.csv

Interestingly, we found these four listening IPs exactly match the first row of Table 3. In other words, they happen to be the same four IP addresses used by the injector 1, and they are used to poison 88 domains:

cut -d";" -f2 injector1.csv | sort | uniq

www.8800.org
www.expressvpn.com
www.google.as
www.google.bf
www.google.bi
www.google.bj
www.google.bs
www.google.bt
www.google.by
www.google.cat
www.google.cd
www.google.cg
www.google.ci
www.google.cm
www.google.co.ao
www.google.co.ck
www.google.co.ls
www.google.com.af
www.google.com.ag
www.google.com.ai
www.google.com.ar
www.google.com.bd
www.google.com.bz
www.google.com.cu
www.google.com.do
www.google.com.eg
www.google.com.et
www.google.com.fj
www.google.com.gh
www.google.com.gi
www.google.com.lb
www.google.com.ly
www.google.com.mm
www.google.com.ng
www.google.com.np
www.google.com.pg
www.google.com.pk
www.google.com.py
www.google.com.sb
www.google.com.sl
www.google.com.tj
www.google.com.vc
www.google.co.mz
www.google.co.tz
www.google.co.ug
www.google.co.uz
www.google.co.ve
www.google.co.zw
www.google.cv
www.google.dj
www.google.ga
www.google.gg
www.google.gl
www.google.gm
www.google.gp
www.google.gy
www.google.hn
www.google.ht
www.google.im
www.google.iq
www.google.it
www.google.je
www.google.kg
www.google.ki
www.google.la
www.google.li
www.google.me
www.google.mg
www.google.ml
www.google.mn
www.google.mv
www.google.mw
www.google.ne
www.google.pn
www.google.ps
www.google.rs
www.google.sm
www.google.sn
www.google.so
www.google.sr
www.google.st
www.google.td
www.google.tg
www.google.tl
www.google.tm
www.google.to
www.google.ws
www.kuniao.com

As listed above, most of these domains are related to Google, with three exceptions to be:

• www.kuniao.com (see the related censorship event),
• www.8800.org (archive),
• www.expressvpn.com

Characterize the listening behavior on G1 IPs

The behaviors of the censoring machine are rather simple. It spoofs these four listening IPs and:

• When client sends a packet with SYN flag turned on AND PSH and ACK flags turned off, it replies a SYN+ACK packet.
• When client sends a packet with PSH flag turned on, it replies a single RST packet.

You can also quickly test it using Nping:

sudo nping -c 0 --tcp 59.24.3.174 -p65535 --flags S
sudo nping -c 0 --tcp 59.24.3.174 -p65535 --flags P

While, in theory, the GFW has the capability of replying to clients’ requests, we did not observe such behaviors. The GFW appears to tear down connections regardless of the data sent by our clients. We tested this by trying to establish typical HTTP or TLS connections with these four IP addresses from China. The three commands we used are as follows:

# HTTP GET
wget http://59.24.3.174 -v
# TLS with SNI=www.google.com
openssl s_client -servername www.google.com -tlsextdebug -msg -connect 59.24.3.174:443
# TLS with SNI=www.baidu.com
openssl s_client -servername www.baidu.com -tlsextdebug -msg -connect 59.24.3.174:443

As a result, all our connections were RSTed after successful handshakes.

The intention of the censor

We have two different conjectures on why the censor behaves this way. The first conjecture is that the censor simply intends to disrupt TCP connections to these four IP addresses. Indeed, the forged SYN+ACKs and RSTs are reminiscent of the GFW’s obsolete way to disrupt TCP connections during residual censorship period. (Note, we say “obsolete” because the GFW does not send any packet during the 60-second residual censorship anymore as of November 10, 2020. We triggered the residual censorship by using a TLS ClientHello with sensitive SNI.) As introduced by Wang et al. in the TCP connection reset section:

“During this period, any SYN packet between the two end-hosts will trigger a forged SYN/ACK packet with a wrong sequence number from the GFW, which will obstruct the legitimate handshake; any other packets will trigger forged RST and RST/ACK packets, which will tear down the connection.”

However, the first conjecture cannot explain why the censor spends extra resources sending packets, instead of simply dropping or null-routing all packets to these four IP addresses.

The second conjecture is that, by pretending to be these injected IP addresses to accept TCP handshakes, the GFW can seduce clients to send data that would never be sent if the handshakes were not accepted. The censor can thus learn more about the effect of its censorship and also the intention of clients. Specifically, the connection attempts to these injected IP addresses provides the censor with a different angel to measure the effectiveness of its DNS censorship. The censor would have underestimated the number of DNS censorship events, if it merely counts the number of sensitive DNS queries it observes at the Internet backbone or border. This is because the majority of the Chinese clients use a local resolver, and their sensitive DNS queries are answered by local resolvers with poisoned cache, not by the GFW. Thus, by observing the traffic to these injected IP addresses, the censor will have a better idea on how many clients received DNS poisoned answers. Furthermore, by seducing the censored clients to send more data than a SYN packet, the GFW can learn more on what the clients would do if its connection was not censored.

We remind our readers that, although the conjectures may sound reasonable, we have no way to validate or falsify them.

The censoring machine appears to be stateless

Many evidences suggest that the censoring machine appears to be stateless, and its implementation is crude. For example:

1. The censoring machine never times out;
2. The censoring machine does not retransmit SYN+ACK, even if the client does not ACK.
3. The censoring machine does not respond to SYN+ACK with RST.
4. The censoring machine does not validate the IP and/or TCP checksum

First, to measure the timeout value of the censoring machine, we used the following command to connect to one of the listening IPs:

nc -v 59.24.3.174 443

After Connection to 59.24.3.174 443 port [tcp/https] succeeded!, we intentionally did not send any data for more than 30 minutes; however, the censoring machine did not send any packet to close the connection. As soon as we sent a chunk of data TEST via nc, the censoring machine sent a single RST to tear down the connection. This experiment shows that the censoring machine either does not timeout, or has an unusually large timeout value. If the censoring machine was indeed stateful, having such a large timeout value would have easily drained up its resources.

Second, the censoring machine does not retransmit SYN+ACK, even if the client does not ACK. One can test it with commands like these:

# capture traffic usig tcpdump,
# as nping will not show if RST was sent by kernel
sudo tcpdump -n host 59.24.3.174 and port 442

## open another terminal:
# drop RST sent by kernel due to unexpected SYN+ACK
sudo iptables -A OUTPUT -p tcp --tcp-flags RST RST -s $(hostname -I) -j DROP
# send SYN packets
sudo nping -c 0 --tcp 59.24.3.174 -p443 --flags S
# delete the dropping RST rule
sudo iptables -D OUTPUT -p tcp --tcp-flags RST RST -s $(hostname -I) -j DROP

Third, the censoring machine does not respond to unexpected SYN+ACK with RST:

# Sending SA to open ports of listening IPs will not get RST
sudo nping -c 0 --tcp 59.24.3.174 -p443 --flags SA
# Sending SA to open ports of common TCP server will get RST
sudo nping -c 0 --tcp 1.1.1.1 -p443 --flags SA

Forth, the censoring machine replies a SYN+ACK, even if the SYN packets has a wrong IP and/or TCP checksum:

sudo nping -c 0 --tcp 59.24.3.174 -p443 --flags S --badsum-ip --badsum

All evidences above suggest the TCP implementation of the censoring machine is simple and crude. Indeed, sometimes, “worse is better”. If all the censor needs is to seduce TCP handshakes, then this stateless implementation means simplicity in code, efficiency in resources, less fingerprints, and a smaller attack surface.

Fingerprint the packets by G1

We fuzzed and sent packets to these listening IPs using the following script:

#!/usr/bin/env python3
from scapy.all import *
packet=Ether() / fuzz(IP(dst="59.24.3.174")) / fuzz(TCP())
sendp(packet, loop=1)

We then compare the packets forged and replied from the censoring machine.

On IP-layer, the fingerprints of the forged replies are:

• Type of service (ToS): fixed value 0x68 (throughput bit is turned on).
• IPID: same as the triggering packet.
• IP flag: the Don't Fragment (DF) bit is same as the triggering packet; the censoring machine does not respond to packets with the More Fragments (MF) bit turned on.
• IP TTL: same as the TTL value of the triggering packet received.

On TCP-layer, the fingerprints of the forged replies are:

• TSVal (of SYN+ACK): same as the SYN packet.
• TCP Sequence number (of SYN+ACK): random from 0 to 2^32.
• TCP flag: if the triggering packet has SYN bit turned on AND PSH and ACK bits turned off, then the flag is a copy of the triggering packet with the ACK bit turned on; if the triggering packets has PSH bit turned on, then the flag is a copy of the triggering packet with RST bit turned on, and with the ACK, PSH, SYN, FIN bits turned off.
• TCP window size: same as the triggering packet.
• Urgent pointer: same as the triggering packet.
• TCP option: almost same as the triggering packet, except that, if MSS specified in the triggering packet, the MSS is fixed value 1424.

If you want to do a quick sanity check to see if the censoring machine has the same fingerprints in your area, you may use commands similar to this:

sudo nping -c 0 --tcp 59.24.3.174 -p443 --id 3333 --tos 0x02 -df --flags S --win 1 -v4

We also note that, some seeming randomly fingerprints of the GFW will exhibits certain pattern, only when the GFW is probed at a very fast speed. One example is the Figure 4.a by Anonymous et al.. We thus SYN-flooded one of the listening IPs (for less than 2 seconds). Although, we received GFW’s forged SYN+ACK at a speed of 15,000 packets per second; as shown in the figure below, the TCP sequence numbers still appear to be random.

Load balancing

We find that SYN or PSH packets do not always trigger the censoring machine to send the responding SYN+ACK or RST. Further investigation reveals that, for a given (srcIP, dstIP, srcPort, dstPort) tuple, whether the packets can trigger the censoring machine or not is deterministic. In addition, when fixing the (srcIP, dstIP, dstPort), and enumerating all srcPort from 1 to 65535, almost exactly 75% (min: 49132/65535; max: 491363/65535) of the srcPorts can trigger the censoring machine. This result suggests that the censoring machine employs some kind of load-balancing.

We could not figure out the exact load balancing algorithm used; but this directory includes the code, data, and analysis we have done. We encourage you to explore this intriguing problem.

Location of the censoring machine

As introduced above, the packets sent by the censoring machine has the same IP TTL value as the triggering packet received. This IP TTL mirroring behavior also exits in some DNS poisoning injectors of the GFW (see Figure 8.

One important implication of IP TTL mirroring is that the censorship machine appears to be much further away from the testing host than it actually is. We took this into consideration, and find the censoring machine is 8-9 hosts away from our host. We then used limited-TTL approach to locate the DNS injection points from this host to a IP addresses within the same /30 of the G2 IP addresses. We find the censoring machine may share the same location with the DNS injection points. We say “may” because we could not accurately locate the censoring machine due to asymmetric routing.

We also find that connections to these listening IPs are not under DNS or SNI censorship; while the IP addresses within the same /30 are under censorship. For example:

# 46.82.174.69 is one of the listening IPs and we get no forged response.
dig @46.82.174.69 www.google.sm
# 46.82.174.70 is within the same /30 and we get forged responses.
dig @46.82.174.70 www.google.sm

G2 IPs

The 7 IP addresses have a few closed ports; and all other ports are filtered when tested from China. When SYN pinging theses addresses from the US, all 65535 ports appear to be filtered. We SYN-pinged these ports with limited TTL. The result shows that we received RSTs even before reaching the actual IP addresses, indicating these RSTs were actually sent by the GFW.

The closed ports are listed as follows:

118.5.49.6,1723,closed,pptp
188.5.4.96,1723,closed,pptp
203.98.7.65,1080,closed,socks
203.98.7.65,1723,closed,pptp
208.101.48.171,5222,closed,xmpp-client
67.228.126.62,443,closed,https
8.7.198.45,443,closed,https
8.7.198.45,1080,closed,socks
8.7.198.45,1723,closed,pptp
93.46.8.89,443,closed,https

Note that 118.5.49.6 and 188.5.4.96 look very similar, as if they were carelessly picked by someone.

Among the 10 closed ports, the typical services running on these ports include PPTP, SOCKS, xmpp-client and HTTPS. PPTP and SOCKS can be used as censorship circumvention protocols.

Censoring machine appears to be on-path

We find we could still get ICMP TTL=0 messages while the initial TTL is large enough to trigger three RSTs.

sudo nping -c 0 --tcp 203.98.7.65 -p1080  --flags S --ttl 10

SENT (0.0019s) TCP host:19825 > 203.98.7.65:1080 S ttl=10 id=951 iplen=40  seq=18295394 win=1480
RCVD (0.2008s) TCP 203.98.7.65:1080 > host:19825 RA ttl=251 id=20366 iplen=40  seq=0 win=3509
RCVD (0.2008s) TCP 203.98.7.65:1080 > host:19825 RA ttl=251 id=20366 iplen=40  seq=0 win=3509
RCVD (0.2008s) TCP 203.98.7.65:1080 > host:19825 RA ttl=251 id=20366 iplen=40  seq=0 win=3509
RCVD (0.2008s) ICMP [202.97.90.114 > host TTL=0 during transit (type=11/code=0) ] IP [ttl=243 id=17562 iplen=96 ]
SENT (1.0033s) TCP host:19825 > 203.98.7.65:1080 S ttl=10 id=951 iplen=40  seq=1829538994 win=1480
RCVD (1.2208s) ICMP [202.97.90.114 > host TTL=0 during transit (type=11/code=0) ] IP [ttl=243 id=17732 iplen=96 ]

This suggests that the on-path GFW injects the three RSTs, without dropping those SYN packets.

Residual censorship on G2 IPs

There is a 31-second residual censorship after the GFW sends the RSTs. During this period of time, the GFW does not send RSTs to our SYNs if our SYNs have the same (srcIP, dstIP, srcPort, dstPort) tuple.

sudo nping -c 0 --tcp 118.5.49.6 -p1723 -g10001 --flags S

SENT (0.0020s) TCP host:10001 > 118.5.49.6:1723 S ttl=64 id=45322 iplen=40  seq=3600506539 win=1480
RCVD (0.2010s) TCP 118.5.49.6:1723 > host:10001 RA ttl=251 id=29186 iplen=40  seq=0 win=2857
RCVD (0.2010s) TCP 118.5.49.6:1723 > host:10001 RA ttl=251 id=29186 iplen=40  seq=0 win=2857
RCVD (0.2010s) TCP 118.5.49.6:1723 > host:10001 RA ttl=251 id=29186 iplen=40  seq=0 win=2857
SENT (1.0022s) TCP host:10001 > 118.5.49.6:1723 S ttl=64 id=45322 iplen=40  seq=3600506539 win=1480
SENT (2.0037s) TCP host:10001 > 118.5.49.6:1723 S ttl=64 id=45322 iplen=40  seq=3600506539 win=1480
SENT (3.0051s) TCP host:10001 > 118.5.49.6:1723 S ttl=64 id=45322 iplen=40  seq=3600506539 win=1480
....
SENT (31.0406s) TCP host:10001 > 118.5.49.6:1723 S ttl=64 id=45322 iplen=40  seq=3600506539 win=1480
RCVD (31.2090s) TCP 118.5.49.6:1723 > host:10001 RA ttl=251 id=29359 iplen=40  seq=0 win=2904
RCVD (31.2090s) TCP 118.5.49.6:1723 > host:10001 RA ttl=251 id=29359 iplen=40  seq=0 win=2904
SENT (32.0410s) TCP host:10001 > 118.5.49.6:1723 S ttl=64 id=45322 iplen=40  seq=3600506539 win=1480
RCVD (32.0411s) TCP 118.5.49.6:1723 > host:10001 RA ttl=251 id=29359 iplen=40  seq=0 win=2904

G3 IPs

Different from the IP addresses in group 1 and group 2, we find no evidence that the group 3 IP addresses are spoofed by the GFW. However, we indeed find packets to some of their ports are dropped by the GFW.

Specifically, we first SYN pinged all their ports from both China and the US:

nmap 31.13.64.49 31.13.72.54 31.13.85.1 -p1-65535 -Pn --min-rate=5000

The result from China is:

31.13.64.49  (443 open; other filtered)
31.13.72.54  (80,443 open; 843,5222,5228,8883 closed; other filtered)
31.13.85.1   (80,443 open; 843,5222,8883 closed; other filtered)

The result from the US is:

31.13.64.49  (80,443 open; 843,5222,5228,8883 closed, other filtered)
31.13.72.54  (80,443 open; 843,5222,5228,8883 closed; other filtered)
31.13.85.1   (80,443 open; 843,5222,5228,8883 closed; other filtered)

We repeated the SYN ping several times to eliminate the inconsistency due to packet losses; however, there are still some ports unreachable from China. Further investigation using a traceroute-like SYN-ping, we find the packets to those ports were dropped by the router at the CHINANET backbone. For example:

# This will get ICMP TTL=0 from the router at the CHINANET backbone
sudo nping -c 0 --tcp 31.13.64.49 -p443  --flags S  -ttl 7
# But this will not get such message, suggesting packets drop
sudo nping -c 0 --tcp 31.13.64.49 -p80  --flags S  -ttl 7

For those ports that are reachable from China, we tried to establish HTTP or TLS connections to them. The result shows that the client can successfully got the corresponding HTTP 400 Bad Request or the TLS certificate from the server, without being disrupted by the GFW. The commands we used are as follows:

wget http://31.13.85.1:80
openssl s_client -tlsextdebug -msg -connect 31.13.85.1:443

Our suggestion

We suggest users encrypt their DNS traffic whenever possible. At the same time, one can use iptables rules to block all outgoing traffic to these IPs to avoid their connection attempts being logged by the GFW. Blocking traffic to these IP addresses do little harm to your Internet connectivity, as theses IP addresses are blocked, spoofed or filtered anyway. Specifically, one can try executing the following commands to add the iptables rules:

#!/bin/bash

# Get the 215 injected IP addresses
wget https://gfw.report/publications/foci20_dns/foci20_anonymous/injected_ips/ips_after_drop.txt

# source: https://www.cyberciti.biz/faq/iptables-read-and-block-ips-subnets-from-text-file/

### Setup our black list ###
# Create a new chain
$IPT -N droplist

# Filter out comments and blank lines
# store each ip or subnet in $ip
while IFS="" read -r p || [ -n "$p" ]
do
    # Append everything to droplist
    iptables -A droplist -s "$ip" -j LOG --log-prefix " Drop Bad IP List "
    iptables -A droplist -s "$ip" -j DROP
done <ips_after_drop.txt

# Finally, insert or append our black list
iptables -I INPUT -j droplist
iptables -I OUTPUT -j droplist
iptables -I FORWARD -j droplist

Limitations

• All measurement experiments conducted from China were done using one vantage point. This means we are not able to find any Ge-location inconsistency if it exists. We encourage you to conduct your measurement experiments and report what you observe.

Acknowledgement

这场演讲介绍了我们的IMC'20论文: Shadowsocks是如何被检测和封锁的。

你可以点击视频下方的cc按钮来选择中文或英文字幕。

这是关于"中国是如何检测和封锁Shadowsocks的"的论文介绍， 该论文出自GFW Report，Jan Beznazwy和Amir Houmansadr。 我是David Fifield。我今天代替作者做论文介绍， 因为他们中大多数都是匿名的。 我有在这个领域的研究经验， 而且在作者的帮助下， 我已经完全熟悉了他们的这项工作。

对这项工作总结成一句话就是： 中国的防火长城已经采用 被动流量分析与主动探测相结合的手段 来检测和封锁Shadowsocks。

现在让我们来讲一下这些词是什么意思。 Shadowsocks是一个加密代理协议， 其设计旨在让协议难以被审查者识别。 这个审查绕过（翻墙）协议在中国非常流行。 而且作为信息管控任务的一部分， 防火长城努力找出并封锁各种类型的代理服务器， 其中包括Shadowsocks。 而事实上，从2019年5月起， 已经有许多来自中国的用户汇报 他们的Shadowsocks服务器被墙。 这些封锁有时发生在政治敏感时期。 但当时对封锁原理都缺乏解释。 这项研究揭示了封锁原理。

首先，在Shadowsocks中， 客户端与服务端之间的连接是加密的。 而且这种加密后， 观察者看到的全部都是密文。 这一点与TLS不同。 TLS的报头中含有明文。 而Shadowsocks流量则全部是密文。 如果你观察就会发现Shadowsocks的流量 就好像一列列均匀随机的字节。 而其就是这么设计的。 这样的设计意味着，你不能仅仅用一个 简单的正则表达来匹配出所有的Shadowsocks流量。 你得更努力一些才行。

现在如果你想流量这样的随机，这样的缺乏特征， 是不是本身就是一种特征呢？ 那你想的绝对正确。 事实上，这项研究揭示了防火长城 使用在TCP流中数据包的长度和熵作为 其识别Shadowsocks流量的第一步。

现在我来解释下刚刚提到的 主动探测是啥意思。 这项研究揭示了： 防火长城识别Shadowsocks流量的过程分两步： 第一步被动，第二步主动。 第一步，防火长城流量分析出 其怀疑是Shadowsocks的连接。 而在第二步中， 防火长城会假装成Shadowsocks客户端， 从自己的IP地址主动去连接它怀疑的服务器。 然后观察被探测的服务器会怎样回应。 你可以把第一步的目的想成“怀疑”， 而第二步的目的则为“确认”。

现在你就能理解了： 主动探测是为了在识别中增加准确率，降低成本。 如果你仅仅用流量分析被动地识别Shadowsocks流量， 那么假阳性可能会高得不可接受。 而另一方面，如果你对每一个经过防火长城的连接， 都进行主动探测的话， 那你又会因为需要发送的 主动探测太多而管不过来。 因此你可以把步骤一想成是对步骤二的预先过滤。

这当然不是第一个记录中国采用 主动探测来识别翻墙协议的研究。 早在2011年，就有研究显示中国以主动探测 的方式来识别Tor和其他VPN协议。 但防火长城用来识别Shadowsocks的技术 可谓又达到了新的高度。

我们又是怎么知道这些的呢？ 你可能已经猜到了文章作者的调查方法。 她们在中国境外搭建了自己的Shadowsocks服务器； 然后又从中国用自己的客户端去 穿过防火长城，连接到搭建好的服务器。 然后再观察还有什么连接 到了她们的服务器上。 她们还搭建了对照服务器， 她们自己的客户端从未连接到上面， 对照服务器只是用来区分主动探测 和一般的互联网扫描。 她们的实验大概持续了4个月。

目前已经有许多许多的Shadowsocks实现了。 而在这项实验中， 作者选用了其中最受欢迎的两个实现： 分别叫做Shadowsocks-libev和Outline。 它们是对同一个协议的两个相互独立的实现。

在为期4个月的实验中得到的一个主要观察是 防火长城会使用各种各不同类型的主动探测： 一部分主动探测是对合法连接的重放， 另一部分则不是。 合法连接可能被储存起来， 然后经过长得惊人的时间才被重放。 另一部分非重放探测的数据包， 有着奇特的长度分布。 主动探测看上去来自上千个不同的源IP地址。

现在我们来谈谈基于重放的主动探测。 首先这些重放基于合法客户端发出的连接。 具体来讲，它们是基于 合法客户端发出的第一个数据包。 有的重放和原连接一模一样， 而有的重放则是在特定位置上 改变了1个，2个或者更多的字节。

那么审查者发送这些重放探测的目的是什么呢？ 原来，审查者是想利用存在于Shadowsocks协议中的某些弱点。 你看，协议并没有规定当一个服务器遇到 重放攻击时应该如何响应。 如果一个实现对重放攻击不做任何的过滤， 那么它就会向代理之前合法客户端发出 的请求一样，去代理重放攻击的请求， 进而将密文的代理流量发送回主动探测者。 虽然审查者因为不知道密码 而无法解密代理流量， 但是服务器向审查者返回了一串密文 这件事本身就足以暴露其为Shadowsocks。

即使是使用了重放过滤的Shadowsocks实现， 仍可能在其服务器关闭连接时的 一些边界情况上暴露信息。 审查者会改变原始连接中一些特定位置上的 字节后再重放，可能是为了绕过重放过滤器。

因为很容易找到重放攻击所基于的原始连接， 所以我们可以测量从合法连接被发送到 其重放攻击被发送的延迟。 请看这幅累积分布函数（CDF）图。 因为同一个合法连接可能被重放多次， 因此暗色线仅表示合法连接第一次被重放的延迟； 而浅色线则表示所有重放的延迟。 如你所见，25%的第一次重放发生在1秒内， 相当于是紧随合法连接之后。 但这幅图还有一个长的惊人的尾巴： 一些重放的延迟是以分钟，小时，甚至是天来计数的。

现在我们来看非重放的主动探测， 这些探测的数据看上去完全随机， 但又并非基于之前的任何合法连接。 你可能发现了这个包长度分布很奇怪。 你看这些长度小于50字节的包， 它们如“三重奏”一般地分布在 8， 12， 16， 22， 33， 41，和49字节。 比如说基于8字节“三重奏”， 包括了数量大致相同的长度为7，8，9字节的探测包。 除此之外，请注意不同类型的非重放探测的比例： 绝大多数的非重放探测的 长度为整整221字节。 这是一个有趣而又发人思考的包长度分布。 论文作者觉得她们至少可以部分地解释

为什么审查者会发送这些长度的探测数据包。 你看，当你向Shadowsock服务器 发送未经认证的随机数据时， 不同的数据长度会引起服务器不同的反应。 如果你发送了过少的数据， 服务器会期待你发送更多的数据， 因此服务器最终会超时。 但当你发送的数据长度超过这一阀值时， 服务器会尝试验证它受到的数据， 而之后又因验证失败而关闭连接。 我不会介绍过多的细节， 但你要知道Shadowsocks 可以配置使用不同的加密方式 而不同加密方式的初始向量长度会不同。

你可以看到，很多的“三重奏”， 长度都是骑跨在服务器不同反应的临界值上的。 服务器的反应包括超时和发送RST关闭连接等。 如第一行所示，如果你向服务器发送了 7或者8字节的数据后，服务器将超时。 但如果你发送了9字节， 那么服务器就会立即发送RST来关闭连接。 因此这个服务器反应可以被识别。 但这个分析并不能完全地 解释“三重奏”的分布。 因为，比如说，长度为32，33，34， 或者长度为40，41，42字节的“三重奏”， 就不位于任何临界值附近。 另外221字节的探测长度也不在临界值上。

现在来讲主动探测的源头。 在为期4个月的实验中， 作者的Shadowsocks服务器收到了 超过50,000次主动探测， 它们来自超过12,000个不同的IP地址。 所有这些IP地址都是中国的。 由上述观察可知，仅仅简单的枚举并屏蔽 所有主动探测的IP地址并不容易。 但这也不是什么令人惊讶的结论， 因为之前的研究工作就早已发现了 审查者会用大量不同的 IP地址来主动探测。 现在我们把这12,000多个在这次研究中 发现的IP地址与之前工作中发现的做对比。 它们之前的重合并不多， 但这也不是很令人惊奇的， 因为之前的研究早已发现 这些主动探测所用的IP地址随时间会发生大量的变化。

虽然这些主动探测看似来自 成千上万的不同IP地址， 但它们很可能是被一小撮进程集中控制的。 证据来自TCP层的旁道信息泄露： TCP时间戳。 TCP时间戳是一个32位的计数器， 其以固定的速率随时间增长。 其被附在每一个(非RST的）TCP包上面。 不同的计算机通常不会有 相同的TCP时间戳序列。 因为这个序列会在系统重启时， 被归零或初始化为一个随机值 这张图显示了来自上千个不同IP地址的 数据包的TCP时间戳随时间的变化， 你可以看到，虽然它们来自不同的IP地址， 但它们都可以被归为很少的几个TCP时间序列。 这些序列的增长速率为250 HZ或1,000 HZ。 那条1,000HZ的线由20个 相距很近的数据点拟合而成。 但可以确定它们的变化率 更接近1000HZ而非250HZ。 TCP时间戳的观察与之前研究中的发现一致。

其他许多不同网络层的指纹 也与之前研究的发现很像。 但TCP源端口号则是一个例外。 先前的工作发现主动探测的TCP源端口号 近乎是均匀随机分布（于1-65535）的。 但在这项研究中， TCP源端口号有很大一部分与 Linux的临时端口相吻合。

现在可以确定防火长城会主动探测 Shadowsocks服务器。 但防火长城是如何怀疑上 Shadowsocks服务器的呢？ 以下两个事实辅助了作者的调查：

1. 重放攻击通常紧随合法连接之后；
2. 重放攻击仅仅基于第一个携带数据的包。

作者因此设计实验， 在建立TCP连接后仅发送一个数据包。 发送的数据包的长度和熵由作者设置。

虽然我们没有在图中看到尖锐的拐点， 但高熵的数据包比低熵的 数据包更有可能被重放。

作者还确定防火长城也会根据 包的长度来怀疑Shadowsocks流量。 请看这幅累积分布函数（CDF）图。 灰线代表作者自己的连接。 她们发送了长度从1到1,000字节 均匀随机分布的数据包。 现在你看，非重放的探测包 的长度集中在221字节。 而绝大部分重放探测包 的长度则在160到700字节之间。 这个区间长度之外的连接极少被重放。 而即使是在这个区间之内， 一些长度的连接也更有可能被重放。 你会发现代表重放探测的这条线呈阶梯状。 事实上，详细结构如下： 对于长度为160到384字节的包， 如果其长度对16取模后为9， 则其更有可能被重放。 对于长度为246到700字节的包， 如果其长度对16取模后为2， 则其更有可能被重放。 这长度在两个区间的重叠区域的包， 其长度对16取模后的结果则以2和9为主。 作者对这一现象没有好的解释， 她们只是发现了这个让人着迷的包长度分布。

那么我们又能怎么缓解 对Shadowsocks的主动探测呢？ 我们已经知道了探测分两步， 那么阻挡其中任何一步就足以挫败探测。 你既可以选择绕过被动地流量分析， 也可以选择绕过主动探测模块。 绕过流量分析就是说去改变会被 防火长城用来分析的特征： 也就是包的熵和长度。 改变包的熵并不容易， 因为这需要根本性地改变协议本身。 而你有更多地余地来改变包的长度。 比如说新版本的Outline会 合并两个连续的包： 这样一来其包的长度分布就可能 不同于防火长城期待的流量特征。

另一个有意思的发现是基于一个叫 Brdgrd(Bridge Guard)的工具. 这是一个你可以安装在 Shadowsocks服务器上的工具。 它可以让客户端发送的每个包的长度变小， 这是通过在每次连接的初始阶段， 改变服务器TCP窗口大小来实现的。 Brdgrd配合Shadowsocks使用 有着种种的缺点和注意事项， 但可以看到在这项实验中， 当Brdgrd被开启时， 主动探测被有效地缓解了， 尽管不是完完全全地阻止了。 你还可以改变服务器对探测的 回应方式，从而避免被识别。

刚刚我给你展示过这个表格， 但我其实撒了一点点谎： 这个表格展示的其实是旧版本的Shadowsocks。 部分因为这项研究的成果， 一些新的Shadowsocks版本， 试图消除超时和关闭连接这样的不一致反应。 新版本的Shadowsocks的反应 更像是这幅图所示的那样。 我不会介绍太多关于AEAD 在新的Shadowsocks协议中的细节， 但你可以看到当在新版Shadowsocks 中使用了推荐的AEAD后， 由于缺乏验证，在使用流加密协议时， 服务器的反应无法做到完全一致地超时。 但开发者已经尽力保持反应一致， 来消除其特征。

总而言之，中国的防火长城已经开始 使用被动的流量分析和主动探测来识别Sadowsocks 主动探测可被连接中的第一个带有数据的包触发。 而且拥有特定长度的包， 和高熵的包更容易触发重放。 与许多种不同的主动探测： 一些是重放攻击，另一些则不是。 主动探测来自许多不同的IP地址， 但有被集中控制的迹象。 可以通过阻断防火长城对Shadowsocks 识别过程中两步的任一一步， 来阻止Shadowsocks被识别。

感谢您的收看。如果您有任何的疑问或评论， 尽情直接与作者们联系。 研究中的源代码和数据在下方链接中。

论文摘要

在这项研究中，我们揭示了中国的防火长城（GFW）是如何检测并封锁Shadowsocks及其变种的。通过网络测量实验，我们发现GFW会根据每个连接中的第一个数据包的长度和熵来识别Shadowsocks流量；然后再向被怀疑是Shadowsocks的服务器分阶段地发送7种不同的主动探测，来验证其怀疑。

我们开发了一个主动探测模拟器，并用它来分析不同的主动探测对不同版本的Shadowsocks的作用，从而猜出不同种的主动探测分别利用了Shadowsocks的哪些弱点。我们还分析了GFW主动探测的指纹，并与之前关于GFW主动探测的研究所发现的指纹比较了异同。分析TCP层的旁道信息，我们还发现：来自上千个不同IP地址的主动探测，其实很有可能是受一小撮中心化结构的集中控制。

根据我们从实验中获得的理解，我们展示了一个成功缓解流量分析的方法。我们还讨论了对抗主动探测所必要的策略。我们向Shadowsocks开发者们负责任地披露了我们的发现和建议，从而已经让Shadowsocks变得更加地难以封锁。

更新

• 我们在不伤害匿名性的前提下，已于2020年10月7日将论文相关的源代码和数据集开源。这些代码和数据可以支持我们论文的所有主要结论。
• 如果您有任何的问题，评论或反馈，请使用下方的评论区，或者用这个pad。

This is the talk for our paper How China Detects and Blocks Shadowsocks.

You can select English or Chinese subtitle by clicking the cc button on the video.

This is “How China Detects and Blocks Shadowsocks”, by GFW Report, Jan Beznazwy and Amir Houmansadr. I’m David Fifield and I’m presenting this work on behalf of the authors, most of whom are anonymous. I have experience researching in this field and the authors have acquainted me thoroughly with this work.

The grand summary of this research is that the Great Firewall of China detects and blocks Shadowsocks using a combination of passive traffic analysis and active probing. And let’s talk about what those terms mean.

Shadowsocks is an encrypted proxy protocol and it’s designed to be difficult to detect. It’s really popular in China as a means of censorship circumvention, a way of getting around the Great Firewall. And the Great Firewall for its part, as part of its general mission of information control tries to find and block all types of different proxy servers, Shadowsocks included. And in fact, since about May 2019, there have been anecdotal reports of people’s Shadowsocks servers being blocked from China sometimes during politically sensitive times, but without a good explanation. This research helps provide an explanation for how this has been happening.

Now in Shadowsocks, the connection between the client and the server is encrypted and furthermore it’s encrypted in a way that it reveals only ciphertext to an observer. So unlike TLS, for example, which has plaintext framing bytes, there’s nothing like that in Shadowsocks. If you flatten out a Shadowsocks stream, it looks like just a sequence of uniformly random bytes and that’s by design. This quality means that it’s not possible to, for example, write a simple regular expression that will match all Shadowsocks traffic, you have to work a little harder than that. Now if you’re thinking that this randomness, this lack of a fingerprint is itself a kind of fingerprints, you’re absolutely right. And in fact this research shows that the Great Firewall uses the entropy and the length of packets in a TCP stream as part of its first step in classifying Shadowsocks traffic.

Now what do I mean by active probing? This research shows that the Great Firewall discovers Shadowsocks servers in a two-step process: the first step is passive and the second step is active. In the first step, it looks for possible or potential Shadowsocks connections; and in the second step, it connects to the servers involved in those connections from its own IP addresses as if it were a Shadowsocks client and watches how the server responds. You can think of step 1 as guess and step 2 as confirm.

Now you can understand this process of active probing as a way of increasing precision or reducing cost in network classification. If you were to write a purely passive classifier for Shadowsocks, it may yield unacceptably high false positives. On the other hand, if you were to try to active probe every single connection that passes through the firewall that may be more probes than you can manage to send. So you can think of step one as being a sort of pre-filter for step two.

Now this is certainly not the first time that active probing has been documented to be used in China against censorship circumvention protocols. There is research going back all the way to 2011, showing it being used against Tor, against various VPN protocols like that. But the level of detection now in using Shadowsocks reaches a new heights of sophistication.

How do we know all this? Well, the authors investigated it in the way you might expect. They ran an experiment, they set up their own Shadowsocks servers outside of China; They set up their own Shadowsocks clients inside China and then they connected to their own servers through the Firewall and watched for what else connected to those same servers. They also set up some control servers and never connected to them, just to be able to distinguish the connection triggered active probes from random internet scanning. And they ran this experiment for about four months. Now there are many many implementations of Shadowsocks out there. For this experiment, the authors chose two of the most popular, which are called Shadowsocks-libev and Outline. These are two independent implementations of the same protocol.

The main observations of the four month server experiment are that active probers send a variety of probe types, some of them look like replay attacks and some of them do not. The ones that are replays may be stored and replayed after a surprisingly long delay. The ones that are not replays have a peculiar distribution of packet lengths. And active probes come from apparently thousands of different source IP addresses.

Let’s talk about the replay based probes. First these are copies of the author’s own legitimate connection from their authenticated Shadowsocks clients. And specifically, they’re copies of the first data packets in an authenticated Shadowsocks connection. Sometimes the replay is identical, sometimes it has certain bytes changed, one or two or maybe a dozen bytes changed, but usually at fixed positions.

So what could be the intention behind sending replay probes? Well, potentially it’s exploiting a vulnerability in the Shadowsocks protocol. See the protocol doesn’t specify what should happen when a server gets a replay of a previous properly authenticated client connection. Now if an implementation doesn’t do any sort of replay filtering, any prevention of replay attacks, what’s likely to happen is that it will do the exact same proxy request that it did earlier for the authenticated client, and send back to the active prober a big blob of ciphertext. Now the active prober won’t be able to decrypt that blob because it doesn’t know the password for that Shadowsocks server. But the fact that it received a large amount of ciphertext back is a giveaway that the server is in fact Shadowsocks. And even in implementations that try to filter out or prevent replays, there are certain edge conditions in how connections are closed, for example, that can be characteristic of Shadowsocks. And the fact that certain bytes are sometimes changed in these replay based probes may be an attempt to evade implementations that have a replay filter.

Replay-based probes are convenient for analysis because it’s easy to match the active probe with the legitimate connection that it is a replay of. It makes it possible to, for example, measure how long the delay is between when a legitimate connection is sent and then replays based on that connection are sent. So take a look at this graph, this is a CDF. Because a probe may be replayed more than once, the darker line here only considers the first replay, and then the paler line considers all replays. And as you can see, for first replays anyway at least around 25 percent of replay probes come within one second, so almost immediately; but there is a surprisingly long tail and some replay probes are sent after a delay of minutes, hours, even days.

Now the non-replay probes: these ones had a payload that was to all appearances random; but didn’t match any prior legitimate connection. And you notice there’s a very strange distribution of packet lengths: looking at the ones of length below 50, you’ll see that they’re roughly uniformly distributed in what I’ll call triplets centered on lengths 8, 12, 16, 22, 33, 41, and 49. So the triplet at 8, for example, that represents a length of 7, a length of 8, and a length of 9. All being about equally likely to be sent. Besides those notice the different scales here, the great majority of the non-replay probes had length exactly 221 bytes, and this is an interesting and thought-provoking distribution of packet lengths.

The authors think they have at least a partial explanation for why active probers send probes of these lengths. You see when you send random unauthenticated data to a Shadowsock server, the server may react differently depending on how much data you send it. So if you send too little data, the server is going to wait to receive the rest of the data that it’s expecting, and eventually timeout. But if you send beyond that threshold, the server will attempt to authenticate the data that it’s received, be unable to authenticate it, and close the connection.

Now I won’t get in too far into the details here, but you can configure Shadowsocks with a variety of different ciphers and initialization vectors of different lengths, and things like that.

But you’ll notice in this table that those triplets many of them straddle what I’ll call byte thresholds, between where the server times out and when it closes the connection with a RST or otherwise. So looking at the first row here, if you send a server so configured a packet of seven bytes or eight bytes, it’s going to time out but if you send it nine bytes, you’ll get an immediate RST.

So that’s a distinguishable difference in how the server reacts. This analysis doesn’t fully explain the triplet distribution, because, for example, the triplet at 32, 33, 34, and the one at 40, 41, 42, don’t match up with any byte thresholds and neither does the 221.

Alright, moving on to the origin of the probers. Over those four months, the authors’ Shadowsock servers received over 50,000 active probes and those came from over 12,000 different IP addresses, which all geolocate to China. So a consequence of this observation is that it’s not possible to simply enumerate all the active prober IP addresses and ban them from your server. It also isn’t surprising because prior research studying active probing has also found large numbers of IP addresses being used to send active probes.

Now comparing the 12,000 IP addresses in this work, with previously compiled lists of prober IP addresses, there is not much overlap although there is some; however this is not really that surprising, because past research has found that there is a lot of churn in the IP addresses used for active probing over time.

Now despite the fact that there seemed to be these thousands and thousands of different active probers, it’s likely that they are all centrally managed by a small number of processes; and the evidence for that comes from a TCP layer side channel, namely the TCP timestamp. So the TCP timestamp is a 32-bit counter that increases at a fixed rate, and it’s attached to every outgoing TCP segments. Different computers will generally not have synchronized TCP timestamp sequences, because it’s going to be relative to usually when the computer was last rebooted, and the counter was reset to zero or initialized to a random value. So this graph shows the TCP timestamp sequences over time, of a few thousand active prober IP addresses in one sub-experiment. And you can see that even though they come from many different IP addresses, they fall into a small number of distinct TCP timestamp sequences and these sequences increase at typical rates so 250 HZ or 1,000 HZ. That 1,000 hertz line goes through a cluster of about 20 data points that are very closely spaced, but within that space they’re much more like 1,000 HZ than 250 HZ. So this TCP timestamp observation is consistent with prior work, as are most of the other network layer fingerprints that you might think to look of.

Look at the exception is TCP source port numbers. Prior work has found a roughly uniform distribution of source port numbers, whereas in this work the authors found a marked bias towards the default ephemeral port range used by Linux.

So it’s clear that active probing of Shadowsocks is a phenomenon. It happens what features is the Great Firewall looking for. The authors investigated this aided by the fact that:

1. replay-based probes are often sent almost immediately and
2. they are copies only of the first data carrying packet.

So the authors designed an experiment to establish a TCP connection and then send one TCP packet with a configurable entropy and a configurable packet length. A configurable payload size and, from this graph, we can see although there isn’t a real sharp distinguishing threshold that high entropy packets are more likely to be replayed than low entropy packets, and the length of the packets matters as well.

So here we have another CDF: the gray line in the back is the author’s own trigger connections and they tested packet lengths between 1 and 1,000 bytes uniformly distributed. Now you can see the non-replay probes there with the expected peak at 221. The replay probes only occur between in the interval of about 160 to 700 bytes lengths. Outside that interval were almost never replayed and even within that interval certain lengths, are more likely to be replayed than others.

So you’ll notice the replay line has a sort of chunky stair-step pattern, and there’s actually some structure to that: so between lengths about 160 to 384, packets were more likely to be replayed if they had a length whose remainder was 9 when divided by 16. And in the interval about 264 to 700, they were more likely to be replayed if they had a length whose remainder was 2 when divided by 16. And in the area where those two intervals overlap, there was a mix of remainders 2 and 9. The authors don’t have an explanation for this phenomenon, it’s just an intriguing feature of the packet length distribution.

Taking active probing of Shadowsocks as a given, what can be done to mitigate it? Well, because we know that the detection process is a two-step process, it is sufficient to disrupt either of those two steps. So you can either evade the passive traffic analysis, or you can invade the active probing components.

Evading the passive traffic analysis means changing the features that the Great Firewall is looking for: so entropy and packet lengths. Changing entropy in Shadowsocks is not easy without kind of fundamentally changing how the protocol works; but with packet lengths, you have a little bit of leeway. And, for example, newer versions of Outline will coalesce consecutive packets: maybe something that would be sent as two packets could send as one packet instead, as a way of disguising the characteristic packet length distribution that the Firewall may be looking for.

Another interesting observation is with a tool called Brdgrd (Bridge Guard). So this is software that you can install on a Shadowsocks server and it causes clients to send smaller than usual packets. When they’re in the early stages of their connection, it does this by rewriting the server’s TCP window size. Although there are some drawbacks and caveats to using Brdgrd with Shadowsocks, it’s clear that here in this experiment while Brdgrd was active, the incidence of active probing is notably diminished, although not quite to zero.

The other thing you can do to avoid detection is changing the way that you respond to active probes. So I showed you this table earlier and it was a little bit of a lie because that table described the behavior of some older versions of Shadowsocks. Some newer versions of Shadowsocks, partially as a result of this research, try to disguise the distinction between timing out a connection and terminating the connection. So their reactions in newer versions of Shadowsocks looks more like this. Now I don’t want to get into the details but the AEAD is the newer, currently recommended version of the Shadowsocks protocol. And you can see in this version, in these two implementations, at least the server always times out, no matter the length of the unauthenticated probe. In this older deprecated stream version of the protocol, for compatibility reasons, it’s not possible to completely eliminate that distinction, but they have done it as far as possible.

In summary, the Great Firewall of China detects Shadowsocks servers using a combination of passive traffic analysis and active probing. Probing is triggered by the first packet in a data connection and it’s more likely when packets have high entropy or have certain payload lengths. There are many different types of active probe: some are replays, some are not. Probes come from many IP addresses but they show signs of being centrally managed and it’s possible to mitigate the effects of active probe into Shadowsocks by disrupting either of the two steps in the classification process.

Thank you for your attention if you have questions or comments, it’s best to get in touch with the authors directly. Source code and data for this research is available at the URL you see.

Explanations on datasets

The open dataset contains code and datasets for the paper: How China Detects and Blocks Shadowsocks.

We introduce the major components of the dataset below.

Paper

The paper directory includes:

1. the CSVs of probe metadata, along with the code that generates CSVs from PCAPs
2. the source code, including Makefile, that reproduces all figures (except Figure 1 and 10) in the paper
3. the source code of latex

pp.sh

code/pp.sh Parallelly Parses all pcap files in a specified directory; then it sorts packets by the timestamps and prints to stdin.

Example usage:

pp.sh path/to/pcap/dir > experiment_name.csv

The script requires tshark and parallel:

apt install tshark parallel # Debian-based
dnf install tshark parallel # Fedora-based

triggering_client_server

code/triggering_client_server includes the source code of the clients and sink/responding servers used in Section 4.1.

Prober simulator

code/prober_simulator includes the source code of the prober simulators introduced in Section 5.1. It can simulate both random and replay-based probes. One can use it to check if other Shadowsocks implementations (or other circumvention tools) have similar vulnerabilities introduced in the paper.

Updates

• As of October 7, 2020, we have released our code and datasets to the maximum extend that does not harm our anonymity. These code and datasets support all major findings in our paper.
• If you have any question, comment or feedback, please feel free to leave them in the comments section or use this pad.
• We will have a summary of our paper on this page. Please check back!

The open dataset contains code and datasets for the paper: Triplet Censors: Demystifying Great Firewall’s DNS Censorship Behavior.

Updates

• If you have any question, comment or feedback, please feel free to leave them on the pad.
• We will have a summary of our paper on this page within days. Please check back!
• As of August 11, 2020, we have released all our code and datasets to the maximum extend that does not harm our anonymity. These code and datasets support all major findings in our paper. We will continue anonymizing and releasing the remaining code and datasets with a goal to make our work highly reproducible within days.

Explanations on datasets

The largest and the most important dataset is ./all_more_fields.csv. This 14 GB file contains 120 millions forged responses injected by the GFW. It is extracted from a set of pcap files across 9 months using:

bash -x extract_all_pcap_to_csv_more_fields.sh

For easier analysis on different injectors of the GFW, we categorize packets sent by different injectors into injector1.csv, injector2.csv, injector3.csv. To generate these files, you need to run one of the two following commands yourself:

# This is the faster way
bash -x split_by_awk_new.sh

or

# This is a more readable but much slower way
python3 split_with.py all_more_fields.csv

README on subdirectories

We categorized the code and datasets supporting different findings in the paper into different subdirectories. There is a README.md file in each subdirectory, explaining what findings are supported by the code and dataset there.

For example, ./delay_differences/README.md reads as:

The code and dataset under this directory were used to support the following findings in our work:
    "We also compare the time between sending our DNS query and when we receive the injected reply to get a sense of wherethe injectors are located. Specifically, we compare the delays of the three injectors and find that more than 90% of the time the delays are within 0.2 ms of each other. This would support the theory that these three devices are installed in the samephysical location."

iyouport于2020年7月30日报告 (存档)中国封锁了带有ESNI扩展的TLS连接。 iyouport称初次封锁见于2020年7月29日。

我们确认中国的防火长城（GFW）已经开始封锁ESNI这一TLS1.3和HTTPS的基础特性。我们在本文中实证性地展示如何触发审查，并研究"残余审查"的延续时长。 我们还将展示7种用Geneva发现的基于客户端或服务端的绕过审查策略。

什么是加密服务器名称指示（ESNI）？

TLS是网络通讯的安全基础（HTTPS）。TLS提供的认证加密使得用户可以确定他们在与谁通讯， 并确保通讯信息不被中间人看到或篡改。 虽然TLS可以隐藏用户通讯的内容，但其并不能总是隐藏与用户通讯的对象。 比如TLS握手可以携带一个叫做加密服务器名称指示（SNI）的扩展, 这个扩展帮助客户端告诉服务器其想要访问的网站的域名。 包括中国在内的审查者利用这一扩展来检查并阻止用户访问特定的网站。

TLS1.3引入了加密SNI（ESNI）。 简而言之就是用加密了的SNI阻止中间人查看客户端要访问的特定网站。 （更多ESNI的益处请见Cloudflare的介绍文章）。 ESNI有让审查HTTPS流量变得更加困难的潜能; 因为不知道用户使用ESNI访问的网站，审查者要么不封锁任何ESNI连接，要么封锁所有的ESNI连接。 我们现在确认中国的审查者选择了后者。

主要结论

• GFW通过丢弃从客户端到服务器的数据包来阻止ESNI连接。
• 封锁可以从GFW内外双向触发。
• 0xffce扩展标识是触发封堵的必要条件。
• 封锁可以发生在1到65535的所有端口上。
• 一旦GFW阻断了一个连接，残留的审查就会继续阻断与（原IP，目标IP，目标端口）三元组相关的所有TCP流量，持续120或180秒。
• 我们已经发现了6种可以部署于客户端和4种可以部署于服务端的规避策略。

我们是怎么知道的？

我们写了一个简单的Python程序，它可以：

1. 完成与指定服务器的TCP握手；
2. 然后发送一个带有ESNI扩展名的TLS ClientHello消息；ClientHello的指纹和Firefox 79.0所发送的一样正常。

我们让程序发送带有ESNI的ClientHello消息。我们既尝试了从墙内向墙外发送，也尝试了从墙外向墙内发送。我们同时采集客户端和服务端两边的流量进行分析，我们确保发送ClientHello的服务器会完成TCP握手，但不会向客户端发送任何数据包，也不会首先关闭连接。所有的实验都在7月30日到8月6日之间进行。

关于封锁的细节

通过丢弃数据包来阻止，而不是注入RST

对比两端捕获的流量，我们发现GFW通过丢弃从客户端到服务器的数据包来阻止ESNI连接。

这与GFW对其他常用协议的阻断方式有两点不同。 首先，GFW审查SNI和HTTP的方式是向服务器和客户端注入伪造的TCP RST。但是我们没有观察到GFW注入任何数据包来阻断ESNI流量。其次，GFW通过丢弃服务器到客户端的流量封锁Tor和Shadowsocks服务器的端口和IP；然而，GFW会丢弃客户端到服务器的ESNI流量。

我们还注意到，GFW在丢弃TCP包时，并不区分TCP包的标志。（这与伊朗的一些审查系统不同，后者不丢弃带有RST或FIN标志的数据包。）

封堵可以双向触发

我们发现封堵是可以双向触发的。 换句话说，从防火墙外向防火墙内发送一个ESNI握手，和从防火墙内向外发送一样，都可以触发审查。

得益于这种双向性，人们可以在不控制任何位于中国的服务器的情况下，从GFW外部远程测试这种基于ESNI的审查。 我们指出，除了ESNI审查外，GFW对DNS、HTTP、SNI、FTP、SMTP和Shadowsocks的审查也可以从墙外进行测量。

GFW对ESNI进行审查，但不封锁没有SNI扩展的ClientHello

我们确认没有ESNI和SNI扩展的TLS ClientHello不能触发封锁。 换句话说，encrypted_server_name 扩展的 0xffce 扩展标识是触发封堵的必要条件。

我们将ClientHello中的0xffce替换为0x7777然后进行测试。替换后，发送这样的ClientHello就不能再触发封锁了。

这个确认是很重要的，因为有人观察到其他审查者可能会屏蔽任何没有SNI扩展的ClientHello消息，这将导致ESNI和无SNI都会被屏蔽。

新的扩展值还未被封锁

riseup pad上的匿名人士指出， 现有的ESNI使用0xffce作为扩展值（详见 Section 8.1）。 但时新的ECH使用0xff02，0xff03和0xff04作为扩展值（Section 11.1). 我们确认这些新的扩展值还未被封锁。

具体而言， 我们把一个能够触发审查的ClientHello的0xffce替换为了0xff02，0xff03和0xff04。 发送修改后的ClientHello不能触发审查。

GFW需要看到一个完整的TCP握手以触发ESNI封锁

我们发现GFW需要看到一个完整的TCP握手以触发ESNI封锁。

我们从外部对中国的服务器进行了两次实验。在第一个实验中，在没有发送任何SYN包的情况下，我们的客户端每2秒发送一个带ESNI扩展的ClientHello消息。在第二个实验中，我们的客户端每2秒发送一个SYN数据包和一个带ESNI扩展的ClientHello消息；但服务器不会响应任何数据包（也不会发送SYN+ACK来完成握手）。

在每次实验中，我们都发送了10条ClientHello消息。结果发现没有触发过任何审查或残余审查。所有的ClientHello消息实际上都到达了服务器。这个结果表明，在触发基于ESNI的审查之前，TCP握手是必要条件。 这也表明，与GFW基于SNI的审查机类似，ESNI的审查机也是有状态的。

封锁发生在所有端口上

我们发现ESNI封锁不仅会发生在443端口，也会发生在1到65535的所有端口。

具体来说，我们从外部向中国服务器的1-65535的每个端口连续发送了两次ESNI握手。 对于每个端口，我们先发送一次ESNI握手；然后在连接超时后（20秒后），我们再尝试与服务器完成一次TCP握手。如果第二次没有收到服务器发来的SYN+ACK，我们就认为该端口被封锁了。 结果是，在1到65535的所有端口上都观察到了对ESNI的封锁。这个特性可以让我们高效地测试ESNI封锁，因为我们可以对同一IP地址的多个端口进行同时测试。

审查残留

我们发现在阻断ESNI握手后，GFW会继续阻断与（源IP，目标IP，目标端口)3元组相关的任何连接一段时间。确切的审查残留时间可能会有所不同。我们观察到它有时持续120秒，有时持续180秒。我们注意到，在残留审查时间内发送额外的ESNI握手不会重置审查持续的定时器。这与之前观察到的基于SNI的拦截GFW的残余审查类似；而且它与伊朗的残余审查不同，在伊朗，定时器将被重置。

这些发现部分基于以下实验。从外部，我们每秒向中国服务器的443端口发送一条ClientHello消息。 第一秒，第二秒和第121秒的TCP握手被接受。其他所有的握手尝试都不成功，因为由于残留的审查，SYN包甚至没有到达服务器。

这个结果表明，与之前发现的GFW对SNI的残留审查类似，GFW也对ESNI也采用了残留审查。此外，第二秒的握手可以完成，意味着GFW至少需要1秒的时间来反应并启用拦截规则。

如何规避封锁？

**Geneva(Genetic Evasion)**是我们中位于马里兰大学的研究人员开发的一种遗传算法，它在进化过程中自动发现新的审查规避策略。 它在不影响原始连接的前提下，通过注入、替换、分割或丢弃数据包流来迷惑审查者。 与大多数反审查系统不同的是，它不需要在连接的两端都进行部署：它只在一方（客户端或服务器）运行。

Geneva利用审查机器来实时地训练自己的遗传算法。 截至今日，其已经找到了许多可以规避不同国家审查的策略。 Geneva的审查规避策略描述了应该如何修改流量。 由于Geneva将不断发展这些策略，因此它们用领域特定语言(domain-specific language)来表达，这些语言构成了每个策略的 “DNA”。(关于Geneva的完整代码及文档，请参见我们的Github页面)。

要了解更多关于Geneva（或Geneva策略引擎）工作原理的信息，请参阅我们的论文或关于页面。

为了让Geneva能够直接利用GFW对ESNI审查进行训练，我们写了一个自定义的插件来执行以下步骤。

1. Geneva在位于墙外的TCP服务器上随机开放一个端口。随机开放端口是为了避免此前审查的残留。
2. Geneva驱动一个位于中国境内的TCP客户端连接到服务器。
3. 客户端发送一个带有加密SNI（ESNI）扩展的TLS 1.3 ClientHello。
4. 客户端休眠2秒，等待GFW审查机制启动。
5. 客户端发送一个简短的测试消息"test"来测试是否已经被审查。
6. 重复步骤4和5。
7. 服务器确认是否收到了客户端发送的完整的TLS ClientHello以及测试消息。如果收到了，该策略就会得到正向的适合度奖励；如果没有收到（或者客户端在发送测试消息时超时了），该策略就会受到惩罚。

利用这个适合度函数，Geneva可以直接针对ESNI审查制度测试和训练策略。 在短短几个小时内，它就发现了多个绕过审查的策略。在本节中，我们将详细描述它们。

Geneva策略引擎在我们的Github上是开源的，所以所有这些策略都可以被任何人部署和使用。由于它们在 TCP 层运行，因此可以应用于任何需要使用 ESNI 的应用程序：随着 Geneva 的运行，即使是未经修改的网络浏览器也可以成为一个简单的审查规避工具。

请注意，Geneva不是通用的翻墙软件，也不提供任何额外的加密、隐私或保护。它是一个测试版的研究原型，而且它并没有针对速度进行优化。使用这些策略，风险自担。

策略

我们在48小时的时间里，从客户端和服务器端对Geneva进行了训练。我们总共发现了6种策略来打败对ESNI的封锁机制。其中有4个可以在服务器端使用，所有6个都可以在客户端使用。

以下是TCP层的策略，只需部署在客户端，即可击败针对ESNI的封锁。

策略 1: 三重 SYN

第一种客户端策略的工作原理是用三个SYN数据包启动TCP的三次握手，这样第三个SYN的序列号就是错误的。

在Geneva的语法中，这个策略是这样的：

[TCP:flags:S]-duplicate(duplicate,tamper{TCP:seq:corrupt})-| \/

这种策略是对于GFW的跳出同步攻击（synchronization attack）。GFW会转而追踪这个新的错误序列号，而错过ESNI请求。

这个策略也可以部署在服务端：

[TCP:flags:SA]-tamper{TCP:flags:replace:S}(duplicate(duplicate,tamper{TCP:seq:corrupt}),)-| \/

虽然这种策略使得服务器永远不会发送SYN+ACK数据包，但这并不会破坏三次握手。在三次握手过程中，服务器没有像往常一样发送一个SYN+ACK数据包，而是发送三个SYN数据包（第三个数据包的校验和是错误的）。

第一个SYN数据包的作用是启动TCP的同步打开（Simultaneous Open），这是所有主流操作系统都支持的一个古老的TCP功能，用于处理两个TCP栈同时发送SYN数据包的情况。当客户端收到服务器发送的SYN时，客户端发送一个 SYN+ACK 数据包，服务器响应一个 ACK 来完成握手。这样就有效地将传统的三次握手改为四次握手。带有损坏序列号的SYN使GFW跳出同步（但被客户端忽略），成功地击败了封锁机制的同时并不伤害到原有连接。

策略2：四字节分割

我们发现的下一个策略也可以从客户端或服务器使用。在这个策略中，客户端将ESNI请求分为两段TCP发送，第一段TCP的长度小于或等于4个字节。

客户端策略的Geneva语法是这样的： [TCP:flags:PA]-fragment{tcp:4:True}-| \/

这已经不是Geneva第一次发现分段策略，但令人惊讶的是，这种策略在中国竟然有效。毕竟防火长城拥有著名的TCP重建能力已经有近十年了（见brdgrd）。TLS头是5个字节长，所以我们推测，像这样将TLS头分割到多个数据包中，打破了GFW将包含ESNI的数据包识别为TLS的能力。这对GFW如何通过指纹识别连接有有趣的影响：它表明GFW中负责识别的组件并不能从TCP层面中重组所有的上层协议。这一理论得到了Geneva过去所发现的其他基于分段的策略的支持（详见这篇论文）。

这个策略也可以从服务器端触发。通过在三次握手期间减少TCP窗口大小，服务器可以强制客户端分割请求。在Geneva的语法中，可以通过以下方式实现：[TCP:flags:SA]-tamper{TCP:window:replace:4}-| \/

策略3：TCB Teardown

下一个策略是经典的TCB（TCP Control Block）Teardown：客户端向连接中注入一个带有错误的校验和的RST数据包。这将欺骗GFW，使其认为连接已被中断。

在Geneva的语法中，这个策略看起来是这样的： [TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:RA}(tamper{TCP:chksum:corrupt},))-| \/

TCB Teardowns并不是什么新鲜事：Khattak et al.在几乎十年前就已经演示过了，Geneva过去也多次发现了针对GFW的Teardown攻击。

令人惊讶的是，这种策略也可以从服务器端诱导。 在三次握手过程中，服务器可以发送一个带有错误ACK的SYN+ACK数据包，从而诱导客户端发送RST。这将导致RST的序列号不正确（ACK为0，但仍足以引起TCB Teardown）。

策略4：FIN+SYN

下一个策略似乎是另一种角度的跳出同步攻击。在这一策略中，客户端（或服务器）发送一个数据包，在三次握手过程中同时设置 FIN 和 SYN。 客户端的Geneva语法是[TCP:flags:A]-duplicate(tamper{TCP:flags:replace:FS},)-| \/ 服务端的Geneva语法是 [TCP:flags:SA]-duplicate(tamper{TCP:flags:replace:FS},)-| \/

在过去，我们发现GFW在对其他协议进行审查时，对FIN数据包有特殊的处理。 似乎FIN的出现会使GFW立即同步到对当前连接的跟踪，但SYN的出现会使它认为实际的序列号与实际值相差+1，使GFW与实际连接偏离1。

我们对以上假设进行了测试：在这个策略运行的时候，将客户端实际请求的序列号增加1后，连接被阻断了。

在服务端，这个策略不需要FIN就可以运行。

策略5：TCB Turnaround

TCB Turnaround策略很简单：在客户端发起三次握手之前，首先向服务器发送一个SYN+ACK数据包。SYN+ACK使让GFW混淆了客户机和服务器的角色，从而使客户端能够不受阻碍地进行通信。TCB Turnaround攻击在哈萨克斯坦仍然有效，但在GFW上，TCB Turnaround攻击对其他协议不起作用。

在Geneva的语法中：[TCP:flags:S]-duplicate(tamper{TCP:flags:replace:SA},)-| \/

该策略只适用于客户端，因为当SYN数据包到达服务器时，GFW已经知道哪一方是客户端了。

策略6：TCB 跳出同步

最后，Geneva发现了简单的基于载荷的TCB跳出攻击。从客户端，注入一个带有载荷和错误的校验和的数据包就足以使GFW跳出对当前连接的同步。 Geneva在研究GFW对其他协议的审查时，已经发现过这一策略了。

Geneva的语法是这样的 [TCP:flags:A]-duplicate(tamper{TCP:load:replace:AAAAAAAAAA}(tamper{TCP:chksum:corrupt},),)-|

这个策略不能在服务器端使用。

对审查规避策略的总结

我们已经找到6种可以部署在客户端，和4种可以部署在服务端的审查规避策略。 每一种都有近乎100%的可靠性，并可以用于规避针对ESNI的审查。 遗憾的是，这些策略并非是一劳永逸的：就像猫捉老鼠一般，防火长城也会继续提升其审查能力。

未解决的问题

我们仍不清楚为何会观察到不同的残余审查时长。 与所有类似研究一样，在与我们所用节点不同的中国地区，可能存在着不同于我们所观察到的审查方式。 如果你观察到了不同于上述的审查方式，或者我们的审查规避策略对你并不奏效， 尽请联系我们。

鸣谢

我们想在此感谢所有在riseup留言板上提问，反馈和建议的匿名人士。 这些评论帮助我们给予社区所最关心的问题更高的优先级，并从而加速了我们的研究。

我们还感谢OONI和OTF社区所有人的支持。

联系我们

Geneva 团队:

• Kevin Bock (PGP key)
• Dave Levin (PGP key)

GFW Report:

• Anonymous (PGP key)
• Amir Houmansadr (PGP key)

iYouPort:

• onoketa (PGP key)

这篇报告首发于censorship.ai。我们在iyouport.org，gfw.report，net4people和ntc.party上同步更新了这篇报告。

On 2020-07-30, iyouport reported (archive) the apparent blocking of TLS connections with the encrypted SNI (ESNI) extension in China. iyouport says that the first occurrence of blocking was one day earlier, on 2020-07-29.

We confirm that the Great Firewall (GFW) of China has recently begun blocking ESNI—one of the foundational features of TLS 1.3 and HTTPS. We empirically demonstrate what triggers this censorship and how long residual censorship lasts. We also present several evasion strategies discovered by Geneva that can be run either client-side or server-side to evade blocking.

What is Encrypted Server Name Indication (ESNI)?

TLS is the foundation of secure communication on the web (HTTPS). It provides authenticated encryption so that users can know with whom they are communicating, and that their information cannot be read or tampered with by an intermediary. Although TLS hides the content of a user’s communication, it does not always hide with whom the user is communicating; the TLS handshake optionally contains a Server Name Indication (SNI) field that allows the user’s client to inform the server which website it wishes to communicate with. Nation-state censors have used the SNI field to block users from being able to communicate with certain destinations. China, for one, has long been censoring HTTPS in this manner.

TLS 1.3 introduced Encrypted SNI (ESNI) that, put simply, encrypts the SNI so that intermediaries cannot view it. (To learn more about ESNI and its benefits, see Cloudflare’s article). ESNI has the potential to complicate nation-states’ abilities to censor HTTPS content; rather than be able to block only connections to specific websites, ESNI would require censors to block all TLS connections to specific servers. We do confirm that this is now happening in China!

Our Main Findings

• The GFW blocks ESNI connections by dropping packets from client to server.
• The blocking can be triggered bidirectionally.
• The 0xffce extension is necessary to trigger the blocking.
• The blocking can happen on all ports from 1 to 65535.
• Once the GFW blocks a connection, it will continue blocking all traffic associated with the 3-tuples of (srcIP, dstIP, dstPort) for 120 or 180 seconds.
• We have discovered 6 client-side and 4 server-side evasion strategies.

How Do We Know These?

We have made a simple Python program that performs the following:

1. completes a TCP handshake with a specified server;
2. and then sends a TLS ClientHello message with an ESNI extension; the fingerprint of the ClientHello is as normal as what Firefox 79.0 would send.

The program sends ClientHellos with ESNI both inside-out and outside-in, while capturing traffic on both sides for analysis. The servers to which we send ClientHellos complete the TCP handshake, but they do not send any data packets back to the client, nor do they are first to close the connection. All experiments were conducted between July 30th and August 6st.

Details About the Blocking

Blocking by dropping packets, not injecting RSTs

Comparing the traffic captured on both endpoints, we find the GFW blocks ESNI connections by dropping packets from clients to servers.

This has two differences from how the GFW censors other commonly-used protocols. First, the GFW censors (non-encrypted) SNI and HTTP by injecting forged TCP RSTs to both server and client; conversely, we have observed no injected packets from the GFW to censor ESNI traffic. Second, the GFW drops traffic from server to client to block Tor and Shadowsocks servers; however, it drops only client-to-server packets when censoring ESNI.

We further note the GFW does not distinguish the flags of TCP packets when dropping them. (This is different from some censorship systems in Iran which do not drop packets with RST or FIN flags.)

The blocking can be triggered bidirectionally

We find the blocking can be triggered bidirectionally. In other words, sending an ESNI handshake from outside the firewall to inside can get blocked in the same way as sending it inside-out.

Thanks to this bidirectional feature, one can test this ESNI-based censorship remotely from the outside of the GFW without having control of any Chinese server. The GFW’s censorship on DNS, HTTP, SNI, FTP, SMTP, and Shadowsocks can also be measured outside-in.

The GFW censors ESNI, but not omit-SNI

We confirm a TLS ClientHello without ESNI/SNI extensions cannot trigger the blocking. In other words, the 0xffce payload of the encrypted_server_name extension is necessary to trigger the blocking.

We tested this by replacing the 0xffce in a triggering ClientHello with 0x7777. After the replacement, sending such a ClientHello could not trigger the blocking anymore.

This confirmation is important because some censors have been observed blocking any ClientHello message without the SNI extension, which would result in the blocking of both ESNI and omitting-SNI.

New extension values are not blocked

As informed by an anonymous reviewer on the riseup pad, the currently deployed ESNI uses extension value 0xffce (see Section 8.1). However, the newer ECH uses extension value 0xff02, 0xff03 and 0xff04(Section 11.1). We confirm no censorship has been observed on these extension values yet.

Specifically, we replace the 0xffce in a triggering ClientHello with the values of 0xff02, 0xff03, and 0xff04 respectively. And no blocking is observed after sending such modified ClientHellos.

A complete TCP handshake is required before triggering the blocking

We find a complete TCP handshake is necessary in order to trigger the ESNI blocking.

We conducted two experiments from the outside to a server in China. In the first experiment, without sending any SYN packet, our client sent one naked ClientHello message with ESNI extension every 2 seconds. In the second experiment, our client sent a SYN packet and a ClientHello message with ESNI extension; but the server would not respond with any packet (not even to complete the TCP three-way handshake).

In total, we sent 10 ClientHello messages in each experiment. The result shows no blocking or residual censorship was ever triggered; all ClientHello messages reached the server. This means a TCP handshake is necessary before triggering ESNI-based censorship. It also indicates, similar to the SNI-based censorship by the GFW, the censorship machine for ESNI is stateful.

The blocking can happen on all ports

We find the ESNI blocking can happen not only on port 443, but on all ports from 1 to 65535.

Specifically, we sent two ESNI handshakes in a row to the port 1-65535 of a Chinese server from the outside. For each port, we first sent an ESNI handshake; then after the connection timeout (after 20 seconds), we tried to complete a TCP handshake with the server again. If we do not receive any SYN+ACK from the server the second time, we consider the censorship occurred on that port. As a result, the ESNI blocking was observed on all ports from 1 to 65535.

This feature allows us to test ESNI censorship efficiently, as we can conduct testings on multiple ports of the same IP address simultaneously.

Residual Censorship

We find that the GFW employs “residual censorship” of ESNI connections. This means that, for some amount of time after triggering censorship for a given connection, it will continue blocking any connections with the same 3-tuple of source IP, destination IP, and destination port.

The precise duration of residual censorship appears to vary by vantage point. We observed residual censorship for 120 seconds at two of our vantage points, and 180 seconds at another vantage point.

Sending additional ESNI handshakes during residual censorship time does not reset the timer of the censoring machine. This is similar to the previously observed residual censorship on SNI-based blocking of the GFW. (Conversely, each additional packet set while residual censorship in effect in Iran resets the timer.)

These findings are partially based on the following experiment. From the outside, we sent one ClientHello message per second to port 443 of a Chinese server. The 1st, 2nd, and 121st TCP handshakes were accepted. All other handshake attempts were unsuccessful because the SYNs did not reach the server.

This result shows, similar to previously discovered SNI-based residual censorship, the GFW also employs residual censorship for ESNI. In addition, the fact that second handshake could complete means that it takes at least 1 second for the GFW to react and enable the blocking rules.

How Can We Circumvent the Blocking?

Geneva (Genetic Evasion) is a genetic algorithm developed by those of us at the University of Maryland that automatically discovers new censorship evasion strategies. Geneva manipulates packet streams—injecting, altering, fragmenting, and dropping packets—in a manner that bypasses censorship without impacting the original underlying connection. Unlike most other anti-censorship systems, Geneva does not require deployment at both sides of the connection: it runs exclusively at one side (client or server).

Geneva trains its genetic algorithm against live censors, and to date has found dozens of censorship evasion strategies in various countries. Geneva’s strategies are expressed in a domain-specific language. Details of the language, along with the entire Geneva codebase, are available at the Geneva GitHub repository.

To learn more about how Geneva (or the Geneva strategy engine) works under the hood, see our papers or about page.

To allow Geneva to train directly against the GFW’s ESNI censorship, we wrote a custom plugin that performs the following steps:

1. Geneva starts a TCP server on a random open port on a vantage point located outside of China. By randomizing our ports, we do not need to worry about residual censorship.
2. Geneva drives a TCP client located inside of China to connect to the server.
3. The client sends a TLS 1.3 ClientHello with the Encrypted SNI extension.
4. The client sleeps for 2 seconds to allow the GFW censorship to kick in.
5. The client sends a short test message "test" to test if it has been censored.
6. Steps 4 & 5 are repeated.
7. The server confirms that it receives both the full TLS ClientHello from the client and the test messages. If it does, the strategy is rewarded with a positive fitness; if not (or if the client timed out while sending its test messages), the strategy is punished.

With this, Geneva discovered multiple evasion strategies in just a few hours. We describe them in detail below.

The Geneva strategy engine is open source on our Github.

All of these strategies can be run with our open-source Geneva strategy engine (repository). Since they operate at the TCP layer, they can be applied to any application that needs to use ESNI: with Geneva running, even an unmodified web browser can become a simple censorship evasion tool.

Note that Geneva is not designed as a general purpose evasion tool, and does not provide any additional encryption, privacy, or protection. It is a research prototype and it is not optimized for speed. Use these strategies at your own risk.

Evasion strategies

We trained Geneva over the span of 48 hours, both client- and server-side. In total, we discovered 6 strategies to defeat the ESNI censorship: 4 that work from the server, and 6 that work from the client.

The following are TCP-layer strategies that can defeat the ESNI censorship when applied exclusively at the client-side.

Strategy 1: Triple SYN

The first client strategy works by initiating the TCP 3-way handshake with three SYN packets, such that the sequence number of the third SYN is corrupted.

In Geneva’s syntax, this strategy looks like this: [TCP:flags:S]-duplicate(duplicate,tamper{TCP:seq:corrupt})-| \/

This strategy performs a desynchronization attack against the Great Firewall. The GFW synchronizes on the corrupt sequence number, so it misses the ESNI request.

This strategy can also be applied from the server-side:

[TCP:flags:SA]-tamper{TCP:flags:replace:S}(duplicate(duplicate,tamper{TCP:seq:corrupt}),)-| \/

Although this strategy makes it so the server never sends a SYN+ACK packet, this does not break the three-way handshake. During the three-way handshake, instead of the server sending a SYN+ACK packet as usual, the server instead sends three SYN packets (the third with a corrupt sequence number).

The first SYN packet serves to initiate a TCP Simultaneous Open, an archaic feature of TCP supported by all major operating systems to handle the case in which two TCP stacks send a SYN packet at the same time. When the client receives a SYN from the server, the client sends a SYN+ACK packet, and server responds with an ACK to complete the handshake. This effectively changes the traditional three-way handshake to a four-way handshake. The SYN with the corrupt sequence number causes the GFW to desynchronize (but is ignored by the client), successfully defeating censorship without harming the connection.

Strategy 2: Four Byte Segmentation

The next strategy we discover can also be used from client or server. In this strategy, the client sends the ESNI request across two TCP segments, such that the first TCP segment is less than or equal to 4 bytes long.

From the client-side, in Geneva’s syntax this strategy looks like this: [TCP:flags:PA]-fragment{tcp:4:True}-| \/

This is not the first time Geneva has discovered segmentation strategies, but it is surprising that this strategy works in China. The Great Firewall has been famous for its ability to reassemble TCP segments for almost a decade now (see brdgrd). The TLS header is 5 bytes long, so by segmenting specifically the TLS header across multiple packets, we hypothesize this breaks the GFW’s ability to protocol fingerprint ESNI packet as TLS. This has interesting implications for how the GFW fingerprints connections: it suggests the component of the GFW that performs connection fingerprinting cannot reassemble TCP segments for all protocols. This theory is supported by other segmentation-based strategies identified by Geneva in the past (see this paper).

This strategy can also be triggered from the server-side. By reducing the TCP window size during the 3-way handshake, a server can force the client to segment their request. In Geneva’s syntax, this can be accomplished with: [TCP:flags:SA]-tamper{TCP:window:replace:4}-| \/.

Strategy 3: TCB Teardown

The next strategy is a classic TCB (TCP Control Block) Teardown: the client injects a RST packet with a broken checksum into the connection. This tricks the GFW into thinking the connection has been torn down.

In Geneva’s syntax, this strategy looks like: [TCP:flags:A]-duplicate(,tamper{TCP:flags:replace:RA}(tamper{TCP:chksum:corrupt},))-| \/

TCB Teardowns are not new: they were demonstrated almost a decade ago by Khattak et al., and Geneva has discovered Teardown attacks repeatedly in the past against the GFW.

Surprisingly, this strategy also can be induced from the server-side. During the three-way handshake, the server can send a SYN+ACK packet with a corrupt acknowledgement number, thereby inducing the client to send a RST. This causes the RST to have an incorrect sequence number (and an acknowledgement number of 0, but it still is sufficient to cause a TCB Teardown.

Strategy 4: FIN+SYN

The next strategy appears to be another desychronization attack, but via a different attack vector. In this strategy, the client (or the server) sends a packet with the FIN and SYN flags both set during the three-way handshake. For the client, in Geneva’s syntax: [TCP:flags:A]-duplicate(tamper{TCP:flags:replace:FS},)-| \/ For the server, in Geneva’s syntax: [TCP:flags:SA]-duplicate(tamper{TCP:flags:replace:FS},)-| \/

In the past, we’ve found the GFW against other protocols has special handling for FIN packets when it comes to resynchronization. In this case, it looks like the presence of the FIN causes the GFW to immediately resynchronize, but the presence of the SYN causes it to think the actual seqno is +1 from the actual value, making the GFW off by 1 from the real connection.

We tested this hypothesis by incrementing the sequence number of the actual request by 1 while this strategy was running, and saw that the client got censored.

From the server-side, the FIN flag is not required for this strategy to work.

Strategy 5: TCB Turnaround

The TCB Turnaround strategy is simple: before the client initiates the three-way handshake, it first sends a SYN+ACK packet to the server. The SYN+ACK causes the GFW to confuse the roles of the client and server, thereby allowing the client to communicate unimpeded. TCB Turnaround attacks still work in Kazakhstan, but turnaround attacks do not work against the GFW for any other protocols.

In Geneva’s syntax: [TCP:flags:S]-duplicate(tamper{TCP:flags:replace:SA},)-| \/

This strategy is client-only, since by the time the SYN packet arrives at the server, the censor already knows which side is the client.

Strategy 6: TCB Desynchronization

Finally, Geneva identified simple payload-based TCB desynchronization. From the client, injecting a packet with a payload and a broken checksum is sufficient to desynchronize the GFW from the connection. Geneva has identified these in the past against the GFW’s censorship of other protocols as well.

In Geneva’s syntax: [TCP:flags:A]-duplicate(tamper{TCP:load:replace:AAAAAAAAAA}(tamper{TCP:chksum:corrupt},),)-|

This strategy cannot be used from the server-side.

Summary on Circumvention Strategies

In total, we have discovered 6 strategies that work from the client-side, and 4 that work from the server-side. Each of these works with near 100% reliability, and can be used to evade the ESNI censorship. Unfortunately, these specific strategies may not be a long-term solution: as the cat and mouse game progresses, the Great Firewall will likely to continue to improve its censorship capabilities.

Unresolved Questions

It is not yet clear why we observe different durations of residual censorship from different vantage points. As with all such research, it is also possible that there are some regions of China that are affected in different ways than our vantage points. If you observe different behavior or that some of our evasion strategies do not work, please feel free to contact us!

Thanks

We want to thank all anonymous reviewers who offered us valuable and immediate questions, feedback and suggestions on the riseup pad. These comments guided us to prioritize the questions that interest the community the most; and thus greatly accelerated our research.

We are also thankful to the OONI and OTF community for all of their support.

Contacts

Geneva team:

• Kevin Bock (PGP key)
• Dave Levin (PGP key)

GFW Report:

• Anonymous (PGP key)
• Amir Houmansadr (PGP key)

iYouPort:

• onoketa (PGP key)

This report first appeared on censorship.ai. We maintain an up-to-date copy of the report on iyouport.org, gfw.report, net4people and ntc.party.

This report first appeared on GFW Report. We also maintain an up-to-date copy of the report on both net4people and ntc.party.

We encourage you to share your comments publicly or privately. Our private contact information can be found at the footer of GFW Report.

Several weaknesses were discovered in the V2Ray recently, which could be used to identify V2Ray clients or servers that run VMess, TLS or HTTP protocol. Below is our summary and understanding on these weaknesses.

In general, these weaknesses fall into three categories:

• Inappropriate authentications in VMess, making the servers vulnerable to replay attacks.
• Hardcoded unique ciphersuites, leading to the rarely-seen fingerprints of the TLS ClientHello messages.
• Failed attempt to parrot/mimic the HTTP server.

Replay Attacks against the VMess Protocol

As introduced in the specification (English version) of the VMess protocol, a VMess request looks like this:

• The 16-byte Authentication Credential is a HMAC associated with the user ID and a UTC timestamp.
• The Command is encrypted using AES-128-CFB(iv, key), where the iv is the md5 hash value of the UTC timestamp, and key is the preshared one associated with user ID.

The following table shows the structure of Command after decryption:

• The Encryption IV and the Encryption Key are used to decrypt Data, not Command.
• The Margin P and Random Value are used as a padding scheme. Specifically, the 4-bit Margin P specifies the length of the Random Value to be between 0 and 15 bytes.
• The Checksum F, serving as a MAC, should be the FNV1a hash of all plaintext in Command, excluding itself.

Inappropriate authentication

On May 31, 2020, @p4gefau1t reported that VMess servers could be identified by replay-based active probing, due to the inappropriate authentications.

VMess authenticates each request in two steps, using Authentication Credential and checksum. Unfortunately both of them can be circumvented.

First, the VMess server validates whether the timestamp in Authentication Credential is expired. The expiration time is 120 seconds at maximum and 60 seconds on average (see here and here for implementation details). That is to say, an attacker can record and replay a legitimate Authentication Credential within around 60 seconds to bypass this authentication.

Second, since the aes-cfb used to encrypt the Command does not provide any authentication, a MAC-then-Encrypt mechanism is used. As pointed out by @p4gefau1t, VMess fell into the same pitfall as Shadowsocks OTA mode did (See the English summary on the weakness of Shadowsocks OTA mode here). Specifically, since the length of the Random Value varies, the server will not be able to know where the Checksum F (MAC) is located, unless it blindly trusts the value in Margin P without any authentication (see here for implementation details). In other words, only after reading P+4 bytes, V2Ray will be able to validate whether the decrypted content is legal. If not legal, the V2Ray server will close the connection.

VMess server indeed has a replay defense mechanism. In particular, the server records the (Encryption IV, Encryption Key) of each request, regardless of the validity of the requests; and close the connection immediately when the (Encryption IV, Encryption Key) is seen before. Depending on her needs, an attacker can:

1. bypass this replay defense by alternating the (ciphertext of) Encryption IV or Encryption Key.
2. or intentionally trigger the replay defense to expect the inconsistency behaviors of the servers when seeing the same (Encryption IV, Encryption Key) first time and more times.

Exploiting these weaknesses, many replay-based probes are creatively crafted to identify the VMess server. We introduce them below in separate sections.

Replays with padding length field changed

Based on @p4gefau1t’s findings, @studentmain proposed and @p4gefau1t improved an attack to identify the VMess servers. For simplicity, below we present this attack in a slightly different way.

The malicious probe is a replay of the legitimate request, with many bytes changed as follows:

The Malicious Incomplete Command includes:

In total, the attacker makes 16 connections to the server. In each connection, the attacker:

1. first sends a replay of the first 16 + 41 bytes of the legitimate connection, with both the last byte of the Encryption Key and the 4-bit Margin P changed to a value different from the ones in other connections;
2. then sends M bytes of zero (or random) data one byte per second, until the server closes the connection.

If the Ms recorded among 16 connections happen to be a list of non-repeated integers with the delta of max and min is 15, then it is very likely that the server runs VMess protocol.

The explanations of the attack are as follows:

• To circumvent the authentication based on Auth Info, the attacker replays an Auth Info sent by the legitimate client in around 60 seconds.
• To circumvent the replay defense based on (Encryption IV, Encryption Key), the attacker uses a different value of the Encryption Key in each connection.
• To avoid the bit errors propagating to the Margin P, the attacker carefully chooses the last byte of the Encryption Key to alter. This is because this byte happens to be within the same 16-byte cipher block as the Margin P. (Note that, the bit error propagation of AES-128-CFB works as follows: changing a bit in cipher block Ci, will change 1) the specific corresponding bit in plaintext block Pi; 2) as well as the Random bit errors in all subsequent blocks.)
• The attacker then exploits the malleability of the stream cipher to enumerate all possible values of the 4-bit Margin P in 16 connections.
• After reading the 16+41 bytes, the server waits for the Address, Paddings and Checksum before closing the connection due to checksum error. Thus, the M measured here is actually N-byte address + P-byte padding + 4-byte checksum.
• The attacker can thus infer the value of Margin P from M because the Paddings is the only field with varied length. (The length of the Address is a fixed value, because the address type is not changed.)

Replays that trigger inconsistent draining behaviors

After the patches to defeat the probes above, @nametoolong found two more types of replay-based probes that can still detect the VMess servers. Both of them are related to how the server closes the connection. Below, we introduce the first of them, and we leave the explanations of the second attack as an exercise to reader.

@nametoolong described the probes and the behaviors of the server as follows:

    Vector 1:
    Let M1 be the first 54 bytes of a valid session.
    Let M2=M1. Tamper with M2[48] (i.e. alter the 49th byte of M2).
    Replay M1. Connection is closed immediately.
    Replay M2. Connection is not closed.
    Replay M2 again. Connection is closed immediately.

The byte 48 (counting from 0) that got changed is the last byte of the Encryption Key.

In this attack, the attacker intentionally triggers the replay defense, and expects the inconsistent behaviors of the servers when seeing the same (Encryption IV, Encryption Key) for the first time and for more times. The detailed explanations are as follows:

1. Since the (Encryption IV, Encryption Key) in M1 is the same as the one in the legitimate connection, the server will detect this replay attack and thus close the connection immediately.
2. When it is the first time to send M2, since the server has never seen the altered (Encryption IV, Encryption Key), it will bypass the replay defense. The server thus waits for more bytes to come, rather than close the connection.
3. When it is the second time to send M2, since the server has seen the same (Encryption IV, Encryption Key) before, the server will close the connection immediately.

The V2Ray has actually been patched so that it will close the connections after reading a random number of bytes within a certain range, or after waiting for a random amount of time within a certain range. However, this attack is possible because of the inconsistent usage of the draining methods when different types of errors happen.

@nametoolong thus suggested:

    Drain the connection on all types of errors.
    It still needs to be considered whether draining the connection itself is a attack vector.

Our comments

Although we do not know whether the GFW uses active probing against VMess protocol, the attacks proposed above are feasible to the GFW. For example, it is observed that the GFW is capable of sending replay-based probes with no delay or arbitrarily long delay. We will investigate whether the GFW uses active probing against VMess protocol in the following work. At the same time, it will save us a lot of time if users can report which V2Ray servers were blocked when using what settings.

It may be a good idea to use a replay defense mechanism for the auth info that is based on both expiration time and nonce. On one hand, V2Ray uses a replay defense mechanism based on expiration time. It will thus consider a replay sent within the expiration time as valid. On the other hand, Shadowsocks-libev uses a replay defense mechanism based on nonce. But it requires the servers to remember these nonces forever until the key is changed. This seems to be complicated to implement as it should even still remember the nonce after a reboot of the software. Therefore, a replay defense mechanism based on both expiration time and nonce may be a good choice.

Frolov et al. found that various popular circumvention tools, including obfs4, Shadowsocks Outline, Psiphon’s OSSH and Lantern’s Lampshade, can be identified using the TCP flags and timing information when the servers close the connections. Frolov et al. thus suggested that servers should “forever read” on errors, so that the probers will be the first to close the connection. This way, it not only reduces the information leaked by server’s timeout value, but also lets the server close the connection with FIN/ACK consistently (see Fig. 1 here for more details).

Unique TLS ClientHello Fingerprints

On May 30, 2020, @p4gefau1t reported V2Ray clients would send TLS ClientHello messages with very unique fingerprints. Such unique fingerprints not only gave a censor the opportunity to identify the V2Ray clients and servers, but also allowed a censor to accurately block the TLS traffic by V2Ray without much collateral damage.

@p4gefau1t further identified that these unique fingerprints were partially caused by the use of a hardcoded ciphersuite. Specifically, this rarely seen ciphersuite would be used, when the AllowInsecureCiphers flag was its default value false.

V2Ray developer @xiaokangwang mitigated this weakness by using the default settings of go-tls library since v4.23.4 (see patches #2510, #2512, #2518). @tomac4t summarized a form, comparing the ClientHello fingerprints before and after the patches using tlsfingerprint.io. However, the fingerprints seem to be still quite unique.

To our best knowledge, as early as November, 2019, @klzgrad had already investigated the fingerprints of V2Ray v4.21.3 as well as many other TLS-based circumvention tools. The result shows most of them have rarely seen TLS ClientHello fingerprints.

Side notes:

• As summarized in the Client Hello Fingerprinting section, many works have used ClientHello messages to fingerprint different TLS implementations. Frolov et al. discovered that the TLS ClientHello fingerprints of many popular circumvention tools were very unique (see Table 2 for more details). Frolov et al. thus developed utls and created tlsfingerprint.io.
• @p4gefau1t investigated this issue because @rickyzhang82 demonstrated a machine learning model that can identify the TLS traffic by V2Ray with 0.9999 accuracy. The same model, without additional training, could not accurately identify the new TLS traffic of V2Ray after the developers made changes to the fingerprint.
• @DuckSoft demonstrated that the blocking based on TLS ciphersuites can be written in one line of iptables rules.

Failed to Mimic the HTTP Server

On June 2, 2020, @p4gefau1t reported the V2Ray failed to mimic real HTTP communications. In particular, the two reported issues are:

1. Both V2Ray clients and servers will prepend a HTTP header only to the first TCP payload they send in each connection, making the mimicking traffic easy to be detected.
2. V2Ray servers use a hardcoded 500 response for various types of failures, making the mimicking server easy to be distinguished by active probes.

Since the parrot is dead since 2013, instead of reviving the parrot, using a real HTTP engine may be a more promising solution here. Many circumvention tools have been using the idea of application fronting, which include forwardproxy, naiveproxy and trojan.

Credits

All credit goes to the authors of the corresponding works.

Thanks

We want to thank @studentmain and @p4gefau1t for helping us understand their proposed replay attacks, and for sharing their inspiring thoughts on the future works. We are also grateful to David Fifield and @studentmain for offering detailed feedback on a draft of this summary.

Contacts

This report first appeared on GFW Report. We also maintain an up-to-date copy of the report on both net4people and ntc.party.

We will investigate whether the GFW uses active probing against VMess protocol in the following work. At the same time, it will save us a lot of time if you, as a user, can report which circumvention services were blocked when using what settings. We encourage you to share your comments publicly or privately. Our private contact information can be found at the footer of GFW Report.

近期数个V2Ray的弱点被发现。这些弱点可以被用来识别使用VMess、TLS或HTTP协议的V2Ray客户端和服务器。 以下是我们对这些弱点的总结和理解。

总体上，这些弱点可分为三类：

• VMess服务器没有正确验证客户端的请求，使得服务器可受到重放攻击。
• 客户端硬编码了一套罕见的TLS密码套件，导致客户端发送的TLS ClientHello拥有几乎独一无二的指纹。
• 伪装成HTTP服务器的企图失败。

针对VMess协议的重放攻击

VMess协议的请求构造如下：

• 认证信息 是一个16字节的HMAC，基于用户 ID和UTC 时间辍.
• 指令部分 由AES-128-CFB(iv， key)算法加密， 其中iv 是UTC 时间辍的md5哈希值， key是V2Ray用户自己设置的那个密码。

下表为指令部分解密后的结构：

• 数据加密 IV和数据加密 Key是用来加密/解密数据部分的，不是用来加密/解密指令部分的。
• 余量 P和随机值是填充机制的一部分。其中余量 P占4位，用来表示随机值的长度。长度取值在0到15字节之间。
• 校验 F是一个MAC。在合法的请求中，它的值应该是除自己以外所有指令部分的FNV1a哈希值。

验证客户端请求时的弱点

2020年5月31日，@p4gefau1t报告，由于客户端请求的合法性没有得到恰当的认证，VMess服务器可以被基于重放攻击的主动探测识别。

VMess服务器分两步，分别用认证信息和校验 F来鉴定客户端请求的合法性。不幸的是，这两步都可以被攻击者绕过。

第一步， VMess服务器验证包含在认证信息中的时间辍是否已经过期。 保质期最长为120秒，平均值为60秒。其具体的实现细节请见这里还有这里。 这也就是说， 攻击者可以先记录下正常客户端发出的合法的认证信息，然后只要在60秒左右的时间内，在自己的恶意请求中使用这段合法的认证信息，就可以通过服务器第一步的验证。

第二步， 由于用来加密指令部分的aes-cfb算法本身不提供数据认证， VMess协议使用了一个MAC-then-Encrypt的机制校验数据的真实性和完整性。 @p4gefau1t指出， VMess协议掉入了与Shadowsocks OTA模式当年同样的陷阱。(英文版的Shadowsocks OTA模式漏洞请见这里。) 具体来讲， 由于随机值的长度在每个请求中是变化的， 因此服务器被迫在还没有验证余量 P的真实性之前就盲目的信任这个值。 然后服务器才能确定校验 F (MAC)在请求中的位置。 (具体实现细节请见这里) 也就是说， 在未经校验的情况下读取了P+4字节后， V2Ray才能开始验证请求的真实性和完整性。 如果验证未通过， V2Ray服务器就会断开连接。

VMess服务器确实有一个对抗重放攻击的机制。 不管请求是否合法， 服务器都会记录下每个请求中使用的数据加密 IV-数据加密 Key对； 如果一个请求中的数据加密 IV-数据加密 Key对已经在之前的请求中使用过了， 服务器就会立即断开连接。 因此取决于其需要，攻击者可以对这个对抗重放攻击的机制做两件事：

1. 攻击者可以修改密文中对应数据加密 IV或数据加密 Key的部分，从而绕过这个对抗重放攻击的机制。
2. 攻击者可以故意触发这个对抗重放攻击的机制，然后观察对于同一个数据加密 IV-数据加密 Key对，服务器在第一次和第n>1次见到它时反应是否不同。

利用以上弱点， 各种基于重放攻击的，针对VMess服务器的主动探测被创造出来。 我们现在对这些攻击按类别进行介绍。

修改填充长度的重放攻击

基于@p4gefau1t发现的VMess弱点和攻击， @studentmain提出了一种更强的攻击来识别VMess服务器。这种攻击后又被@p4gefau1t再次修改增强。 为了叙述简洁， 我们用一种稍有不同的方式呈现它。

该主动探测的载荷基于对合法客户端发送的密文请求的修改，其构造如下：

恶意修改的指令部分的结构如下:

攻击者在一次攻击中，共向服务器发起16次连接。在每次连接中攻击者：

1. 首先记录下某一合法请求的前16+41字节。
2. 然后将其中对应数据加密 Key的最后一字节和对应余量 P的部分修改为与原来不同的值，并发送。注意要保证不同连接中修改后的值都不一样。
3. 最后每隔一秒发送一字节的零（或随机）数据，直到服务器主动断开连接。记发送的零（或随机）数据的字节长度为M。

如果16次连接中记录的M没有重复且最大值和最小值的差为15，那么被探测的服务器就很有可能是在使用VMess协议。

对于该攻击的解释如下：

• 为了绕过服务器基于认证信息的校验，攻击者在大约60秒的时间内重复使用同一个来自合法客户端的认证信息。
• 为了绕过服务器基于数据加密 IV-数据加密 Key对的过滤器， 攻击者修改了每次连接中的数据加密 Key。
• 为了避免修改数据加密 Key后错误扩散到余量 P，攻击者精心的选择了只修改数据加密 Key的最后一字节。因为这一字节与余量 P同属于一个16字节的密码块。（AES-128-CFB算法的错误扩散是这样的： 改变密文块Ci中的任意1位，将会 1）改变其对应明文块Pi中对应的1位；2)并可能随机的改变其之后所有明文块中的任意位。）
• 攻击者利用流加密算法的malleability特性， 在16次连接中，通过遍历余量 P所有的密文空间来遍历其所有的明文空间。
• 在读取16+41字节后， 服务器会期待客户端发送N字节的地址 A、P字节的填充以及4字节的校验 F。 因此M的实际值为N+P+4。
• 攻击者因此可以利用M猜出每次连接中余量 P的明文值。 （因为地址类型 T是不变的，所以地址 A占用的长度为固定值。）

触发服务器不同反应的重放攻击

@nametoolong 发现了另外两种重放攻击，可以用来识别即使修复了上述问题的VMess服务器。

这两种攻击都与服务器何时及如何关闭连接有关。 我们在这里仅介绍第一种攻击，并把第二种攻击的原理留给读者作为练习。

@nametoolong描述了第一种攻击载荷和其期待引起服务器反应：

    攻击 1：
    M1为一个合法连接的前54个字节。
    M1修改第48字节，其余字节保持不便，记作M2。
    发送M1，服务器会立即断开连接。
    发送M2，服务器不会立即断开连接。
    再次发送M2，服务器会立即断开连接。

注：被替换的第48字节（从0开始数的）即为的数据加密 Key的最后一个字节。

在这次攻击中， 攻击者故意触发重放防御机制， 并期待着服务器在第一次与更多次见到同一个数据加密 IV-数据加密 Key对时， 会有不同的反应。 具体介绍如下：

1. 因为已经在之前的合法连接中见过M1中的数据加密 IV-数据加密 Key对， 所以服务器会检测出M1是重放攻击，并立即断开连接。
2. 当第一次发送M2时，服务器是第一次见到其包含的数据加密 IV-数据加密 Key对， 所以服务器不会认定M2是重放攻击，因此会继续等待攻击者发送更多的字节，而不断开连接。
3. 当第二次发送M2时，服务器因为已经见过同样的数据加密 IV-数据加密 Key对， 所以服务器会认定这次的M2是重放攻击，并且立即断开连接。

V2Ray已经将断开连接时所需的时长和所需接收的字节数随机化， 但由于不是在遇到所有类型的错误时都统一采用了这一机制， 因此给了攻击者可乘之机。

@nametoolong因此建议：

    断开连接的机制要保持统一。
    但也需要考虑断开连接这一行为本身是否会泄露什么特征。

我们的评论

证据显示，以上提到的攻击以GFW现有的能力来讲是切实可行的。 比如说，GFW被观察到可以记录合法客户端的连接，并在0.4秒到数百小时之间的延迟后，将修改过的（或未修改过的）载荷发送给被怀疑的服务器。

下一步，我们将调查GFW是否已经使用了针对V2Ray的主动探测攻击。与此同时，如果你有任何翻墙服务器被封锁，我们都欢迎你或公开的或私下的与我们分享你的配置。因为这会帮助我们快速定位许多问题的根源。

也许同时基于实效性和重复性的请求验证是抵抗GFW重放攻击的好办法。 一方面， V2Ray在认证信息中仅使用了基于实效性的重放攻击防御措施。 但这会导致在一定时间内的任何重放攻击都是有效的。 另一方面， Shadowsocks-libev使用了一个基于nonce的重放过滤器。 但为了让仅基于nonce的重放过滤器有效检测重放攻击， 服务端被不现实的要求（在主密钥更换之前）必须要一直记住所有合法连接中的nonce，即使是服务端重启之后！ 因此， 同时基于实效性和重复性的防御机制似乎是抵抗GFW重放攻击的更有效办法。

Frolov et al.发现, 包括obfs4, Shadowsocks Outline, 赛风的OSSH和蓝灯的Lampshade在内的许多翻墙协议或工具都可以被服务器断开连接的方式所识别。 具体而言，不同的主动探测可能导致服务器以不同的TCP包或不同的时长来断开连接。 Frolov et al.给出的建议是让服务器在遇到错误时“永远读取buffer”。 直到攻击者自己主动断开连接。 这样做一方面减少了超时信息的泄漏， 另一方面也使得服务器只会使用FIN/ACK断开连接， 而不用RST和FIN/ACK的混合方式（原理请见Fig. 1）。

独特的TLS ClientHello指纹

2020年5月20日，@p4gefau1t 报告V2Ray客户端发送的TLS ClientHello有着非常独特的指纹。 这样独特的指纹不但给了审查者怀疑和识别V2Ray客户端和服务端的机会， 而且还允许审查者在不造成大量误伤（collateral damage）的情况下精确的封锁V2Ray产生的TLS流量。

@p4gefau1t进一步识别出这些独特的指纹是由一套硬编码的密码套件造成的。 具体而言， 当AllowInsecureCiphers的值为默认的false时， 那套硬编码的密码套件就会被使用。

V2Ray开发者@xiaokangwang让V2Ray自v4.23.4起，使用go-tls库默认的设置来缓解这一弱点 （更多补丁细节请见 #2510，#2512和#2518。 @tomac4t总结了一个表格， 里面使用tlsfingerprint.io来比较不同版本及配置下V2Ray使用的ClientHello的指纹的流行程度。 但结果显示， 即使是已经使用go-tls库默认设置后的指纹， 在实际的流量统计中似乎仍是很少见的。

据我们所知， 在2019年11月， @klzgrad就曾调查过V2Ray v4.21.3 以及其他基于TLS的翻墙工具的ClientHello指纹。 其结果显示， 很多工具在当时被调查的版本中的ClientHello有着罕见的指纹。

旁注：

• 如Client Hello Fingerprinting section所总结的， 许多研究工作已经使用ClientHello的指纹来识别TLS的实现。 Frolov et al.发现包括赛风，蓝灯及Snowflake在内的许多翻墙软件，都（曾经）有着不同的指纹（详情见这里的Table 2）。 Frolov et al.因此开发了utls并创建了tlsfingerprint.io。
• @p4gefau1t最开始调查这个问题是因为@rickyzhang82使用的一个机器学习模型可以以0.9999的准确率识别V2Ray的TLS流量。 在不加更多训练的情况下，这个模型不能准确的识别使用了新ClientHello指纹的V2Ray TLS流量。
• @DuckSoft展示了如何使用一行iptables规则来封锁掉使用某一密码套件的所有TLS流量。

伪装成HTTP服务器的企图失败

2020年6月2日，@p4gefau1t 报告 V2Ray没能成功模仿真正的HTTP通讯。 汇报了的问题有两个：

1. V2Ray客户端和服务端都只会在第一个TCP数据包中加入HTTP头部。这种奇特的TCP连接容易被检测和怀疑。
2. V2Ray服务器对任何请求错误都笼统的回复一个硬编码的500回复。

介于2013年起鹦鹉已死， 与其尝试复活鹦鹉， 不如改用真正的HTTP引擎。 现在的许多翻墙软件已经使用了应用前置（application fronting）的概念，这些软件包括但不限于forwardproxy， naiveproxy 和trojan。

贡献

文中提到的一切贡献、成果均属于该工作的原作者。

致谢

我们想在此感谢@studentmain和@p4gefau1t。 他们帮助我们理解了他们所提出的重放攻击的许多细节，并与我们分享了他们对于下一步工作的建议。 我们还想感谢David Fifield和@studentmain对于这篇总结给出的详细反馈和建议。

联系

这篇报告首发于GFW Report。我们还在net4people和ntc.party同步更新了这篇报告。

下一步，我们将调查GFW是否已经使用了针对V2Ray的主动探测攻击。与此同时，如果您有任何翻墙服务器被封锁，我们都欢迎您或公开的或私下的与我们分享您的配置。因为这会帮助我们快速定位许多问题的根源。我们私下的联系方式可见GFW Report的页脚。

I came across a one-liner script by @gfwrev and got seriously impressed by it. Although it does not work anymore, I still would like to have a writeup on it for its beauty and for the author’s creativity.

The one-liner named gfw-looking-glass.sh is as follows:

while true; do printf "\0\0\1\0\0\1\0\0\0\0\0\0\6wux.ru\300" | nc -uq1 $SOME_IP 53 | hd -s20; done

As shown in the figure below, it was able to print out part of the memory of the GFW. But how?

nc

nc -uq1 $SOME_IP 53 sends input from stdin to the port 53 of $SOME_IP as a UDP packet. As explained by @gfwrev, $SOME_IP can be any host that 1) does not response to DNS query on port 53 and 2) is on the other side of the GFW (meaning if the query is sent from China, $SOME_IP should be outside of China). Requirement 1 makes sure any response was from the GFW, rather than the destination host; while requirement 2 makes sure the well-crafted DNS query would be seen by the GFW.

Background

A little bit background on DNS format and DNS compression pointer can be very helpful to understand this exploitation.

General DNS Format

Below is the general format of DNS queries and responses:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         Identification        |              flags            |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|      number of questions      |      number of answer RRs     |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     number of authority RRs   |    number of additional RRs   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                            questions                          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                 answers(varaible number of RRs)               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                anthority(varaible number of RRs)              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         additional information(varaible number of RRs)        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Questions Field Format

The format of questions field is as follows (:

 0                   1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           query name          |
\                               \
|                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           query type          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           query class         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Query Field Format

The query field of a query of www.google.com can be represented as follows. The

 0                   1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|3| www |6|   google  |3| com |0| 1 | 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

When compression pointer is used, one example is as follows:

 0                   1
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|3|  www|1|1|           offset          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

We can see www is followed by a two-byte pointer, whose two higher-order bits are turned on. The 14 bits after the two high-order bits in the pointer are offset. When offset == n, it points to the nth byte of DNS query message.

Explainations on the crafted DNS query

We now take a closer look at this well-crafted DNS query:

printf "\0\0\1\0\0\1\0\0\0\0\0\0\6wux.ru\300" | xxd -b -c 4

00000000: 00000000 00000000 00000001 00000000  ....
00000004: 00000000 00000001 00000000 00000000  ....
00000008: 00000000 00000000 00000000 00000000  ....
0000000c: 00000110 01110111 01110101 01111000  .wux
00000010: 00101110 01110010 01110101 11000000  .ru.

The first 12 bytes is just a typical DNS query where:

• ID=0;
• Recursion Desired Flag is set;
• number of questions = 1;
• number of answer RRs = 0;
• number of authority RRs = 0;
• number of additional RRs = 0.

The most interesting part is in the questions field from byte 12 to 19.

I first thought \6wux.ru was a typo, which was supposed to be \3wux\2ru. But then I realized \6wux.ru was intentionally used to demonstrate how GFW parses the query name. In particular, although \6wux.ru does not follow the query name format, the fact it could equivalently trigger the GFW as what \3wux\2ru could do suggested the GFW “converted query name to string before pattern matching”.

As introduced in the background section, a pointer takes 2 bytes. However, the crafted query has only 1 byte of the pointer. This incomplete pointer caused the GFW treating the following byte in the buffer as part of the offset. It can be inferred the offset in this query ranges from 0 to 2^8-1 and when the offset was greater than the DNS query length, the GFW would jump out of the DNS query and treat some bytes in its memory as part of the domain name. The GFW seemed not to validate if the offset is smaller than the DNS query length.

Now that the GFW has included its memory as part of the query name, all we have to do is to trigger the GFW to send a forged DNS response. @gfwrev used wux.ru as the kw{rnd} like keyword in this query. Note different keyword patterns are summarized in the Table 2 (b) of this paper.

Explainations on the forged response

After receveing the forged DNS response, hd -s20 helps to truncate the first 20 bytes of it. The 20 bytes contain 12 bytes of the fields before questions field and the first 8 bytes of the questions field: \6wux.ru\300.

The parts that are not truncated are 1) what in GFW’s memory 2) followed by a forged answers field. Taking the first hex dump in the screenshot above as one example, the 2) forged answers field is:

c0 0c 00 01 00 01 00 00 01 2c 00 04 cb 62 07 41

• c0 0c represents the domain name, it is essentially a pointer to the query name in the question field.
• 00 01 and 00 01 tells the answer type is A and class type is 1 respetively. This is interesting because back to the time gfw-looking-glass.sh worked, the type 1 DNS responses forged by the GFW would set answer type to A and class type to 1 regardless of the query type and query class type.
• 00 00 01 2c sets the DNS TTL to 300 seconds.
• 00 04 specifies the data length is 4.
• cb 62 07 41 is the forged IP address: 203.98.7.65.

Excluding 2) the bytes for answers field, we thus know 1) the bytes in GFW’s memory.

One thing interesting is the length of the questions field in these forged responses. The questions field started with 8 bytes \6wux.ru\300 and was followed by 122 bytes GFW memory: cb 9e ... 65 61. Interestingly, the hexdump of both exploits in the screenshot have a questions field of exactly 130 bytes. Since the maximum length of a domain name and a label of domain name are 253 bytes and 63 bytes respectively, I conjectured 130 bytes was an artifitial limitation set by the GFW for each question name.

Sidenote

• Story after gfw-looking-glass.sh In November 2014, @gfwrev found GFW “gave up checking the pointers entirely”. @gfwrev thus created a new way to evade DNS detection by crafting compression pointers. The testing results in November 2014 showed both V2EX and Google DNS servers could successfully parse those crafted DNS queries containing pointers.

• Why was pointer left incomplete? One intuitive question to ask is whether it was possible to check certain relative location of GFW device’s memory by specifying an offset greater than the query message length ourselves. Since the GFW has given checking the pointers entirely, it cannot be tested anymore.

• kw{rnd} like keywords I tested a few previously known kw{rnd} like keywords, but none of them can still trigger the GFW as March 2020.

• Conjectured 130 bytes domain name limitation As of March 2020, GFW can prase and generate forged responses with maximum length. One can test it by:

dig $(python -c "print( 'a.'*121 + 'twitter.com')") @"$SOME_IP"

Credits

GFW Report did not contribute in any step of this work. All credit goes to @gfwrev.

Contacts

This report first appeared on GFW Report. We also maintain an up-to-date copy of the report on both net4people and ntc.party.

We encourage you to share your comments publicly or privately. Our private contact information can be found at the footer of GFW Report.