浏览器缓存与nginx缓存
浏览器缓存
优点：使用有效缓存时，没有网络消耗，速度快；即使有网络消耗，但对失效缓存使用304响应做到网络消耗最小化
缺点：仅提升一个用户的体验
nginx 缓存
优点：提升所有用户体验，相比浏览器缓存，有效降低上游服务的负载，通过304响应减少nginx与上游服务间的流量消耗
缺点：用户仍然保持网络消耗
同时使用浏览器与nginx缓存
Etag 头部
ETagHTTP 响应头是资源的特定版本的标识符。可以是缓存更高效，并节省带宽，因为如果内容没有改变。web服务不需要发送完整的响应。如果内容发生变化，使用ETag有助于防止资源的同时更新互换覆盖；如果给定URL中的资源更改,则一定要生成新的Etag值。因此Etge类似于指纹，也可能被某些服务器用于追踪。比较etags能快速确定此资源是否变化，但也可能被追踪服务器永久保存
W/可选
‘W/’（大小写敏感）表示使用弱验证器，弱验证器很容易生成不利于比较，强验证器是比较的理想选择，但很难生成，相同资源的两个弱Etag值可能语义相同，但不是每个字节都相同
etag 指令
Syntax: etag on | off; Default: etag on; Context: http, server, location
生成规则
ngx_sprintf(etag->value.data, ""%xT-%xO"", r->headers_out.last_modified_time, r->headers_out.content_length_n)
If-None-Match
If-Modified-Since 头部
not_modified过滤模块
expires 指令
Syntax: expires [modified] time; expires epoch | max | off; #指的一个绝对时间，表示缓存在这段时间内一直有效 Default: expires off; Context: http, server, location, if in location
配置
[root@python vhast]# cat cache.conf
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on; 启用
expires 1h;缓存1小时
#expires -1h;
#expires @20h30m;
#if_modified_since off;
#proxy_cache two;
#proxy_cache_valid 100 10m;
#add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
#proxy_pass http://127.0.0.1:8012;
}
}
测试
[root@python vhast]# date -u 2019年 07月 17日 星期三 16:12:16 UTC [root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:12:19 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Wed, 17 Jul 2019 17:12:19 GMT Cache-Control: max-age=3600 Accept-Ranges: bytes
配置
[root@python vhast]# cat cache.conf
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on;
#expires 1h;
expires -1h; #设置前一小时失效
#expires @20h30m;
#if_modified_since off;
#proxy_cache two;
#proxy_cache_valid 100 10m;
#add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
#proxy_pass http://127.0.0.1:8012;
}
}
测试
[root@python vhast]# date -u 2019年 07月 17日 星期三 16:16:37 UTC [root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:16:48 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Wed, 17 Jul 2019 15:16:48 GMT Cache-Control: no-cache #缓存失效 Accept-Ranges: bytes
配置指的浏览器缓存到指定的时间
[root@python vhast]# cat cache.conf
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on;
#expires 1h;
#expires -1h;
expires @20h30m; 缓存到最近的20点半
#if_modified_since off;
#proxy_cache two;
#proxy_cache_valid 100 10m;
#add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
#proxy_pass http://127.0.0.1:8012;
}
}
测试
[root@python vhast]# date -u 2019年 07月 17日 星期三 16:18:54 UTC [root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:18:57 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=72663 Accept-Ranges: bytes
not_modified 过滤模块
not_modified 指令
Syntax: if_modified_since off | exact | before; Default: if_modified_since exact; Context: http, server, location
测试
[root@python vhast]# curl cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:47:10 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=70970 Accept-Ranges: bytes [root@python vhast]# curl -H 'If_Modified_Since: Wed, 10 Jul 2019 18:23:02 GMT' -H 'If-None-Match: "5d262d06-264"' cache.com/ -I HTTP/1.1 304 Not Modified#命中缓存 Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:47:22 GMT Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=70958 [root@python vhast]# curl -H 'If_Modified_Since: Wed, 10 Jul 2019 18:23:02 GMT' -H 'If-None-Match: "5d262d06-267"' cache.com/ -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 16:48:12 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT Connection: keep-alive ETag: "5d262d06-264" Expires: Thu, 18 Jul 2019 12:30:00 GMT Cache-Control: max-age=70908 Accept-Ranges: bytes
If-Match
If-Unmodified-Since
nginx 缓存：定义存放缓存的载体
Syntax: proxy_cache zone | off; Default: proxy_cache off; Context: http, server, location Syntax: proxy_cache_path path [levels=levels] [use_temp_path=on|off] keys_zone=name:size [inactive=time] [max_size=size] [manager_files=number] [manager_sleep=time] [manager_threshold=time] [loader_files=number] [loader_sleep=time] [loader_threshold=time] [purger=on|off] [purger_files=number] [purger_sleep=time] [purger_threshold=time]; Default: — Context: http
proxy_cache_path 指令
缓存关键字
Syntax: proxy_cache_key string; Default: proxy_cache_key $scheme$proxy_host$request_uri; Context: http, server, location
缓存什么样的响应
Syntax: proxy_cache_valid [code ...] time; Default: — Context: http, server, location
那些内容不使用缓存
参数为真时响应不存入缓存
Syntax: proxy_no_cache string ...; Default: — Context: http, server, location
参数为真时，不使用缓存内容
Syntax: proxy_cache_bypass string ...; Default: — Context: http, server, location
变更HEAD方法
Syntax: proxy_cache_convert_head on | off; Default: proxy_cache_convert_head on; Context: http, server, location
upstream_cache_status变量
upstream_cache_status
MISS：未命中缓存； HIT：命中缓存； EXPIRED：缓存已过期；STALE：命中陈旧的缓存；UPDATING:内容陈旧，正在更新；REVALIDATED：nginx验证陈旧的内容依然有效 ；BYPASS：内容是从原始服务器获得
缓存流程
对那个method方法使用缓存返回响应
Syntax: proxy_cache_methods GET | HEAD | POST ...; Default: proxy_cache_methods GET HEAD; Context: http, server, location
配置
server {
listen 8012;
default_type text/plain;
root html;
location /{
#add_header X-Accel-Limit-Rate 10;
}
location /test {
return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
';
}
}
[root@python vhast]# cat cache.conf
proxy_cache_path /data/web/cache levels=2:2 keys_zone=two:10m loader_threshold=300 loader_files=200 max_size=200m inactive=1m;
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location /{
etag on;
#expires 1h;
#expires -1h;
#expires @20h30m;
#if_modified_since off;
proxy_cache two;
proxy_cache_valid 200 1m;
add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
proxy_pass http://127.0.0.1:8012;
}
}
测试
[root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:23:40 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: MISS 第一次 Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:23:44 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT 第二次 Accept-Ranges: bytes
X-Accel-Expires头部
配置
server {
listen 8012;
default_type text/plain;
#client_body_in_single_buffer on;
#add_header Cache_Control 'max-age=3,stale-while-revalidate=3';
#add_header Vary *;
add_header X-Accel-Expires 3; #定义代理服务器缓存为3秒
root html;
location /{
#add_header X-Accel-Limit-Rate 10;
}
location /test {
return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
';
}
}
测试
[root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:20 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: EXPIRED Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:21 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:22 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:23 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: HIT Accept-Ranges: bytes [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:35:25 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" X-Cache-Status: EXPIRED Accept-Ranges: bytes
vary 头部
配置
server {
listen 8012;
default_type text/plain;
#client_body_in_single_buffer on;
#add_header Cache_Control 'max-age=3,stale-while-revalidate=3';
add_header Vary *; #配置代理不缓存
add_header X-Accel-Expires 3; #定义上游服务器缓存为3
root html;
location /{
#add_header X-Accel-Limit-Rate 10;
}
location /test {
return 200 '8012 server response.
uri: $uri
method: $request_method
requset: $request
http_name: $http_name
';
}
}
测试
[root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:38:23 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" Vary: * Accept-Ranges: bytes X-Cache-Status: MISS [root@python vhast]# curl cache.com/a.txt -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 17:38:23 GMT Content-Type: text/plain Content-Length: 23 Connection: keep-alive Last-Modified: Fri, 12 Jul 2019 14:07:41 GMT ETag: "5d28942d-17" Vary: * Accept-Ranges: bytes X-Cache-Status: MISS
Set-Cookie头部
Set-Cookie: <cookie-name>=<cookie-value> Set-Cookie: <cookie-name>=<cookie-value>; Expires=<date> Set-Cookie: <cookie-name>=<cookie-value>; Max-Age=<non-zero-digit> Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value> Set-Cookie: <cookie-name>=<cookie-value>; Path=<path-value> Set-Cookie: <cookie-name>=<cookie-value>; Secure Set-Cookie: <cookie-name>=<cookie-value>; HttpOnly Set-Cookie: <cookie-name>=<cookie-value>; SameSite=Strict Set-Cookie: <cookie-name>=<cookie-value>; SameSite=Lax Set-Cookie: <cookie-name>=<cookie-value>; Domain=<domain-value>; Secure; HttpOnly
缓存流程：接收上游的响应
合并回源请求-减少峰值流量下的压力
Syntax: proxy_cache_lock on | off; Default: proxy_cache_lock off; Context: http, server, location 同一时间，仅第一个请求发往上游，其他请求等待第一个响应返回或者超时候，使用缓存响应客户端 Syntax: proxy_cache_lock_timeout time; Default: proxy_cache_lock_timeout 5s; Context: http, server, location 等待第一个请求返回响应的最大时间，到达后直接向上游发送请求，但不缓存响应 Syntax: proxy_cache_lock_age time; Default: proxy_cache_lock_age 5s; Context: http, server, location 上一个请求返回响应的超时时间，到达后再放行一个请求发送向上游
减少回源请求试图stale陈旧的缓存
Syntax: proxy_cache_use_stale error | timeout | invalid_header | updating | http_500 | http_502 | http_503 | http_504 | http_403 | http_404 | http_429 | off ...; Default: proxy_cache_use_stale off; Context: http, server, location Syntax: proxy_cache_background_update on | off; Default: proxy_cache_background_update off; Context: http, server, location
proxy_cache_use_stale指定陈旧缓存的用法
缓存有问题时的响应
Syntax: proxy_cache_background_update on | off; Default: proxy_cache_background_update off; Context: http, server, location
当使用proxy_cache_use_stale允许使用过期响应时，将同步生成一个子请求，通过访问上游服务更新过期的缓存
Syntax: proxy_cache_revalidate on | off; Default: proxy_cache_revalidate off; Context: http, server, location
更新缓存时，使用If-Modified-Since和If-None-Match作为请求头部，预期内容未发生变更时通过304来减少传输的内容
及时清除缓存
模块第三方模块ngx_cache_purge 地址 https://github.com/FRiCKLE/ngx_cache_purge
使用--add-module=指令编译添加到nginx中
功能：接收到指定HTTP请求后立刻清除缓存
•syntax: proxy_cache_purge on|off|<method> [from all|<ip> [.. <ip>]] •default: none •context: http, server, location •syntax: proxy_cache_purge zone_name key •default: none •context: location
编译进第三方模块
git clone https://github.com/FRiCKLE/ngx_cache_purge.git
cd nginx-1.15.9/
./configure --prefix=/data/web --sbin-path=/usr/bin --user=nginx --group=nginx --with-http_stub_status_module --with-http_auth_request_module --with-http_sub_module --add-module=/root/nginx-http-concat --with-http_addition_module --with-http_secure_link_module --with-http_geoip_module --with-http_ssl_module --add-module=/root/ngx_cache_purge
make
mv /usr/bin/nginx{,.01.18.15.41}
cp objs/nginx /usr/bin/
配置
[root@python vhast]# cat cache.conf
proxy_cache_path /data/web/cache levels=2:2 keys_zone=two:10m loader_threshold=300 loader_files=200 max_size=200m inactive=1m;
server {
server_name cache.com;
error_log logs/cacgeee.log;
access_log logs/cache.log main;
root html/;
location ~/purge(/.*) {
proxy_cache_purge two $scheme$1; 匹配上的时候，就清空这个缓存
}
location /{
etag on;
#expires 1h;
#expires -1h;
#expires @20h30m;
#if_modified_since off;
proxy_cache two;
proxy_cache_valid 200 1m;
add_header X-Cache-Status $upstream_cache_status;
#proxy_cache_use_stale error timeout updating;
#proxy_cache_revalidate on;
#proxy_cache_background_update on;
#proxy_hide_header Set-Cookie;
#proxy_ignore_headers Set-Cookie;
#proxy_force_ranges on;
proxy_cache_key $scheme$uri;
proxy_pass http://127.0.0.1:8012;
}
}
测试
[root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:30 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: MISS #未命中 Accept-Ranges: bytes [root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:31 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: HIT #命中 Accept-Ranges: bytes [root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:35 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: HIT #命中 Accept-Ranges: bytes [root@python vhast]# curl cache.com/purge/index.html -I #清除缓存 HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:40 GMT Content-Type: text/html Content-Length: 270 Connection: keep-alive [root@python vhast]# curl cache.com/index.html -I HTTP/1.1 200 OK Server: nginx/1.15.9 Date: Wed, 17 Jul 2019 20:16:42 GMT Content-Type: text/html Content-Length: 612 Connection: keep-alive Last-Modified: Wed, 10 Jul 2019 18:23:02 GMT ETag: "5d262d06-264" X-Cache-Status: MISS #未命中 Accept-Ranges: bytes
七层反向代理对照：构造请求内容
七层反向代理对照：建立连接并发送请求
七层反向代理：接收上游响应
转发响应
七层代理SSL
缓存类的指令
七层反代独有配置
