有时我们希望记录特定的请求头信息和特定的响应头信息，以便能够通过关联请求信息和响应头信息，来实现记录请求和响应的对应关系。这里通过逐步尝试和优化的方式进行尝试。具体包括将需要的请求头和响应头组织到一条日志记录，输出到单独的错误日志文件记录等的配置尝试。
1. 配置转发和日志记录
首先配置记录所有的请求头和响应头信息，记录到error_log指定的文件中。
error_log logs/error.log info;log_format main '$remote_addr [$request_id] - $remote_user [$time_local] "$request" ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent" "$http_x_forwarded_for"';location /ngtest {proxy_pass http://127.0.0.1:8060;header_filter_by_lua 'local reqheaders = ngx.req.get_headers()for key, val in pairs(reqheaders) dongx.log(ngx.INFO, "request:", key, " : ", val)end-- 获取所有响应头local headers = ngx.resp.get_headers()-- 打印所有响应头for key, val in pairs(headers) dongx.log(ngx.INFO, "response:", key, " : ", val)end';}这里的配置是将请求到本地的 http://127.0.0.1:8060 地址，并且记录请求头和响应头到日志文件，日志级别为INFO。
3. postman
通过postman发送http请求。
2. 日志记录
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):4: request:authorization : NTLM TlRMTVNTUAADAAAAGAAYAEAAAADCAMIAWAAAAAAAAAAAAAAAHAAcABoBAAAWABYANgEAAAAAAAAAAAAABYIIAJi+0X/kdkl/ONUQGF7iAm1ii5/nECrNEufop988HOhSTFsJVFvxdXYBAQAAAAAAAKAG+Vfn19oBYouf5xAqzRIAAAAAAgAGAE8AVABQAAEAHgBXAEkATgAtAEkAUgBEAFMAVgBKAE8AMwBVAFQAVAAEAA4AbwB0AHAALgBjAG8AbQADAC4AVwBJAE4ALQBJAFIARABTAFYASgBPADMAVQBUAFQALgBvAHQAcAAuAGMAbwBtAAUADgBvAHQAcAAuAGMAbwBtAAcACAD4****** while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):4: request:user-agent : PostmanRuntime/7.28.4 while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):4: request:connection : keep-alive while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):4: request:accept : */* while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):4: request:accept-encoding : gzip, deflate, br while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):4: request:postman-token : d260c097-2f00-4827-af38-319d255d1d32 while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):4: request:host : 47.93.101.175:8010 while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):11: response:connection : keep-alive while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):11: response:content-length : 0 while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"
2024/10/16 15:59:59 [info] 139655#139655: *530 [lua] header_filter_by_lua(nginx.conf:69):11: response:set-cookie : mycookie=5ea7df386cd64620bfdef149cedb1a8f; Max-Age=3600; Expires=Wed, 16 Oct 2024 08:59:59 GMT; Path=/ while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"查看日志文件error.log，可以看到所有的请求头和响应头都记录下来了。每个头信息一条日志记录。
4. 优化配置
在特定的场景中，我们可能不需要记录所有的日志，而且每个请求头和响应头单独一条记录，也不利于处理请求和响应的对应关系，虽然通过请求进程id，线程id以及请求id（比如日志中的139655#139655: *530）来进行判断，但是也增加的复杂度。最好是能够在一条日志中记录，这样通过行读取就之间关联了。
location /ngtest {proxy_pass http://127.0.0.1:8060;header_filter_by_lua 'local reqheaders = ngx.req.get_headers()local authinfo = "";for key, val in pairs(reqheaders) doif key == "authorization" thenauthinfo = "[authorization:" .. val .. "]"end-- ngx.log(ngx.INFO, "request:", key, " : ", val)end-- 获取所有响应头local headers = ngx.resp.get_headers()-- 打印所有响应头for key, val in pairs(headers) doif key == "set-cookie" thenauthinfo = authinfo .. "[set-cookie:" .. val .. "]"end-- ngx.log(ngx.INFO, "response:", key, " : ", val)endngx.log(ngx.INFO, "authinfo:[", authinfo, "]")';}修改配置后，重新执行访问请求。
5. 优化后的日志输出
2024/10/16 17:33:16 [info] 140274#140274: *541 [lua] header_filter_by_lua(nginx.conf:79):22: authinfo:[[authorization:NTLM TlRMTVNTUAADAAAAGAAYAEAAAADCAMIAWAAAAAAAAAAAAAAAHAAcABoBAAAWABYANgEAAAAAAAAAAAAABYIIAJi+0X/kdkl/ONUQGF7iAm1ii5/nECrNEufop988HOhSTFsJVFvxdXYBAQAAAAAAAKAG+Vfn19oBYouf5xAqzRIAAAAAAgAGAE8AVABQAAEAHgBXAEkATgAtAEkAUgBEAFMAVgBKAE8AMwBVAFQAVAAEAA4AbwB0AHAALgBjAG8AbQADAC4AVwBJAE4ALQBJAFIARABTAFYASgBPADMAVQBUAFQALgBvAHQAcAAuAGMAbwBtAAUADgBvAHQA******][set-cookie:mycookie=5ea7df386cd64620bfdef149cedb1a8f; Max-Age=3600; Expires=Wed, 16 Oct 2024 10:33:16 GMT; Path=/]] while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"可以看到这里记录了需要的字段信息。
需要注意的是，这里记录的是error.log日志，而不是access.log日志。
6. 指定单独的日志文件路径和名称
如果把所有的日志信息都记录在一起，就增加了业务分析的数据量和计算量，而且会出现大部分的日志数据都与业务需求无关，所以，对于特定的日志信息采用单独的日志文件来记录是比较合理的。
通过制定单独的日志文件，避免和系统的日志文件混合在一起，造成处理压力的增加。
location /ngtest {proxy_pass http://127.0.0.1:8060;error_log logs/ngtest-error.log info;header_filter_by_lua 'local reqheaders = ngx.req.get_headers()local authinfo = "";for key, val in pairs(reqheaders) doif key == "authorization" thenauthinfo = "[authorization:" .. val .. "]"end-- ngx.log(ngx.INFO, "request:", key, " : ", val)end-- 获取所有响应头local headers = ngx.resp.get_headers()-- 打印所有响应头for key, val in pairs(headers) doif key == "set-cookie" thenauthinfo = authinfo .. "[set-cookie:" .. val .. "]"end-- ngx.log(ngx.INFO, "response:", key, " : ", val)endngx.log(ngx.INFO, "authinfo:[", authinfo, "]")';}
可以看到，这里在location增加了日志文件的配置。
7. 日志内容
# tail -f logs/ngtest-error.log2024/10/16 17:54:03 [info] 140368#140368: *544 [lua] header_filter_by_lua(nginx.conf:80):22: authinfo:[[authorization:NTLM TlRMTVNTUAADAAAAGAAYAEAAAADCAMIAWAAAAAAAAAAAAAAAHAAcABoBAAAWABYANgEAAAAAAAAAAAAABYIIAJi+0X/kdkl/ONUQGF7iAm1ii5/nECrNEufop988HOhSTFsJVFvxdXYBAQAAAAAAAKAG+Vfn19oBYouf5xAqzRIAAAAAAgAGAE8AVABQAAEAHgBXAEkATgAtAEkAUgBEAFMAVgBKAE8AMwBVAFQAVAAEAA4AbwB0AHAALgBjAG8AbQADAC4AVwBJAE4ALQBJAFIARABTAFYASgBPADMAVQBUAFQALgBvAHQAcAAuAGMAbw******][set-cookie:mycookie=5ea7df386cd64620bfdef149cedb1a8f; Max-Age=3600; Expires=Wed, 16 Oct 2024 10:54:03 GMT; Path=/]] while reading response header from upstream, client: 111.206.85.18, server: localhost, request: "GET /ngtest HTTP/1.1", upstream: "http://127.0.0.1:8060/ngtest", host: "47.93.101.175:8010"从日志文件，看到我们想要的日志记录样式了。