Skip to content

Commit 2fcba62

Browse files
sjbermangithub-actions[bot]nginx-bot
authored
NFR Test Results for NGF version v2.3.0 (#4453) (#4475)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: nginx-bot <[email protected]>
1 parent d1a216c commit 2fcba62

File tree

80 files changed

+1576
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

80 files changed

+1576
-0
lines changed

‎tests/results/dp-perf/2.3.0/2.3.0-oss.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Results
2+
3+
## Test environment
4+
5+
NGINX Plus: false
6+
7+
NGINX Gateway Fabric:
8+
9+
- Commit: 89aee48bf6e660a828ffd32ca35fc7f52e358e00
10+
- Date: 2025-12-12T20:04:38Z
11+
- Dirty: false
12+
13+
GKE Cluster:
14+
15+
- Node count: 12
16+
- k8s version: v1.33.5-gke.1308000
17+
- vCPUs per node: 16
18+
- RAM per node: 65851520Ki
19+
- Max pods per node: 110
20+
- Zone: us-west1-b
21+
- Instance Type: n2d-standard-16
22+
23+
## Summary:
24+
25+
- Latency continues to grow slightly, per the trend of past releases.
26+
27+
## Test1: Running latte path based routing
28+
29+
```text
30+
Requests [total, rate, throughput] 30000, 1000.03, 999.99
31+
Duration [total, attack, wait] 30s, 29.999s, 991.978µs
32+
Latencies [min, mean, 50, 90, 95, 99, max] 816.445µs, 1.069ms, 1.045ms, 1.166ms, 1.217ms, 1.385ms, 23.061ms
33+
Bytes In [total, mean] 4740000, 158.00
34+
Bytes Out [total, mean] 0, 0.00
35+
Success [ratio] 100.00%
36+
Status Codes [code:count] 200:30000
37+
Error Set:
38+
```
39+
40+
## Test2: Running coffee header based routing
41+
42+
```text
43+
Requests [total, rate, throughput] 30000, 1000.04, 1000.00
44+
Duration [total, attack, wait] 30s, 29.999s, 1.132ms
45+
Latencies [min, mean, 50, 90, 95, 99, max] 840.624µs, 1.096ms, 1.073ms, 1.204ms, 1.26ms, 1.44ms, 16.79ms
46+
Bytes In [total, mean] 4770000, 159.00
47+
Bytes Out [total, mean] 0, 0.00
48+
Success [ratio] 100.00%
49+
Status Codes [code:count] 200:30000
50+
Error Set:
51+
```
52+
53+
## Test3: Running coffee query based routing
54+
55+
```text
56+
Requests [total, rate, throughput] 30000, 1000.04, 1000.00
57+
Duration [total, attack, wait] 30s, 29.999s, 1.067ms
58+
Latencies [min, mean, 50, 90, 95, 99, max] 825.3µs, 1.095ms, 1.071ms, 1.201ms, 1.256ms, 1.444ms, 16.845ms
59+
Bytes In [total, mean] 5010000, 167.00
60+
Bytes Out [total, mean] 0, 0.00
61+
Success [ratio] 100.00%
62+
Status Codes [code:count] 200:30000
63+
Error Set:
64+
```
65+
66+
## Test4: Running tea GET method based routing
67+
68+
```text
69+
Requests [total, rate, throughput] 30000, 1000.02, 999.99
70+
Duration [total, attack, wait] 30s, 29.999s, 954.141µs
71+
Latencies [min, mean, 50, 90, 95, 99, max] 818.006µs, 1.079ms, 1.059ms, 1.187ms, 1.241ms, 1.411ms, 14.873ms
72+
Bytes In [total, mean] 4680000, 156.00
73+
Bytes Out [total, mean] 0, 0.00
74+
Success [ratio] 100.00%
75+
Status Codes [code:count] 200:30000
76+
Error Set:
77+
```
78+
79+
## Test5: Running tea POST method based routing
80+
81+
```text
82+
Requests [total, rate, throughput] 30000, 1000.03, 1000.00
83+
Duration [total, attack, wait] 30s, 29.999s, 992.607µs
84+
Latencies [min, mean, 50, 90, 95, 99, max] 808.16µs, 1.086ms, 1.064ms, 1.196ms, 1.248ms, 1.42ms, 17.019ms
85+
Bytes In [total, mean] 4680000, 156.00
86+
Bytes Out [total, mean] 0, 0.00
87+
Success [ratio] 100.00%
88+
Status Codes [code:count] 200:30000
89+
Error Set:
90+
```

‎tests/results/dp-perf/2.3.0/2.3.0-plus.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Results
2+
3+
## Test environment
4+
5+
NGINX Plus: true
6+
7+
NGINX Gateway Fabric:
8+
9+
- Commit: 89aee48bf6e660a828ffd32ca35fc7f52e358e00
10+
- Date: 2025-12-12T20:04:38Z
11+
- Dirty: false
12+
13+
GKE Cluster:
14+
15+
- Node count: 12
16+
- k8s version: v1.33.5-gke.1308000
17+
- vCPUs per node: 16
18+
- RAM per node: 65851520Ki
19+
- Max pods per node: 110
20+
- Zone: us-west1-b
21+
- Instance Type: n2d-standard-16
22+
23+
## Summary:
24+
25+
- Latency looks to have improved slightly.
26+
27+
## Test1: Running latte path based routing
28+
29+
```text
30+
Requests [total, rate, throughput] 30000, 1000.04, 1000.01
31+
Duration [total, attack, wait] 30s, 29.999s, 880.439µs
32+
Latencies [min, mean, 50, 90, 95, 99, max] 691.14µs, 886.932µs, 867.964µs, 976.348µs, 1.018ms, 1.153ms, 10.358ms
33+
Bytes In [total, mean] 4830000, 161.00
34+
Bytes Out [total, mean] 0, 0.00
35+
Success [ratio] 100.00%
36+
Status Codes [code:count] 200:30000
37+
Error Set:
38+
```
39+
40+
## Test2: Running coffee header based routing
41+
42+
```text
43+
Requests [total, rate, throughput] 30000, 1000.04, 1000.01
44+
Duration [total, attack, wait] 30s, 29.999s, 923.361µs
45+
Latencies [min, mean, 50, 90, 95, 99, max] 726.599µs, 948.386µs, 919.848µs, 1.025ms, 1.07ms, 1.262ms, 22.38ms
46+
Bytes In [total, mean] 4860000, 162.00
47+
Bytes Out [total, mean] 0, 0.00
48+
Success [ratio] 100.00%
49+
Status Codes [code:count] 200:30000
50+
Error Set:
51+
```
52+
53+
## Test3: Running coffee query based routing
54+
55+
```text
56+
Requests [total, rate, throughput] 30000, 1000.04, 1000.01
57+
Duration [total, attack, wait] 30s, 29.999s, 980.118µs
58+
Latencies [min, mean, 50, 90, 95, 99, max] 741.198µs, 949.099µs, 920.511µs, 1.025ms, 1.067ms, 1.241ms, 19.154ms
59+
Bytes In [total, mean] 5100000, 170.00
60+
Bytes Out [total, mean] 0, 0.00
61+
Success [ratio] 100.00%
62+
Status Codes [code:count] 200:30000
63+
Error Set:
64+
```
65+
66+
## Test4: Running tea GET method based routing
67+
68+
```text
69+
Requests [total, rate, throughput] 30000, 1000.01, 999.98
70+
Duration [total, attack, wait] 30.001s, 30s, 997.667µs
71+
Latencies [min, mean, 50, 90, 95, 99, max] 716.164µs, 903.954µs, 881.394µs, 978.714µs, 1.019ms, 1.192ms, 21.825ms
72+
Bytes In [total, mean] 4770000, 159.00
73+
Bytes Out [total, mean] 0, 0.00
74+
Success [ratio] 100.00%
75+
Status Codes [code:count] 200:30000
76+
Error Set:
77+
```
78+
79+
## Test5: Running tea POST method based routing
80+
81+
```text
82+
Requests [total, rate, throughput] 30000, 1000.01, 999.97
83+
Duration [total, attack, wait] 30.001s, 30s, 919.688µs
84+
Latencies [min, mean, 50, 90, 95, 99, max] 708.879µs, 925.517µs, 903.767µs, 1.012ms, 1.054ms, 1.21ms, 22.009ms
85+
Bytes In [total, mean] 4770000, 159.00
86+
Bytes Out [total, mean] 0, 0.00
87+
Success [ratio] 100.00%
88+
Status Codes [code:count] 200:30000
89+
Error Set:
90+
```

‎tests/results/longevity/2.3.0/2.3.0-oss.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Results
2+
3+
## Test environment
4+
5+
NGINX Plus: false
6+
7+
NGINX Gateway Fabric:
8+
9+
- Commit: 89aee48bf6e660a828ffd32ca35fc7f52e358e00
10+
- Date: 2025-12-12T20:04:38Z
11+
- Dirty: false
12+
13+
GKE Cluster:
14+
15+
- Node count: 3
16+
- k8s version: v1.33.5-gke.1308000
17+
- vCPUs per node: 2
18+
- RAM per node: 4015672Ki
19+
- Max pods per node: 110
20+
- Zone: us-west2-a
21+
- Instance Type: e2-medium
22+
23+
## Summary:
24+
25+
- Still a lot of non-2xx or 3xx responses, many more than last time. Socket errors are all mostly read errors, with no write errors and fewer timeout errors.
26+
- We observe a continual increase in NGINX memory usage over time which could indicate a memory leak. Will bring this up with the Agent team.
27+
- CPU usage remained consistent with past results.
28+
- Error contacting TokenReview API, but may be a one-off.
29+
30+
## Traffic
31+
32+
HTTP:
33+
34+
```text
35+
Running 5760m test @ http://cafe.example.com/coffee
36+
2 threads and 100 connections
37+
Thread Stats Avg Stdev Max +/- Stdev
38+
Latency 190.35ms 141.74ms 2.00s 83.52%
39+
Req/Sec 289.84 187.59 3.52k 63.68%
40+
195509968 requests in 5760.00m, 66.75GB read
41+
Socket errors: connect 0, read 315485, write 0, timeout 6584
42+
Non-2xx or 3xx responses: 1763516
43+
Requests/sec: 565.71
44+
Transfer/sec: 202.53KB
45+
```
46+
47+
HTTPS:
48+
49+
```text
50+
Running 5760m test @ https://cafe.example.com/tea
51+
2 threads and 100 connections
52+
Thread Stats Avg Stdev Max +/- Stdev
53+
Latency 180.03ms 106.92ms 1.94s 67.25%
54+
Req/Sec 287.34 184.95 1.73k 63.36%
55+
193842103 requests in 5760.00m, 65.22GB read
56+
Socket errors: connect 0, read 309621, write 0, timeout 1
57+
Requests/sec: 560.89
58+
Transfer/sec: 197.88KB
59+
```
60+
## Key Metrics
61+
62+
### Containers memory
63+
64+
![oss-memory.png](oss-memory.png)
65+
66+
### Containers CPU
67+
68+
![oss-cpu.png](oss-cpu.png)
69+
70+
## Error Logs
71+
72+
### nginx-gateway
73+
74+
error=rpc error: code = Internal desc = error creating TokenReview: context canceled;level=error;logger=agentGRPCServer;msg=error validating connection;stacktrace=github.com/nginx/nginx-gateway-fabric/v2/internal/controller/nginx/agent/grpc/interceptor.(*ContextSetter).Stream.ContextSetter.Stream.func1
75+
/opt/actions-runner/_work/nginx-gateway-fabric/nginx-gateway-fabric/internal/controller/nginx/agent/grpc/interceptor/interceptor.go:62
76+
google.golang.org/grpc.(*Server).processStreamingRPC
77+
/opt/actions-runner/_work/nginx-gateway-fabric/nginx-gateway-fabric/.gocache/google.golang.org/[email protected]/server.go:1721
78+
google.golang.org/grpc.(*Server).handleStream
79+
/opt/actions-runner/_work/nginx-gateway-fabric/nginx-gateway-fabric/.gocache/google.golang.org/[email protected]/server.go:1836
80+
google.golang.org/grpc.(*Server).serveStreams.func2.1
81+
/opt/actions-runner/_work/nginx-gateway-fabric/nginx-gateway-fabric/.gocache/google.golang.org/[email protected]/server.go:1063;ts=2025-12-16T17:35:17Z
82+
83+
### nginx

‎tests/results/longevity/2.3.0/2.3.0-plus.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# Results
2+
3+
## Test environment
4+
5+
NGINX Plus: false
6+
7+
NGINX Gateway Fabric:
8+
9+
- Commit: 89aee48bf6e660a828ffd32ca35fc7f52e358e00
10+
- Date: 2025-12-12T20:04:38Z
11+
- Dirty: false
12+
13+
GKE Cluster:
14+
15+
- Node count: 3
16+
- k8s version: v1.33.5-gke.1308000
17+
- vCPUs per node: 2
18+
- RAM per node: 4015672Ki
19+
- Max pods per node: 110
20+
- Zone: us-west2-a
21+
- Instance Type: e2-medium
22+
23+
## Summary:
24+
25+
- Consistent traffic results from 2.2.
26+
- We observe a continual increase in NGINX memory usage over time which could indicate a memory leak. Will bring this up with the Agent team.
27+
- CPU usage remained consistent with past results.
28+
- Still get some "no live upstreams" errors.
29+
30+
## Traffic
31+
32+
HTTP:
33+
34+
```text
35+
Running 5760m test @ http://cafe.example.com/coffee
36+
2 threads and 100 connections
37+
Thread Stats Avg Stdev Max +/- Stdev
38+
Latency 184.82ms 102.91ms 1.45s 65.52%
39+
Req/Sec 284.19 179.74 1.52k 63.62%
40+
192198367 requests in 5760.00m, 65.91GB read
41+
Socket errors: connect 0, read 0, write 0, timeout 108
42+
Non-2xx or 3xx responses: 5
43+
Requests/sec: 556.13
44+
Transfer/sec: 199.96KB
45+
```
46+
47+
HTTPS:
48+
49+
```text
50+
Running 5760m test @ https://cafe.example.com/tea
51+
2 threads and 100 connections
52+
Thread Stats Avg Stdev Max +/- Stdev
53+
Latency 185.02ms 102.92ms 1.50s 65.52%
54+
Req/Sec 283.70 179.19 1.43k 63.75%
55+
191866398 requests in 5760.00m, 64.73GB read
56+
Socket errors: connect 0, read 0, write 0, timeout 114
57+
Non-2xx or 3xx responses: 6
58+
Requests/sec: 555.17
59+
Transfer/sec: 196.40KB
60+
```
61+
## Key Metrics
62+
63+
### Containers memory
64+
65+
![oss-memory.png](oss-memory.png)
66+
67+
### Containers CPU
68+
69+
![oss-cpu.png](oss-cpu.png)
70+
71+
## Error Logs
72+
73+
### nginx-gateway
74+
75+
### nginx
76+
77+
78+
79+
80+
10.168.0.90 - - [16/Dec/2025:15:47:08 +0000] "GET /tea HTTP/1.1" 502 150 "-" "-"
81+
2025/12/16 15:47:08 [error] 26#26: *361983622 no live upstreams while connecting to upstream, client: 10.168.0.90, server: cafe.example.com, request: "GET /tea HTTP/1.1", upstream: "http://longevity_tea_80/tea", host: "cafe.example.com"
82+
10.168.0.90 - - [16/Dec/2025:12:49:07 +0000] "GET /coffee HTTP/1.1" 502 150 "-" "-"
83+
2025/12/16 12:49:07 [error] 25#25: *350621339 no live upstreams while connecting to upstream, client: 10.168.0.90, server: cafe.example.com, request: "GET /coffee HTTP/1.1", upstream: "http://longevity_coffee_80/coffee", host: "cafe.example.com"

‎tests/results/longevity/2.3.0/oss-cpu.png‎

48.3 KB
Loading

‎tests/results/longevity/2.3.0/oss-memory.png‎

41.7 KB
Loading

‎tests/results/longevity/2.3.0/plus-cpu.png‎

50.1 KB
Loading

‎tests/results/longevity/2.3.0/plus-memory.png‎

37.1 KB
Loading

0 commit comments

Comments
 (0)