On 3.3.2023. 21:41, Gabor LENCSE wrote:
> Dear Andrew,
>
> Thank you very much for all your advice.
>
>> Unattended upgrades via sysupgrade usually go smoothly, but if not,
>> you'll have the equivalent of physical access.
> It was smooth indeed.
>> Your paper looks interesting; it will be informative to see how much
>> OpenBSD has improved post-7.1 (and -current may have even further
>> improvements than 7.2, but I'll defer to others on those details).
>>
>> -Andrew
>
> I did an IPv6 packet forwarding performance measurement. The results of
> the 10 tests had some fluctuations. Here are the raw results (time is in
> JST, the last column shows the rate):
>
> root@p094:~/siitperf/results# cat
> p094-10G-p095-OBSD72-MP-RFC4814-IPv6-b-84-2-2-60-2000-200-2023-03-04-0326/rate.csv
> No, Size, Dir, n, m, Duration, Initial Rate, Timeout, Error, Date,
> Iterations needed, rate
> 1, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 00:17:08, 11, 279101
> 2, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 00:38:06, 11, 279491
> 3, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 00:59:03, 11, 266991
> 4, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 01:20:03, 11, 274804
> 5, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 01:41:00, 11, 262695
> 6, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 02:02:00, 11, 282616
> 7, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 02:22:57, 11, 281445
> 8, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 02:43:53, 11, 278710
> 9, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 03:04:51, 11, 275195
> 10, 84, b, 2, 2, 60, 200000, 2000, 200, 2023-03-04 03:25:50, 11, 241210
> root@p094:~/siitperf/results#
>
> My earlier results with OpenBSD 7.1 were much more consistent.
>
> I did two tests with 300,000 fps and and one with 400,000 fps (per
> direction) to check what top shows:
>
> load averages: 0.45, 0.41, 0.20 p095.lab.starbed.org 05:04:11
> 36 processes: 35 idle, 1 on processor up 0 days 00:03:19
> CPU00 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU01 states: 0.0% user, 0.0% nice, 81.0% sys, 11.2% spin, 0.0% intr,
> 7.8% idle
> CPU02 states: 0.2% user, 0.0% nice, 66.9% sys, 5.6% spin, 0.0% intr,
> 27.3% idle
> CPU03 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU04 states: 0.0% user, 0.0% nice, 20.6% sys, 1.0% spin, 0.0% intr,
> 78.4% idle
> CPU05 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU06 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU07 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU08 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU09 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 34.3% intr,
> 65.7% idle
> CPU10 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU11 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU12 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU13 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU14 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU15 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU16 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU17 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU18 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU19 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU20 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU21 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU22 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU23 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU24 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU25 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 30.3% intr,
> 69.7% idle
> CPU26 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU27 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU28 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU29 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU30 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU31 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> Memory: Real: 33M/1438M act/tot Free: 371G Cache: 715M Swap: 0K/256M
>
>
> load averages: 0.59, 0.54, 0.29 p095.lab.starbed.org 05:06:56
> 36 processes: 35 idle, 1 on processor up 0 days 00:06:05
> CPU00 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU01 states: 0.0% user, 0.0% nice, 80.0% sys, 11.4% spin, 0.0% intr,
> 8.6% idle
> CPU02 states: 0.0% user, 0.0% nice, 85.8% sys, 5.8% spin, 0.0% intr,
> 8.4% idle
> CPU03 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU04 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU05 states: 0.0% user, 0.0% nice, 0.2% sys, 0.0% spin, 0.0% intr,
> 99.8% idle
> CPU06 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU07 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU08 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU09 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 30.7% intr,
> 69.3% idle
> CPU10 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU11 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU12 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU13 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU14 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU15 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU16 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU17 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU18 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU19 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU20 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU21 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU22 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU23 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU24 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU25 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 31.5% intr,
> 68.5% idle
> CPU26 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU27 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU28 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU29 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU30 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU31 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> Memory: Real: 33M/1437M act/tot Free: 371G Cache: 715M Swap: 0K/256M
>
> load averages: 0.92, 0.68, 0.39 p095.lab.starbed.org 05:09:34
> 36 processes: 35 idle, 1 on processor up 0 days 00:08:43
> CPU00 states: 0.0% user, 0.0% nice, 10.2% sys, 0.8% spin, 0.0% intr,
> 89.0% idle
> CPU01 states: 0.0% user, 0.0% nice, 57.3% sys, 6.8% spin, 0.0% intr,
> 35.9% idle
> CPU02 states: 0.0% user, 0.0% nice, 39.5% sys, 3.6% spin, 0.0% intr,
> 56.9% idle
> CPU03 states: 0.0% user, 0.0% nice, 11.6% sys, 0.2% spin, 0.0% intr,
> 88.2% idle
> CPU04 states: 0.0% user, 0.0% nice, 44.7% sys, 3.8% spin, 0.0% intr,
> 51.5% idle
> CPU05 states: 0.0% user, 0.0% nice, 5.2% sys, 0.4% spin, 0.0% intr,
> 94.4% idle
> CPU06 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU07 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU08 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU09 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 38.5% intr,
> 61.5% idle
> CPU10 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU11 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU12 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU13 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU14 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU15 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU16 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU17 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU18 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU19 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU20 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU21 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU22 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU23 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU24 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU25 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 37.7% intr,
> 62.3% idle
> CPU26 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU27 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU28 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU29 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU30 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> CPU31 states: 0.0% user, 0.0% nice, 0.0% sys, 0.0% spin, 0.0% intr,
> 100% idle
> Memory: Real: 33M/1443M act/tot Free: 371G Cache: 715M Swap: 0K/256M
>
> All interrupts were processed by same two cores (CPU09 and CPU25). It is
> so because I used a single IPv6 address pair with bidirectional traffic
> thus the interrupts were hashed to two cores.
>
> It seems that packet forwarding were executed by 2-5 cores: only they
> had more than 1% system load. It is definitely and improvement compared
> to OpenBSD 7.1, where only a single core did all that work.
>
> As for the performance, the above results are frames per second per
> direction values, so (except for the last test) they mean more than
> 500,000fps were forwarded. The median IPv6 forwarding performance of
> OpenBSD 7.1 was 331,054fps, so it is a significant improvement.
>
> And I suspect that maybe the root cause of the fluctuations could be an
> "unfortunate" scheduling of multiple packet forwarding tasks to the same
> core.
>
> I hope that perhaps my results could help OpenBSD developers to improve
> the performance further.
Hi,
from my experience, data that would help developers are tests before
and after their diffs and maybe some analysis if box behaves
differently. Diffs, snapshots, releases, test what ever you want, just
test before and after so people can see differences.
From me important thing is consistency and knowing what to expect from
DUT box. Let's say that I'm getting cca 7Mpps through ipsec tunnel and
net guys send some SMR diffs that totally unlocks isakmpd, and after
that diff I'm getting 5Mpps, this is visible regression and I will
report that to list. :)))
No comments:
Post a Comment