I had a similar speed drop on an Edge Router 4. I don't know if it's the same situation on the Lite, but I believe it's expected due to hardware acceleration support (or lack of) and single core performance on the pf side.
Scott
> On Oct 4, 2020, at 17:24, Amarendra Godbole <amarendra.godbole@gmail.com> wrote:
>
> Sorry I forgot including "ifconfig" output:
>
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 32768
> index 5 priority 0 llprio 3
> groups: lo
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
> inet 127.0.0.1 netmask 0xff000000
>
> cnmac0: flags=808843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,AUTOCONF4> mtu 1500
> lladdr a8:28:dc:cc:2e:6f
> index 1 priority 0 llprio 3
> groups: egress
> media: Ethernet autoselect (1000baseT full-duplex,master)
> status: active
> inet 73.xx.xx.xx netmask 0xfffffe00 broadcast 73.xx.xx.255
>
> cnmac1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
> mtu 1500
> lladdr 78:8a:20:46:a8:c1
> index 2 priority 0 llprio 3
> media: Ethernet autoselect (1000baseT full-duplex)
> status: active
>
> cnmac2: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
> mtu 1500
> lladdr 78:8a:20:46:a8:c2
> index 3 priority 0 llprio 3
> media: Ethernet autoselect (none)
> status: no carrier
> enc0: flags=0<>
> index 4 priority 0 llprio 3
> groups: enc
> status: active
>
> bridge0: flags=41<UP,RUNNING>
> index 6 llprio 3
> groups: bridge
> priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp
> cnmac2 flags=7<LEARNING,DISCOVER,BLOCKNONIP>
> port 3 ifpriority 0 ifcost 0
> cnmac1 flags=7<LEARNING,DISCOVER,BLOCKNONIP>
> port 2 ifpriority 0 ifcost 0
> vether0 flags=7<LEARNING,DISCOVER,BLOCKNONIP>
> port 7 ifpriority 0 ifcost 0
>
> vether0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
> lladdr fe:e1:ba:d0:c8:a9
> index 7 priority 0 llprio 3
> groups: vether
> media: Ethernet autoselect
> status: active
> inet 192.168.10.1 netmask 0xffffff00 broadcast 192.168.10.255
>
> pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33136
> index 8 priority 0 llprio 3
> groups: pflog
>
>> On Sun, Oct 4, 2020 at 2:22 PM Amarendra Godbole
>> <amarendra.godbole@gmail.com> wrote:
>>
>> Hi misc@
>>
>> I recently introduced an OpenBSD firewall inline and noticed a
>> reduction in overall download speeds. I am trying to understand why
>> this may be so. The firewall is Ubiquiti ERL running 6.7 release.
>> Internet connection is Comcast xfinity via cable modem, plan 200
>> Mbits/s down and 10 Mbits/s up. Details follow:
>>
>> 1. config #1: MacBook - Linksys WRT1200AC - xfinity cable modem
>> (speed: ~210 Mbits/s down, 6 Mbits/s up)
>> 2. config #2: MacBook - Linksys WRT1200AC - Ubiquiti ERL - xfinity
>> cable modem (speed: ~90 MBits down, 6 Mbits/s up)
>> 3. config #3 (Line speed): MacBook wired to cable modem (~230 Mbits/s
>> down, ~8 Mbits/s up).
>>
>> Linksys is running latest OpenWrt, and speed tests were run on MacBook
>> connected wired to Linksys. It was difficult to try tcpbench since the
>> setup was cumbersome, and iperf3 public servers end up being busy more
>> often than not (and threads on misc@ indicated iperf3 wasn't as
>> reliable either). Test numbers come from speedtest.net and
>> speed.cloudflare.com. While I realize this speed test is hardly
>> accurate, I have tried to maintain the same configuration (no ERL and
>> inline ERL) to obtain relative numbers.
>>
>> I am trying to understand the reduction from 210 Mbits/s down to 90
>> Mbits/s down between config #1 and config #2 above. The slowdown is
>> not noticeable to family, so this is more of my intellectual curiosity
>> than screams over a buffering video! :-)
>>
>> Relevant system information (dmesg, etc.) below. All sysctl values
>> attached as sysctl.txt I gathered it by reading similar threads on
>> misc@. If I missed anything, please let me know. Thanks in advance.
>>
>> -Amarendra
>>
>>
>> dmesg:
>>
>> Copyright (c) 1982, 1986, 1989, 1991, 1993
>> The Regents of the University of California. All rights reserved.
>> Copyright (c) 1995-2020 OpenBSD. All rights reserved. https://www.OpenBSD.org
>> OpenBSD 6.7 (GENERIC.MP) #134: Thu May 7 16:05:06 MDT 2020
>> deraadt@octeon.openbsd.org:/usr/src/sys/arch/octeon/compile/GENERIC.MP
>> real mem = 536870912 (512MB)
>> avail mem = 506740736 (483MB)
>> mainbus0 at root: board 20002 rev 2.18
>> cpu0 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation
>> cpu0: cache L1-I 32KB 4 way D 16KB 64 way, L2 128KB 8 way
>> cpu1 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation
>> cpu1: cache L1-I 32KB 4 way D 16KB 64 way, L2 128KB 8 way
>> clock0 at mainbus0: int 5
>> octcrypto0 at mainbus0
>> iobus0 at mainbus0
>> simplebus0 at iobus0: "soc"
>> octciu0 at simplebus0
>> octsmi0 at simplebus0
>> octpip0 at simplebus0
>> octgmx0 at octpip0 interface 0
>> cnmac0 at octgmx0: RGMII, address 78:8a:20:46:a8:c0
>> atphy0 at cnmac0 phy 7: AR8035 10/100/1000 PHY, rev. 2
>> cnmac1 at octgmx0: RGMII, address 78:8a:20:46:a8:c1
>> atphy1 at cnmac1 phy 6: AR8035 10/100/1000 PHY, rev. 2
>> cnmac2 at octgmx0: RGMII, address 78:8a:20:46:a8:c2
>> atphy2 at cnmac2 phy 5: AR8035 10/100/1000 PHY, rev. 2
>> com0 at simplebus0: ns16550a, 64 byte fifo
>> com0: console
>> dwctwo0 at iobus0 base 0x1180068000000 irq 56
>> usb0 at dwctwo0: USB revision 2.0
>> uhub0 at usb0 configuration 1 interface 0 "Octeon DWC2 root hub" rev
>> 2.00/1.00 addr 1
>> octrng0 at iobus0 base 0x1400000000000 irq 0
>> /dev/ksyms: Symbol table not valid.
>> umass0 at uhub0 port 1 configuration 1 interface 0 "Lexar USB Flash
>> Drive" rev 2.10/11.00 addr 2
>> umass0: using SCSI over Bulk-Only
>> scsibus0 at umass0: 2 targets, initiator 0
>> sd0 at scsibus0 targ 1 lun 0: <Lexar, USB Flash Drive, 1100> removable
>> serial.21c40cd1719080003000
>> sd0: 30526MB, 512 bytes/sector, 62517248 sectors
>> vscsi0 at root
>> scsibus1 at vscsi0: 256 targets
>> softraid0 at root
>> scsibus2 at softraid0: 256 targets
>> boot device: sd0
>> root on sd0a (2124441bc835a462.a) swap on sd0b dump on sd0b
>> WARNING: No TOD clock, believing file system.
>> WARNING: CHECK AND RESET THE DATE!
>>
>>
>> pftcl -s:
>>
>> match in all scrub (no-df random-id max-mss 1440)
>> block drop in quick on ! cnmac0 inet from xx.xx.xx.xx/23 to any
>> block drop in quick inet from xx.xx.xx.xx to any
>> block drop all
>> pass out quick on egress inet from (vether0:network) to any flags S/SA
>> nat-to (egress) round-robin
>> pass out quick inet all flags S/SA
>> pass in on vether0 inet all flags S/SA
>> pass in on cnmac1 inet all flags S/SA
>> pass in on cnmac2 inet all flags S/SA
>>
>> pfctl -si:
>>
>> Status: Enabled for 3 days 18:14:11 Debug: err
>>
>> Interface Stats for egress IPv4 IPv6
>>
>> Bytes In 64537867779 0
>> Bytes Out 7005140381 0
>> Packets In
>> Passed 56024370 0
>> Blocked 43050 0
>> Packets Out
>> Passed 22271061 0
>> Blocked 0 0
>>
>> State Table Total Rate
>> current entries 847
>> half-open tcp 23
>> searches 158989755 489.4/s
>> inserts 1095307 3.4/s
>> removals 1094460 3.4/s
>> Counters
>> match 1343178 4.1/s
>> bad-offset 0 0.0/s
>> fragment 0 0.0/s
>> short 1 0.0/s
>> normalize 0 0.0/s
>> memory 0 0.0/s
>> bad-timestamp 0 0.0/s
>> congestion 41 0.0/s
>> ip-option 26164 0.1/s
>> proto-cksum 0 0.0/s
>> state-mismatch 10758 0.0/s
>> state-insert 0 0.0/s
>> state-limit 0 0.0/s
>> src-limit 0 0.0/s
>> synproxy 0 0.0/s
>> translate 0 0.0/s
>> no-route 0 0.0/s
>>
>> pfctl -s memory:
>>
>> states hard limit 100000
>> src-nodes hard limit 10000
>> frags hard limit 16384
>> tables hard limit 1000
>> table-entries hard limit 200000
>> pktdelay-pkts hard limit 10000
>>
>> The netlivlocks value keeps on increasing regularly:
>> kern.netlivelocks=57911
>>
>> netstat -m:
>>
>> 1009 mbufs in use:
>> 917 mbufs allocated to data
>> 5 mbufs allocated to packet headers
>> 87 mbufs allocated to socket names and addresses
>> 801/7256 mbuf 2048 byte clusters in use (current/peak)
>> 0/15 mbuf 2112 byte clusters in use (current/peak)
>> 0/24 mbuf 4096 byte clusters in use (current/peak)
>> 0/8 mbuf 8192 byte clusters in use (current/peak)
>> 0/0 mbuf 9216 byte clusters in use (current/peak)
>> 0/0 mbuf 12288 byte clusters in use (current/peak)
>> 0/0 mbuf 16384 byte clusters in use (current/peak)
>> 0/8 mbuf 65536 byte clusters in use (current/peak)
>> 6512/17088/131072 Kbytes allocated to network (current/peak/max)
>> 0 requests for memory denied
>> 0 requests for memory delayed
>> 0 calls to protocol drain routines
>>
>> netstat -i:
>>
>> Name Mtu Network Address Ipkts Ifail Opkts Ofail Colls
>> lo0 32768 <Link> 198 0 198 0 0
>> lo0 32768 localhost/1 localhost 198 0 198 0 0
>> lo0 32768 fe80::%lo0/ fe80::1%lo0 198 0 198 0 0
>> lo0 32768 127/8 localhost 198 0 198 0 0
>> cnmac0 1600 <Link> a8:28:dc:cc:2e:6f 56088774 0 22283491 2688 0
>> cnmac0 1600 73.231.60/2 c-73-231-60-128.h 56088774 0 22283491 2688 0
>> cnmac1 1600 <Link> 78:8a:20:46:a8:c1 23646497 4 56569853 48 0
>> cnmac2 1600 <Link> 78:8a:20:46:a8:c2 14823 0 226198 226198 0
>> enc0* 0 <Link> 0 0 0 0 0
>> bridge0 1500 <Link> 23187238 0 57022219 0 0
>> vether0 32768 <Link> fe:e1:ba:d0:c8:a9 23056709 0 56795991 0 0
>> vether0 32768 192.168.10/ 192.168.10.1 23056709 0 56795991 0 0
>> pflog0 33136 <Link> 0 0 26171 0 0
>
No comments:
Post a Comment