Ping Spikes

Hello,
Trying to replace my existing Linux firewall with OpenBSD and PF and noticed that I am seeing random ping spikes to anything on my existing network:

# ping 10.255.255.2
PING 10.255.255.2 (10.255.255.2): 56 data bytes
64 bytes from 10.255.255.2: icmp_seq=0 ttl=64 time=13.024 ms
64 bytes from 10.255.255.2: icmp_seq=1 ttl=64 time=8.367 ms
64 bytes from 10.255.255.2: icmp_seq=2 ttl=64 time=3.442 ms
64 bytes from 10.255.255.2: icmp_seq=3 ttl=64 time=0.873 ms
64 bytes from 10.255.255.2: icmp_seq=4 ttl=64 time=0.796 ms
64 bytes from 10.255.255.2: icmp_seq=5 ttl=64 time=0.752 ms
64 bytes from 10.255.255.2: icmp_seq=6 ttl=64 time=4.074 ms
64 bytes from 10.255.255.2: icmp_seq=7 ttl=64 time=0.661 ms
64 bytes from 10.255.255.2: icmp_seq=8 ttl=64 time=0.698 ms
64 bytes from 10.255.255.2: icmp_seq=9 ttl=64 time=1.002 ms

Doing some light research it seems that some hardware has an issue with the inteldrm driver, especially when it is headless[1], however my hardware does not use this. My hardware is as follows:

Motherboard: Supermicro X10SDV-4C-TLN2F
CPU: Intel Xeon processor D-1521
NIC: em0/1 (the devices I'm using) Intel I350
Graphics: Aspeed AST2400 BMC

This board does have integrated graphics with its BMC from ASPEED. I also tried hooking up a VGA monitor to the port but the issues still occur. Is there something else I should be looking at here? Is the graphics driver a red herring in my case? Any one else using a Supermicro XeonD based system without issues?

Thanks,
Jarod

[1]: https://www.reddit.com/r/openbsd/comments/105c0zk/issues_with_openbsd_72_on_protectli/jg4aq13/