Wednesday, February 05, 2025

Re: bnxt(4) media 10GbaseLR

On Mon, Feb 03, 2025 at 05:51:39PM +0100, Julian Huhn wrote:
>On Mon, Feb 03, 2025 at 04:05:34PM +0100, Julian Huhn wrote:
>>On Mon, Feb 03, 2025 at 02:33:59PM +0000, Stuart Henderson wrote:
>>>On 2025-02-03, Julian Huhn <julian@huhn.dev> wrote:
>>>>Moin!
>>>>
>>>>I'm trying to connect to an Internet exchange with a bnxt(4) interface, but I
>>>>only get 10GbaseSR as the media type, not 10GbaseLR as expected. When I try to
>>>>set the media type manually, it fails with an "Invalid argument". In fact,
>>>>10GbaseLR isn't listed as a supported media type for the interface. Is this
>>>>correct? Is SR the only supported mode on this type of network card?
>>>
>>>You can't switch between SR and LR in software, it's dependent on
>>>the transceiver.
>>
>>Ok, but the transceiver is LR as far as I can see, so it was
>>misidentified by bnxt(4), right?
>>
>>>Is it working OK anyway? (i.e. is this just a cosmetic issue?)
>>
>>Unfortunately, I cannot establish connectivity with the interface. However,
>>both sides claim that the interfaces are active.
>
>Perhaps the transceiver is to blame. On Linux (Debian), a similar
>behavior can be observed. So this is not specific to OpenBSD. I shut
>down the VM and looked at the active, passed-through interface on the
>host, which also showed that only SR is supported. However, Linux
>advertises all expected media types on the inactive interface.

I don't know why I didn't test earlier setting up the IPs on the interface on
Linux, but on Linux I do get connectivity.

A short summary of my findings:

- OpenBSD has no connectivity with the combination of bnxt(4) and the
transceiver
- Linux has connectivity
- Both incorrectly show supported and advertised link modes as SR instead of
LR

As a workaround, I set up a VirtIO interface for the OpenBSD VM and bridged it
to the bnxt(4) interface on Linux. It's pretty ugly, but it works.

If I can help debug why bnxt(4) doesn't get connectivity on OpenBSD in this
case, I'd be pleased to provide more command output and information.

>On Linux:
>
># ethtool enp1s0f0np0
>Settings for enp1s0f0np0:
> Supported ports: [ FIBRE ]
> Supported link modes: 25000baseSR/Full
> 10000baseSR/Full
> Supported pause frame use: Symmetric Receive-only
> Supports auto-negotiation: Yes
> Supported FEC modes: RS BASER
> Advertised link modes: 25000baseSR/Full
> 10000baseSR/Full
> Advertised pause frame use: Symmetric
> Advertised auto-negotiation: Yes
> Advertised FEC modes: Not reported
> Speed: 10000Mb/s
> Lanes: 1
> Duplex: Full
> Auto-negotiation: on
> Port: FIBRE
> PHYAD: 1
> Transceiver: internal
> Supports Wake-on: d
> Wake-on: d
> Current message level: 0x00002081 (8321)
> drv tx_err hw
> Link detected: no
>
># ethtool enp1s0f1np1
>Settings for enp1s0f1np1:
> Supported ports: [ FIBRE ]
> Supported link modes: 1000baseT/Full
> 10000baseT/Full
> 1000baseKX/Full
> 10000baseKX4/Full
> 10000baseKR/Full
> 25000baseCR/Full
> 25000baseKR/Full
> 25000baseSR/Full
> 1000baseX/Full
> 10000baseCR/Full
> 10000baseSR/Full
> 10000baseLR/Full
> Supported pause frame use: Symmetric Receive-only
> Supports auto-negotiation: Yes
> Supported FEC modes: RS BASER
> Advertised link modes: 1000baseT/Full
> 10000baseT/Full
> 1000baseKX/Full
> 10000baseKX4/Full
> 10000baseKR/Full
> 25000baseCR/Full
> 25000baseKR/Full
> 25000baseSR/Full
> 1000baseX/Full
> 10000baseCR/Full
> 10000baseSR/Full
> 10000baseLR/Full
> Advertised pause frame use: No
> Advertised auto-negotiation: Yes
> Advertised FEC modes: Not reported
> Speed: Unknown!
> Duplex: Unknown! (255)
> Auto-negotiation: on
> Port: FIBRE
> PHYAD: 1
> Transceiver: internal
> Supports Wake-on: d
> Wake-on: d
> Current message level: 0x00002081 (8321)
> drv tx_err hw
> Link detected: no
>
>On OpenBSD:
>
># ifconfig bnxt0 media
>bnxt0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> lladdr 6c:92:cf:12:94:e0
> index 1 priority 0 llprio 3
> media: Ethernet autoselect (10GbaseSR full-duplex)
> status: active
> supported media:
> media 1000baseSX
> media 10GbaseSR
> media 25GbaseSR
> media autoselect
>
># ifconfig bnxt1 media
>bnxt1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
> lladdr 6c:92:cf:12:94:e1
> index 2 priority 0 llprio 3
> media: Ethernet autoselect
> status: no carrier
> supported media:
> media 1000baseT
> media 10GSFP+Cu
> media autoselect
>

No comments:

Post a Comment