Everything is working after the newest patch! Thank you all!
On 7/25/23 11:18 AM, Jag Talon wrote:
> I ran into the same issue with the "2048.00 MB AMD High Performance, 2
> vCPU" on my end. Fortunately I had a snapshot and I was able to roll
> back.
>
> Here's my dmesg output if that's helpful:
>
> OpenBSD 7.3 (GENERIC.MP) #1125: Sat Mar 25 10:36:29 MDT 2023
> deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 2130558976 (2031MB)
> avail mem = 2046648320 (1951MB)
> random: good seed from bootblocks
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5b70 (9 entries)
> bios0: vendor Vultr
> bios0: Vultr VHP
> acpi0 at bios0: ACPI 3.0
> acpi0: sleep states S3 S4 S5
> acpi0: tables DSDT FACP APIC HPET MCFG WAET
> acpi0: wakeup devices
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: AMD EPYC-Rome Processor, 1996.53 MHz, 17-31-00
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
> cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB
> 64b/line 8-way L2 cache, 16MB 64b/line 16-way L3 cache
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
> cpu0: apic clock running at 1000MHz
> cpu1 at mainbus0: apid 1 (application processor)
> cpu1: AMD EPYC-Rome Processor, 1996.51 MHz, 17-31-00
> cpu1:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,AMCR8,ABM,SSE4A,MASSE,OSVW,TOPEXT,CPCTR,FSGSBASE,BMI1,AVX2,SMEP,BMI2,CLFLUSHOPT,CLWB,SHA,UMIP,IBRS,IBPB,SSBD,IBPB,STIBP,XSAVEOPT,XSAVES
> cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 512KB
> 64b/line 8-way L2 cache, 16MB 64b/line 16-way L3 cache
> cpu1: smt 1, core 0, package 0
> ioapic0 at mainbus0: apid 0 pa 0xfec00000, version 11, 24 pins
> acpihpet0 at acpi0: 100000000 Hz
> acpimcfg0 at acpi0
> acpimcfg0: addr 0xb0000000, bus 0-255
> acpiprt0 at acpi0: bus 0 (PCI0)
> "ACPI0006" at acpi0 not configured
> acpipci0 at acpi0 PCI0: 0x00000010 0x00000011 0x00000000
> "PNP0A06" at acpi0 not configured
> "PNP0A06" at acpi0 not configured
> "PNP0A06" at acpi0 not configured
> "QEMU0002" at acpi0 not configured
> acpicmos0 at acpi0
> "ACPI0010" at acpi0 not configured
> acpicpu0 at acpi0: C1(@1 halt!)
> acpicpu1 at acpi0: C1(@1 halt!)
> pvbus0 at mainbus0: KVM
> pvclock0 at pvbus0
> pci0 at mainbus0 bus 0
> pchb0 at pci0 dev 0 function 0 "Intel 82G33 Host" rev 0x00
> vga1 at pci0 dev 1 function 0 "Bochs VGA" rev 0x02
> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
> wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
> ppb0 at pci0 dev 2 function 0 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci1 at ppb0 bus 1
> virtio0 at pci1 dev 0 function 0 "Qumranet Virtio 1.x Network" rev 0x01
> vio0 at virtio0: address 56:00:04:81:d8:d4
> virtio0: msix shared
> ppb1 at pci0 dev 2 function 1 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci2 at ppb1 bus 2
> ppb2 at pci2 dev 0 function 0 vendor "Red Hat", unknown product 0x000e
> rev 0x00
> pci3 at ppb2 bus 3
> "Intel 6300ESB WDT" rev 0x00 at pci3 dev 1 function 0 not configured
> ppb3 at pci0 dev 2 function 2 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci4 at ppb3 bus 4
> xhci0 at pci4 dev 0 function 0 vendor "Red Hat", unknown product
> 0x000d rev 0x01: apic 0 int 22, xHCI 0.0
> usb0 at xhci0: USB revision 3.0
> uhub0 at usb0 configuration 1 interface 0 "Red Hat xHCI root hub" rev
> 3.00/1.00 addr 1
> ppb4 at pci0 dev 2 function 3 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci5 at ppb4 bus 5
> virtio1 at pci5 dev 0 function 0 "Qumranet Virtio 1.x Storage" rev 0x01
> vioblk0 at virtio1
> scsibus1 at vioblk0: 1 targets
> sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, >
> sd0: 61440MB, 512 bytes/sector, 125829120 sectors
> virtio1: msix per-VQ
> ppb5 at pci0 dev 2 function 4 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci6 at ppb5 bus 6
> virtio2 at pci6 dev 0 function 0 vendor "Qumranet", unknown product
> 0x1045 rev 0x01
> viomb0 at virtio2
> virtio2: apic 0 int 22
> ppb6 at pci0 dev 2 function 5 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci7 at ppb6 bus 7
> virtio3 at pci7 dev 0 function 0 "Qumranet Virtio 1.x RNG" rev 0x01
> viornd0 at virtio3
> virtio3: msix shared
> ppb7 at pci0 dev 2 function 6 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci8 at ppb7 bus 8
> ppb8 at pci0 dev 2 function 7 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 22
> pci9 at ppb8 bus 9
> ppb9 at pci0 dev 3 function 0 vendor "Red Hat", unknown product 0x000c
> rev 0x00: apic 0 int 23
> pci10 at ppb9 bus 10
> ppb10 at pci0 dev 3 function 1 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 23
> pci11 at ppb10 bus 11
> ppb11 at pci0 dev 3 function 2 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 23
> pci12 at ppb11 bus 12
> ppb12 at pci0 dev 3 function 3 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 23
> pci13 at ppb12 bus 13
> ppb13 at pci0 dev 3 function 4 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 23
> pci14 at ppb13 bus 14
> ppb14 at pci0 dev 3 function 5 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 23
> pci15 at ppb14 bus 15
> ppb15 at pci0 dev 3 function 6 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 23
> pci16 at ppb15 bus 16
> ppb16 at pci0 dev 3 function 7 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 23
> pci17 at ppb16 bus 17
> ppb17 at pci0 dev 4 function 0 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 20
> pci18 at ppb17 bus 18
> ppb18 at pci0 dev 4 function 1 vendor "Red Hat", unknown product
> 0x000c rev 0x00: apic 0 int 20
> pci19 at ppb18 bus 19
> azalia0 at pci0 dev 27 function 0 "Intel 82801I HD Audio" rev 0x03: msi
> azalia0: No codecs found
> pcib0 at pci0 dev 31 function 0 "Intel 82801IB LPC" rev 0x02
> ahci0 at pci0 dev 31 function 2 "Intel 82801I AHCI" rev 0x02: msi,
> AHCI 1.0
> ahci0: port 2: 1.5Gb/s
> scsibus2 at ahci0: 32 targets
> cd0 at scsibus2 targ 2 lun 0: <QEMU, QEMU DVD-ROM, 2.5+> removable
> ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic
> 0 int 16
> iic0 at ichiic0
> isa0 at pcib0
> isadma0 at isa0
> pckbc0 at isa0 port 0x60/5 irq 1 irq 12
> pckbd0 at pckbc0 (kbd slot)
> wskbd0 at pckbd0: console keyboard, using wsdisplay0
> pms0 at pckbc0 (aux slot)
> wsmouse0 at pms0 mux 0
> pcppi0 at isa0 port 0x61
> spkr0 at pcppi0
> uhidev0 at uhub0 port 5 configuration 1 interface 0 "QEMU QEMU USB
> Tablet" rev 2.00/0.00 addr 2
> uhidev0: iclass 3/0
> ums0 at uhidev0: 3 buttons, Z dir
> wsmouse1 at ums0 mux 0
> vscsi0 at root
> scsibus3 at vscsi0: 256 targets
> softraid0 at root
> scsibus4 at softraid0: 256 targets
> root on sd0a (50928d3f17d03654.a) swap on sd0b dump on sd0b
>
> On 7/25/23 2:37 AM, Kevin wrote:
>> Hey gang,
>>
>> After applying today's zenbleed patches and running fw_update and
>> installboot -v sd0, ALL of our AMD servers running 7.3 at Vultr that
>> were--as part of the patch process--rebooted are now dead in the
>> water and
>> won't boot.
>>
>> Intel Vultr virtual machines appear to be unaffected, as are all of my
>> Intel bare metal machines, laptops, desktops, and servers alike.
>>
>> (Thankfully, with one exception everything running an AMD cpu is
>> redundant
>> OR the patches have been applied but haven't yet been rebooted, so we're
>> not completely dead in the water.... just close.
>>
>> I was waiting for the machines to come up after reboot to reboot the
>> sibling servers, which are now patched but not being rebooted 'til we
>> have
>> a fix.)
>>
>> All machines are single CPU and were rebooted via SSH (and when that
>> didn't
>> work via the Vultr console).
>>
>> All ddbs look similar to these. (Sorry for the screenshots, but it's the
>> best I can do given these are remote virtual instances.)
>>
>> [image: Screen Shot 2023-07-24 at 23.18.40.png]
>> [image: Screen Shot 2023-07-24 at 23.19.10.png]
>> [image: Screen Shot 2023-07-24 at 23.19.27.png]
>> [image: Screen Shot 2023-07-24 at 23.19.40.png]
>>
>>
>> Any developer who wants access to one of our machines is welcome to it.
>>
>> We can also easily provision a new instance with 7.3 so someone can
>> get in
>> and break shit as needed.
>>
>> All affected machines are Vultr's "1024.00 MB AMD High Performance"
>> instances.
>>
>> Available to help any way I can.
>>
>> Thanks,
>> Kevin
>> P.S. If this is a PEBKAC on my part, I'm happy to take the requisite 50+
>> lashes with the clue stick.
>>
>
No comments:
Post a Comment