[Bug]: [QEMU_EFI] Not Loading Network Protocol Drivers and http efi command

[AndreasBBS]

Is there an existing issue for this?

• I have searched existing issues

Bug Type

• Firmware
• Tool
• Unit Test

Code first?

• Yes

What packages are impacted?

NetworkPkg, OvmfPkg

Which targets are impacted by this bug?

DEBUG, NOOPT

Current Behavior

In the latest stable releases I haven't been able to make the X64 Ovmf build load the networking drivers when launching the Ovmf.fd in QEMU.

This is what my debug.log looks like

Driver 6490F1C5-EBCC-4665-8892-0075B9BB49B7 was discovered but not loaded!!
Driver A2F436EA-A127-4EF8-957C-8048606FF670 was discovered but not loaded!!
Driver E4F61863-FE2C-4B56-A8F4-08519BC439DF was discovered but not loaded!!
Driver 025BBFC7-E6A9-4B8B-82AD-6815A1AEAF4A was discovered but not loaded!!
Driver 529D3F93-E8E9-4E73-B1E1-BDF6A9D50113 was discovered but not loaded!!
Driver 94734718-0BBC-47FB-96A5-EE7A5AE6A2AD was discovered but not loaded!!
Driver 9FB1A1F3-3B71-4324-B39A-745CBB015FFF was discovered but not loaded!!
Driver 6D6963AB-906D-4A65-A7CA-BD40E5D6AF2B was discovered but not loaded!!
Driver DC3641B8-2FA8-4ED3-BC1F-F9962A03454B was discovered but not loaded!!
Driver 95E3669D-34BE-4775-A651-7EA41B69D89E was discovered but not loaded!!
Driver 5BEDB5CC-D830-4EB2-8742-2D4CC9B54F2C was discovered but not loaded!!
Driver D912C7BC-F098-4367-92BA-E911083C7B0E was discovered but not loaded!!
Driver 99F03B99-98D8-49DD-A8D3-3219D0FFE41E was discovered but not loaded!!
Driver 1A7E4468-2F55-4A56-903C-01265EB7622B was discovered but not loaded!!
Driver B95E9FDA-26DE-48D2-8807-1F9107AC5E3A was discovered but not loaded!!
Driver B219E140-DFFC-11E3-B956-0022681E6906 was discovered but not loaded!!
Driver 2366C20F-E15A-11E3-8BF1-E4115B28BC50 was discovered but not loaded!!
Driver 22EA234F-E72A-11E4-91F9-28D2447C4829 was discovered but not loaded!!
Driver ECEBCB00-D9C8-11E4-AF3D-8CDCD426C973 was discovered but not loaded!!
Driver 86CDDF93-4872-4597-8AF9-A35AE4D3725F was discovered but not loaded!!
Driver A487A478-51EF-48AA-8794-7BEE2A0562F1 was discovered but not loaded!!
Driver 19618BCE-55AE-09C6-37E9-4CE04084C7A1 was discovered but not loaded!!
Driver AD416CE3-A483-45B1-94C2-4B4E4D575562 was discovered but not loaded!!

Inside QEMU when I run drivers this is what I get:

            T   D
D           Y C I
R           P F A
V  VERSION  E G G #D #C DRIVER NAME                         IMAGE NAME
== ======== = = = == == =================================== ==========
58 0000000A B - -  1  6 PCI Bus Driver                      PciBusDxe
5A 00000010 D - -  1  - Virtio PCI Driver                   VirtioPciDeviceDxe
5B 00000010 ? - -  -  - Virtio 1.0 PCI Driver               Virtio10
5C 00000010 ? - -  -  - Virtio Block Driver                 VirtioBlkDxe
5D 00000010 ? - -  -  - Virtio SCSI Host Driver             VirtioScsiDxe
5E 00000010 ? - -  -  - Virtio Serial Driver                VirtioSerialDxe
5F 0000000A D - -  3  - Platform Console Management Driver  ConPlatformDxe
60 0000000A D - -  3  - Platform Console Management Driver  ConPlatformDxe
61 0000000A B - -  3  3 Console Splitter Driver             ConSplitterDxe
62 0000000A ? - -  -  - Console Splitter Driver             ConSplitterDxe
63 0000000A ? - -  -  - Console Splitter Driver             ConSplitterDxe
64 0000000A B - -  3  3 Console Splitter Driver             ConSplitterDxe
65 0000000A B - -  2  2 Console Splitter Driver             ConSplitterDxe
69 0000000A D - -  1  - Graphics Console Driver             GraphicsConsoleDxe
6A 0000000A B - -  2  2 Serial Terminal Driver              TerminalDxe
6B 0000000A D - -  3  - Generic Disk I/O Driver             DiskIoDxe
6C 0000000B B - -  1  1 Partition Driver(MBR/GPT/El Torito) PartitionDxe
6F 0000000A B - -  1  1 SCSI Bus Driver                     ScsiBus
70 0000000A D - -  1  - Scsi Disk Driver                    ScsiDisk
71 0000000A D - -  1  - Sata Controller Init Driver         SataController
72 00000010 D - -  1  - AtaAtapiPassThru Driver             AtaAtapiPassThruDxe
73 00000010 B - -  1  1 ATA Bus Driver                      AtaBusDxe
74 00000010 ? - -  -  - NVM Express Driver                  NvmExpressDxe
75 00000010 B - -  1  3 OVMF Sio Bus Driver                 SioBusDxe
76 0000000A B - -  2  2 PCI SIO Serial Driver               PciSioSerialDxe
77 0000000A D - -  1  - PS/2 Keyboard Driver                Ps2KeyboardDxe
7A 0000000A D - -  1  - FAT File System Driver              Fat
7B 00000010 ? - -  -  - UDF File System Driver              UdfDxe
7C 00000010 ? - -  -  - Virtio Filesystem Driver            VirtioFsDxe
81 00000010 B - -  1  1 Virtio Network Driver               VirtioNetDxe
82 00000020 ? - -  -  - Usb Uhci Driver                     UhciDxe
83 00000030 ? - -  -  - Usb Ehci Driver                     EhciDxe
84 00000030 ? - -  -  - Usb Xhci Driver                     XhciDxe
85 0000000A ? - -  -  - Usb Bus Driver                      UsbBusDxe
86 0000000A ? - -  -  - Usb Keyboard Driver                 UsbKbDxe
87 00000011 ? - -  -  - Usb Mass Storage Driver             UsbMassStorageDxe
88 00000010 B - -  1  1 QEMU Video Driver                   QemuVideoDxe
89 00000010 ? - -  -  - Virtio GPU Driver                   VirtioGpuDxe
8A 00000010 ? - -  -  - Virtio Random Number Generator Driv VirtioRngDxe

And when I run http inside QEMU I get

'http' is not recognized as an internal or external command, operable program, or script file.

Expected Behavior

I expect the drivers to be loaded and the http command to be present and work.

The last commit I had it working was 5340219. I couldn't quite yet figure out exactly in which commit it stopped working but if you do the steps to reproduce in this commit you'll see it works.

Steps To Reproduce

My building process goes as follows. I create an image for a specific commit of edkii with docker:

FROM ghcr.io/tianocore/containers/ubuntu-22-build:latest AS edk2-main

# Shallow clone of edk2
RUN git clone --depth 1 https://github.com/tianocore/edk2.git edk2
WORKDIR /edk2

FROM edk2-main AS edk2-commit
ARG EDK2_COMMIT
RUN git fetch --depth 1 origin ${EDK2_COMMIT}
RUN git checkout ${EDK2_COMMIT}
# The https://github.com/Zeex/subhook is not maintained anymore, so we use the tianocore/edk2-subhook fork
RUN git submodule set-url -- UnitTestFrameworkPkg/Library/SubhookLib/subhook https://github.com/tianocore/edk2-subhook.git
RUN git submodule update --init --recursive

RUN pip install -r pip-requirements.txt
RUN /bin/bash -c ". ./edksetup.sh " && make -C BaseTools 

In this way I can use docker build -t edkii:edk2-stable202411 --build-arg EDK2_COMMIT=edk2-stable202411 . to create an image with edkii in a specific commit.

I also have a Dockerfile.qemu to run QEMU:

FROM alpine:latest as qemu
RUN apk update && \
    apk add --no-cache qemu-system-x86_64 python3

Which I build with docker build -t qemu:latest -f Dockerfile.qemu.

Then I have a docker-compose.yml with:

services:
  edkii:
    image: edkii:edk2-stable202411
    command: sleep infinity
    volumes:
      - ./build/full:/edk2/Build
      - ./build/dist:/edk2/Dist
      - ./Makefile:/edk2/Makefile

  qemu:
    image: registry.access.softi9.pt/softi9/cuco/uefi/edkii/qemu:35f6a2780e
    working_dir: /app
    volumes:
      - ./build/dist:/app/dist
      - ./logs/qemu:/app/logs
      - ./partition:/app/partition
    command: >
      qemu-system-x86_64 
      -boot splash-time=0,reboot-timeout=0,strict=off
      -smbios type=1,serial=SOFTI9UEFI1
      -fw_cfg name=opt/org.tianocore/X-Cpuhp-Bugcheck-Override,string=yes 
      -nographic 
      -no-reboot
      -serial file:/app/logs/output.log 
      -serial mon:stdio 
      -debugcon file:/app/logs/debug.log -global isa-debugcon.iobase=0x402 
      -bios /app/dist/OVMF.fd
      -drive file=fat:rw:/app/partition,format=raw
      -netdev user,id=net0
      -device virtio-net-pci,netdev=net0,rombar=0

And the following Makefile:

ovmf_networking:
  OvmfPkg/build.sh \
  -a X64 \
  -b DEBUG \
  -D DISABLE_NEW_DEPRECATED_INTERFACES \
  -D NETWORK_ENABLE=TRUE \
  -D NETWORK_HTTP_ENABLE=FALSE \
  -D NETWORK_SNP_ENABLE=TRUE \
  -D NETWORK_VLAN_ENABLE=TRUE \
  -D NETWORK_IP6_ENABLE=TRUE \
  -D NETWORK_TLS_ENABLE=TRUE \
  -D NETWORK_HTTP_BOOT_ENABLE=TRUE \
  -D NETWORK_ALLOW_HTTP_CONNECTIONS=TRUE \

  cp -r Build/OvmfX64/DEBUG_GCC/FV/OVMF.fd Dist

Which allows me to build with docker compose run --rm edkii make ovmf_networking and run the output OVMF.fd in QEMU with docker compose run --rm qemu

The debug logs with the "Driver ??? was discovered but not loaded!!" messages will be in the folder logs/qemu.
To get the list of drivers I run drivers in the EFI SHELL. Same for the http command. Also the ifconfig command gives no output.

Build Environment

- OS(s): Ubuntu
- Tool Chain(s): GCC5

Version Information

Tag: edk2-stable202411

Urgency

Low

Are you going to fix this?

I will fix it

Do you need maintainer feedback?

No maintainer feedback needed

Anything else?

I've been struggling with this bug but I've been keeping my work in the 5340219 commit. I'm at a point at my exploration where I can't seem to get the TLS working in that commit so I'd like to try more recent versions of the NetworkPkg as I've noticed that a lot of development has been happening there but for some reason I can't get it to build in a way that QEMU will load the drivers... Any help will be highly appreciated and I really hope it's not something really obvious that I miss that's now wasting someone's time. Thanks for the attention in advance!