@whitequark @swetland RP2350 ROM is a mixture of Armv8-M Base and Armv8-M Main, plus an armulator for executing the Armv8-M Base parts on RISC-V

@wren6991 @swetland ohhh okay yeah makes sense

Getting closer... flash agent code is running on the pico2 under debug but blows up when I attempt to call into the ROM to look up ROM functions needed for flash operations. Landing at the infinite loop fault handler at 0x2f8.

Odd to not just get an error. Possibly bad machine state for calling into the ROM (maybe different than the rp2040 requirements?).

Hmmm. Looks like a NOCP exception at 0x74 (first instr of the ROM function).

#Projects #Xdebug

@swetland Yeah, you're hitting the RCP checks. If you don't consider it spoilers you can grab the initialisation stub from here: https://github.com/raspberrypi/openocd/blob/cd4873400c881ce3019c74620afb19e964a1f235/src/flash/nor/rp2xxx.c#L89-L119

@wren6991 Interesting. I've tried attaching with the debugger and calling rom code from both just the bootloader idling (on a factory fresh pico2) and after using the recovery restart feature (figuring that should put it in a known debugger-friendly state). Neither of those cases fire up the RCP? (Sorta figured the usual bootrom paths would).

@swetland It's set up very early as we want to have checks covering as much of the ROM as possible. It's mainly something you have to worry about if you reset a core in the debugger and then want to call ROM code. We also inject that same initialisation stub into RAM binaries for the same reason. Init happens here: https://github.com/raspberrypi/pico-bootrom-rp2350/blob/c6cdb1711f32c3e34faaebd58618a6d096dbd52e/src/main/arm/varm_boot_path.c#L354

The setup is after the RESCUE flag check though.

@wren6991 Ah, I see. RESCUE stops things very early (which makes sense). Besides setting up the RCP and ensuring the basic processor state (stack, psrs, etc) is sane, is there anything else I should worry about before calling the ROM?

(Also, thanks for all the help!)

@swetland In USB boot you have a different set of problems which is that the chip has been locked down to sandbox the USB stack in Non-secure and just have a small Secure stub for handling flash programming calls etc from the NS bootloader.

Mainly you just need to reset ACCESSCTRL and make sure the core is in the Secure state before doing any ROM calls. The ROM also locks itself out of any OTP pages that don't have BL access before running the NS bootloader.

@wren6991 I'm leaning toward using the RESCUE feature to grab control of the part for debugger driven flash and the like. Feels like the simplest way to force the world into a known very-early-start state.

@swetland Check rp2350_init_arm_core0() in the same file for some other traps, like flipping the core to Secure before running an algo in case you halted it in the Non-secure state (like if you interrupted the USB bootloader)

@swetland Yep, that sounds sensible. In that case you should just have to set the CPACR and seed the RCP.

If you're doing RISC-V there is one additional step which is calling the bootrom_state_reset() and set_bootrom_stack() APIs before calling other ROM funcs (needed to set up some re-entrancy state in the armulator, and either grant use of the default armulator stack or provide your own)

@wren6991 Saw the notes about the RISC-V side leaning on ARM emulation in the ROM earlier while poking around in the boot code repo. Did not expect that. Not entirely surprised though (file under the ever-expanding list of "things sw teams get to do to avoid incurring hw costs"...).

@swetland Yep, it's all emulated apart from a few hoisted native functions like memcpy/memset, and a native USB IRQ handler. I think you would enjoy picking apart how that all works 😅

@wren6991 @swetland waitwaitwait, you emulate what?

@swetland The reason it's not just an error is because the RCP is trying to bring the core to a safe halt using a combination of NMI and coprocessor stalls, see datasheet section 3.6

(it might seem like a weird mechanism but there are some limitations on the coprocessor interface, and there was a requirement that the debugger would stay attached after a fault)

@wren6991 @whitequark @swetland I'm sure there was a non-x86 server platform (can't remember if AArch64 or POWER) that had an x86 emulator in the firmware for option ROMs

@kbity @wren6991 @swetland pretty sure it was aarch64, and it even has a special hack in the architecture for supporting x86 video BIOS and such

@ewenmcneill @kbity @wren6991 @swetland https://www.suse.com/c/revolutionizing-arm-technology-x86_64-option-rom-aarch64/

@wren6991 @whitequark @swetland i'm mildly surprised that an armv8-m emulator that runs on risc-v is less bytes of bootrom than simply having the relevant rom functions doubled up for arm/rv32 mode.

How much of a net savings did you get?

@azonenberg @whitequark @swetland I don't remember all the numbers but the emulator core is 2.3 kB, and it's a 32 kB ROM containing mostly Arm code.

@wren6991 @whitequark @swetland I guess it doesn't have to be a particularly complex or fast emulator, and you're just trying to implement the basic v8-m stuff and not all the complex things like NEON or floating point.

@azonenberg @whitequark @swetland @azonenberg @whitequark @swetland The dialect is actually v6-M + b.w, movw, movt, cbz, cbnz. We ifdef'd out the div/divu because there is only one divide instruction in the ROM, and it's not needed on RISC-V (24-bit version thermometer calculations in secure boot)

We found v8-M Base to give better code density than v8-M Main anyways (~1%), because Main ends up using 32-bit opcodes all over the place to access the high registers

@wren6991 @whitequark @swetland Interesting.

And of course you're not trying to JIT or make it fast either so it's probably just a loop and a bunch of bitshifts and nested switch statements or something

@azonenberg @whitequark @swetland Yup, it's a big dispatch table, all the per-instruction handlers tail back into the dispatch

@azonenberg @whitequark @swetland I mean we are trying to make it fast, but within tight size constraints, and in the context of a bootloader that needs to be able to load and verify ~all of RAM so has very little memory of its own 😅 this is the main emulator source: https://github.com/raspberrypi/armulet/blob/master/varmulet/varmulet_armv6m_core.S

@wren6991 @whitequark @swetland I'm still a little surprised and impressed you went that route vs say a 48 or 64 kB dual ISA rom. My recollection is that the bootrom was a tiny fraction of overall die area so the impact on device cost would have been negligible and it'd probably have saved engineering costs.

But it's a clever hack for sure

@azonenberg @whitequark @swetland My canonical answer to this question is "where were you when we decided the ROM size?"

@wren6991 @whitequark @swetland Lol. Yeah if you had set rom capacity in advance and had to work within that constraint rather than writing the code first and finding a memory IP big enough, different story.

@wren6991 @whitequark @swetland I guess the other possible architecture that comes to mind would be having 99% of the rom be running on the arm regardless of boot mode strap, load the image from spi flash or whatever into ram, and only just before jumping to it do an optional mode switch operation poking some SFR that wakes up the hazard3 and shuts down the arm core.

Was that considered and rejected as too complex RTL wise?

@azonenberg @whitequark @swetland We wanted to be able to strap it for Arm-only and RISC-V-only use, even if we never exercised that

@wren6991 @whitequark @swetland ah makes sense.

The entire concept of a switchable isa mcu is a bit mind boggling lol... Has this ever been done before to your knowledge?

@azonenberg @wren6991 @whitequark These two diagrams from the datasheet are useful if you haven't looked into RP2350 previously...

@azonenberg
@wren6991 @whitequark @swetland a russian cpu arch called elbrus can do it too, but from what i can tell is that the cpu translates x86 instructions into the native isa instead of there being two physical cores you can switch between

@charlotte @swetland @whitequark @wren6991 so more like transmeta?

@azonenberg @charlotte @swetland @wren6991 yeah

@azonenberg

@swetland @whitequark @wren6991 and cpu level translation of x86 into a different isa isn't new from them either, the transmeta crusoe did it first but there i do not think you could execute directly in the underlying risc architecture