Hacking Hardware Picofly - a HWFLY switch modchip

[Piorjade]

-They have a 1 second watchdog timeout, and if the watchdog triggers they set the LED colour to red(?) and reset the SOC.

-Before this it (I think) waits up to 300ms for a sequence on the CMD/gpio27 pin. It has a bunch of other logic involving the time since startup that would probably make more sense if I knew how the hwfly worked. It's here that it sets the LED colour to blue which represents "glitching".

Thanks for your input. From the limited testing that we did, we can probably say that the sequence it waits on the CMD pin is most likely six times emmc "CMD63", this is a OEM reserved command per emmc specification and thus we don't exactly know what it does.

This might be inaccurate though, the guy that had the test setup with a rp2040 and the ubuntu fw didn't have a proper logic analyzer so we used another Pico set up as a logic analyzer with sigrok and it isn't always 100% accurate.
We basically analyzed the boot 2 times, both times it looked like the emmc got initialized and then CMD63 gets issued six times, with the CPU pin going HIGH after the 6th CMD63 got sent.

Post automatically merged: Mar 1, 2023

-They do seem to be changing the clock speed to 333Mhz and setting the SOC voltage to 1.30 volts. I can't exactly tell what it's doing due to the abundance of divmod_u32u32 calls that I don't want to sift through.

Do you mean they set the whole RP2040's clock speed to 333Mhz directly upon booting? Also, do you really mean Mhz or did you mean kHz?

 

[ghjfdtg]

Thanks for your input. From the limited testing that we did, we can probably say that the sequence it waits on the CMD pin is most likely six times emmc "CMD63", this is a OEM reserved command per emmc specification and thus we don't exactly know what it does.

This might be inaccurate though, the guy that had the test setup with a rp2040 and the ubuntu fw didn't have a proper logic analyzer so we used another Pico set up as a logic analyzer with sigrok and it isn't always 100% accurate.
We basically analyzed the boot 2 times, both times it looked like the emmc got initialized and then CMD63 gets issued six times, with the CPU pin going HIGH after the 6th CMD63 got sent.

That's almost certainly not correct. To properly capture the traffic between SoC and eMMC you need 3 times the bandwidth. So about 600 MSa/s (megasamples per second or alternatively 600 MHz) sampling rate. The only logic analyzers capable of this are the ones claiming 1 GHz and they are expensive.

In reality it's sending CMD17/18 (READ_SINGLE_BLOCK/READ_MULTIPLE_BLOCK) to read blocks of data from different locations.

 

[Mansi]

Thanks for your input. From the limited testing that we did, we can probably say that the sequence it waits on the CMD pin is most likely six times emmc "CMD63", this is a OEM reserved command per emmc specification and thus we don't exactly know what it does.

This might be inaccurate though, the guy that had the test setup with a rp2040 and the ubuntu fw didn't have a proper logic analyzer so we used another Pico set up as a logic analyzer with sigrok and it isn't always 100% accurate.
We basically analyzed the boot 2 times, both times it looked like the emmc got initialized and then CMD63 gets issued six times, with the CPU pin going HIGH after the 6th CMD63 got sent.

Post automatically merged: Mar 1, 2023

Do you mean they set the whole RP2040's clock speed to 333Mhz directly upon booting? Also, do you really mean Mhz or did you mean kHz?

CMD is needed to initialize the type of memory, so that the controller understands what it is working with and whether there is any memory at all.
DLK is needed to coordinate the clock frequency of operation.
If you say that the chip is being polled 6 times, it becomes interesting. Or the connection is lost or there is a very fast reset of the chip. Perhaps during this time there is a recording of the emmc itself.

Post automatically merged: Mar 1, 2023

That's almost certainly not correct. To properly capture the traffic between SoC and eMMC you need 3 times the bandwidth. So about 600 MSa/s (megasamples per second or alternatively 600 MHz) sampling rate. The only logic analyzers capable of this are the ones claiming 1 GHz and they are expensive.

In reality it's sending CMD17/18 (READ_SINGLE_BLOCK/READ_MULTIPLE_BLOCK) to read blocks of data from different locations.

Error, center frequency 52 MHz +-

 

[Piorjade]

That's almost certainly not correct. To properly capture the traffic between SoC and eMMC you need 3 times the bandwidth. So about 600 MSa/s (megasamples per second or alternatively 600 MHz) sampling rate. The only logic analyzers capable of this are the ones claiming 1 GHz and they are expensive.

In reality it's sending CMD17/18 (READ_SINGLE_BLOCK/READ_MULTIPLE_BLOCK) to read blocks of data from different locations.

I don't think we need to run at 3 times the bandwidth to be able to sniff/analyze the traffic. Otherwise this whole Picofly project wouldn't be possible.
The RP2040 can't run that high without frying itself, you'll also need to cool it to extremes

Post automatically merged: Mar 1, 2023

If you say that the chip is being polled 6 times, it becomes interesting. Or the connection is lost or there is a very fast reset of the chip. Perhaps during this time there is a recording of the emmc itself.

I doubt that we lost connection and that there was a very fast init between each CMD63.
On init, per eMMC specification, you have to run 0-400kHz (on stock boot we saw a CLK freq of around 500khz, so the switch uses a bit higher CLK freq but still very managable for the rp2040). So we would've seen the inits, even with this trashy pico LA setup.
So basically either it wasn't CMD63 (I honestly think it really was CMD63 EDIT: look below) and just read_blck or it really was CMD63 and we just don't know what CMD63 does EDIT: look below.
Either way, I believe that the ubuntu fw really looked for this "6x this command" pattern +- some offset to trigger the CPU pin with X pulsewidth.

Post automatically merged: Mar 1, 2023

(I honestly think it really was CMD63)

I'll correct myself. I've taken a look at the LA recording again and yes, it might be READ_BLOCK, not CMD63. Pulseview is just messing with me and displays the CMD incorrectly. There was also data on the DAT0 line after each of these commands.
Anyway, still, there was 6x the same command, after the 6th command + a small delay the RP2040 pulled CPU high for a short amount of time. Then the recording ends because the LA couldn't sample more.

 

[ghjfdtg]

I don't think we need to run at 3 times the bandwidth to be able to sniff/analyze the traffic. Otherwise this whole Picofly project wouldn't be possible.
The RP2040 can't run that high without frying itself, you'll also need to cool it to extremes

The PIO program they used will synchronize with the clock which is why it can get away running at 333 MHz.

Let's pretend your door bell is broken and you check every 10 minutes for guests. What if your guests arrive right inbetween your last 2 checks? They will wait a few minutes and go away thinking you are not at home. Next time you check you won't even notice you had guests.
This is what happens with logic analyzer captures if your sample interval is too low. A few logic analyzers support sampling on clock edges but this is also a feature you mostly find on higher end ones.

edit:

Error, center frequency 52 MHz +-

Nope, see HS400 mode in the eMMC specification.

 

[Mansi]

Nope, see HS400 mode in the eMMC specification.

my bad)

 

[Piorjade]

Let's pretend your door bell is broken and you check every 10 minutes for guests. What if your guests arrive right inbetween your last 2 checks? They will wait a few minutes and go away thinking you are not at home. Next time you check you won't even notice you had guests.
This is what happens with logic analyzer captures if your sample interval is too low. A few logic analyzers support sampling on clock edges but this is also a feature you mostly find on higher end ones.

edit:

Nope, see HS400 mode in the eMMC specification.

I understand what you mean, but, again, the emmc CLK is (per JEDEC's specification) supposed to run 0-400khz. With the pico LA setup running at 120Msps this is more than enough to capture the boot procedure.

The PIO program they used will synchronize with the clock which is why it can get away running at 333 MHz.

I know that it will synchronize with the Tegra's CLK. But even with that, the CLK cannot be running that fast when the 6 commands were issued.

At least I haven't seen the command to switch to a higher frequency.

 

[ghjfdtg]

At least I haven't seen the command to switch to a higher frequency.

Look for CMD6 (yes it really is called SWITCH) and see the argument bits.

 

[Piorjade]

Look for CMD6 (yes it really is called SWITCH) and see the argument bits.

This is the order of commands that I can see:
CMD0, CMD1 (multiple times), CMD2, CMD3, CMD9, CMD7, CMD13, [CLK STOPS FOR A BRIEF TIME, THEN RETURNS AT A SLIGHTLY HIGHER FREQ], CMD31 (?) and at the same time something is coming on the DAT0 line, reply to CMD31(?) arrives and the clk stops briefly again, then the BLKread commands get sent and data arrives after each of these commands

 

[Mansi]

i hope, rehius upload source code

 

[Piorjade]

This is the order of commands that I can see:
CMD0, CMD1 (multiple times), CMD2, CMD3, CMD9, CMD7, CMD13, [CLK STOPS FOR A BRIEF TIME, THEN RETURNS AT A SLIGHTLY HIGHER FREQ], CMD31 (?) and at the same time something is coming on the DAT0 line, reply to CMD31(?) arrives and the clk stops briefly again, then the BLKread commands get sent and data arrives after each of these commands

And after 6 of these BLKread (some of them might also be blkwrite, I can't tell because the LA probably was too slow for it at that point, the "curves" all looked the same at least) the CPU pin gets driven high

 

[vittorio]

i hope, rehius upload source code

@rehius

 

[Mansi]

@rehius

Right. If the source code is available, third-party development will speed up.

 

[Piorjade]

Right. If the source code is available, third-party development will speed up.

Sorry I forgot about that, what did rehius do?

 

[Mansi]

Sorry I forgot about that, what did rehius do?

Published a firmware that can run hecate and ubuntu

Post automatically merged: Mar 1, 2023

https://gbatemp.net/threads/pikofly...-modchips-or-not.622701/page-34#post-10066639

 

[Piorjade]

Published a firmware that can run hecate and ubuntu

Post automatically merged: Mar 1, 2023

https://gbatemp.net/threads/pikofly...-modchips-or-not.622701/page-34#post-10066639

aah that was him

Post automatically merged: Mar 1, 2023

This is the order of commands that I can see:
CMD0, CMD1 (multiple times), CMD2, CMD3, CMD9, CMD7, CMD13, [CLK STOPS FOR A BRIEF TIME, THEN RETURNS AT A SLIGHTLY HIGHER FREQ], CMD31 (?) and at the same time something is coming on the DAT0 line, reply to CMD31(?) arrives and the clk stops briefly again, then the BLKread commands get sent and data arrives after each of these commands

ookay, so, we tested again with proper Pulseview settings and this time with a OLED HWFLY and I finally saw the CMD6
it does not switch to HS200 / HS400 @ghjfdtg ... it switches to normal high speed mode and continues with about 25mhz

Then, it does a CMD6 again and switches to 8bit bus width

Aaaand then it switches to boot0 R/W mode

After that follow CMD17s, then our recording stopped, sadly. We also forgot to record with the glitch pin so idk where it glitches

 

[ghjfdtg]

ookay, so, we tested again with proper Pulseview settings and this time with a OLED HWFLY and I finally saw the CMD6
it does not switch to HS200 / HS400 @ghjfdtg ... it switches to normal high speed mode and continues with about 25mhz

That's weird. It can technically use 52 MHz in normal high speed mode. I guess that's better for us. Saves money on expensive equipment.

The hardware does support HS400 mode. Both the open source driver used in hekate/Atmosphere and Nintendos driver do take advantage of it.

 

[Mansi]

25mhz

Such a frequency is needed for the stability of DAT0.
After all, people use different wires and can be interference.
At this frequency, we will have a stable connection with the EMMC.

 

[Piorjade]

That's weird. It can technically use 52 MHz in normal high speed mode. I guess that's better for us. Saves money on expensive equipment.

The hardware does support HS400 mode. Both the open source driver used in hekate/Atmosphere and Nintendos driver do take advantage of it.

yep, but it seems like that doesn't matter for the glitching part because as far as I know, after that capture hekate booted so the CMD17s we're seeing here are most likely already coming from the tegra

if they are, then that means we probably won't even need to overclock. with 120mhz CLK speed one PIO instruction will take 8ns, so the smallest CPU pulse will theoretically be 8ns

speaking of which, this is in the HWFLY-NX source code in glitch.c at line 217:

    const uint16_t erista_offsets[] = {825, 830, 835, 840, 845, 850, 855, 860, 865, 870, 875, 880, 885, 890, 895, 900, 905};
    const uint16_t mariko_offsets[] = {800, 805, 810, 815, 820, 825, 830, 835, 840, 845, 850, 855, 860, 865, 870, 875, 880};

these are the timing offsets for the glitching, that's cool and all but I've got one question:
What time scale is that? Like, is the first mariko offset 800ns or is it 800 * X ns or what?

 

[Mansi]

I want to make it easier to understand how the current firmware changes boot0.
I want to do a full emmc dump of a clean console and then run hecate with rp2040 on it. In it, I will dump boot0 and be able to analyze what the firmware has changed.

I did this trick with my old Erista console, I think it will work with OLED too)