A call to devs: Stop using libfat

[V10lator]

So why this call? Simple: Cause of caching issues. libFAT use its own cache, so distracts itself from CafeOS/IOSUs FAT driver. This leads to libfat and IOSUs FAT driver having different on-disc states in their caches which leads to filesystem corruptions.

Examples:

• Currently the Wii U needs a restart after updating a homebrew with Wii U FTP Server (cause the homebrew launchers libfat will load from its cache instead of the SD card?).
• SaveMii WUT has crashing issues since using libfat (exact source unknown but maybe cause SaveMii utilizes MCP which uses IOSU FAT).
• Tried to switch to libfat with NUSspli one time. The result was horror (homebrew codes and MCP accessing the SD card at the same time, leading to horrible filesystem corruptions. Example:
  

  /wiiu/apps/NUSspli  and
  /install/Yoshi's Island_ Super Mario Advance 3 [0005000010156a00]
    share clusters.

  )

While the impact on these caching issues might be extremely small (how many homebrews are using their own FS access while also utilizing OS libraries which do their own FAT access?) now Aroma will have an FTP plugin running always in the background so problems might rise in the future.

 

[QuarkTheAwesome]

Currently the Wii U needs a restart after updating a homebrew with Wii U FTP Server (cause the homebrew launchers libfat will load from its cache instead of the SD card?).

I'm not sure the libfat page cache continues to exist after the parent app exits (in this case HBL, which I am pretty sure doesn't use libfat anyway?). Assuming the app in question doesn't do something silly like not unmounting the card it should all get flushed.

Race conditions with MCP sounds like a real issue though. Can't we unmount the card on the IOSU side? I thought we had to manually request for it to be mounted in the first place.

 

[V10lator]

Race conditions with MCP sounds like a real issue though. Can't we unmount the card on the IOSU side?

That would break tools like WUP Installer.

I thought we had to manually request for it to be mounted in the first place.

Each WUT homebrew auto-mounts the card:
https://github.com/devkitPro/wut/blob/master/libraries/wutcrt/wut_crt.c#L19
https://github.com/devkitPro/wut/blob/master/libraries/wutdevoptab/devoptab_fs.c#L77

Also keep in mind that Aroma will have that FTP plugin running always in the background. This FTP plugin will have SD access, too, so the card will be mounted all the time.

 

[QuarkTheAwesome]

Each WUT homebrew auto-mounts the card:
https://github.com/devkitPro/wut/blob/master/libraries/wutcrt/wut_crt.c#L19
https://github.com/devkitPro/wut/blob/master/libraries/wutdevoptab/devoptab_fs.c#L77

Fair. wut already supports replacing the built-in socket impl using weak symbols, maybe something similar should be added for the devoptab, which libfat can then hook into. Not only would this prevent the automount, this would also make development of apps using libfat less error-prone since the crt handles setup and teardown (at the potential cost of HaxchiFW compatibility, but who cares?)

Someone who isn't blacklisted (like I almost definitely am ) should ask them about that. @Maschell ?

Also keep in mind that Aroma will have that FTP plugin running always in the background. This FTP plugin will have SD access, too, so the card will be mounted all the time.

This would be a big problem, yes. Maybe the FTP plugin and/or the PluginSystem could have a callback to request it stop all access to the SD card?

I'm more inclined to fix things rather than stop using libfat outright, it's such a significant performance boost that it's worth investing into.