Android will be using ext4 starting with Gingerbread

I received a trackback from Tim Bray’s Saving Data Safely post on the Android Developer’s blog to my Don’t fear the fsync! blog entry, so I guess the cat’s out of the bag.  Starting with Gingerbread, newer Android phones (starting with the Nexus S) will be using the ext4 file system.  Very cool!  So just as IBM used to promote Linux by saying that it was scalable enough to run on everything between watches and mainframes, I can now talk about ext4 as running in production on cell phones to Google data centers.

How much am I worried about Tim Bray’s caution to Android programmers that they use fsync() correctly?   Not a lot.  Sure, they should make sure they use fsync(), or if they want to be clever, sync_file_range(), to make sure files are appropriately written to disk (or, in Android’s case, to flash).   But unlike Ubuntu’s running on random PC’s, with users downloading the latest (possibly buggy) Nvidia drivers, handset manufacturers test their systems very carefully before they let them ship.  So if there are any problems, they tend to be found before the phone ships to end-users.  So at least in my experience, my Nexus One has been very reliable; it’s never crashed on me.   So the chances of random crashes when you exit the 3D game (as one Ubuntu user reported — and considered acceptable?!? — I’d be roasting Ubuntu and/or Nvidia under a slow fire if that was a reliably reproducible bug, not considering it par for the course) are very remote.   And fsync() is important if systems crash or aren’t shut down cleanly.  Still, if users are randomly ripping out their batteries to turn off their cell phone in a hurry, because they’re too impatient to wait for a controlled shutdown, then sure, we might have problems, and it’s a good reason for application writers to use fsync() correctly.

By the way, I had nothing to do with the choice to use ext4 on Android.  So if you’re curious about why ext4 was chosen, I can’t say anything authoritatively, since I wasn’t consulted before the decision was made (although obviously I’m delighted).   As far as I can tell after the fact, one of the reasons for choosing ext4 was better performance, especially in the light of dual-core ARM CPU’s which are becoming available in large quantities in the near future; YAFFS is single threaded, so it would have been a bottleneck on dual-core systems.  Why not btrfs?   Well, for all of btrfs’s features, it’s not out of beta yet, and Gingerbread is shipping today.  (Well, in less than a week to Best Buy, if we want to be precise.)

This is another reason why I’m glad to see ext4 being used on Android is that it validates my decision to keep working on ext4 2-3 years ago, even though newer filesystems like btrfs were promised to be coming down the pike.   As I’ve said many times before, file systems are like fine wine, and they take many years to mature.  So having ext4 ready today is a great way of giving more choices to developers and system administrators about what file system they want to use.

27 thoughts on “Android will be using ext4 starting with Gingerbread

  1. “if users are randomly ripping out their batteries to turn off their cell phone in a hurry, because they’re too impatient to wait for a controlled shutdown”

    No, that’s not how it happens. The typical situation is “I’m using my phone and by accident I let it fall to the ground, and the battery escapes”. Sometimes it’s just a short loss of contact, and the phone ends up in a buggy state (frozen or other weird behaviour).

    All this is ok as long as people only lose the data that was just being written. Losing your years-old contacts because a new one was being added is a no-no (don’t expect customers to backup their important data, they will only do it when they buy a new phone).

  2. > So at least in my experience, my Nexus One has been very reliable; it’s never crashed
    > on me. So the chances of random crashes when you exit the 3D game are very remote.

    My Samsung Galaxy (not S, not Spica, the old crap that hardly received official updates) has crashed on me quite a few times, not even doing anything fancy.

  3. This also mean there moving from raw nand (mtd + yaffs) to inand/emmc (sdcard).
    Performance and reliability will depends of the FTL of the emmc.

    Some are crap, other optimized for FAT but maybe some will work fine. ..

  4. My Nexus One has been great to me this past year, but even I have to say mine has crashed every now and then or even frozen to the point where I’ve had to remove the battery in order to get it to respond to anything!

  5. @1: Corsac,

    They did consult me on a few issues, but it was after they had made the decision to use ext4. They had actually done quite a lot of work before they pulled me in, including making an Apache-licensed version of mke2fs (some of our handset and carrier partners are scared of GPL’ed licensed userspace, or so I’ve been told. It doesn’t make any sense, but sometimes it’s easier to fix a perception issue with engineering than it is to try to convince an executive that no really, the GPL isn’t all that scary, especially since the kernel is GPL’ed).

    @5: The typical situation is “I’m using my phone and by accident I let it fall to the ground, and the battery escapes”.

    Good point. I’ve had pretty good luck with my Nexus One (despite my butter fingers), but the battery has escaped once or twice. So yes, it’s something to be concerned about.

    I should check whether Gingerbread forces a sync(2) when the system suspends and the screen is turned off. Otherwise, the data could be left buffered for a long time and then if the device is dropped, data written potentially hours earlier might get lost if the application is badly written. Of course, it’s the application author’s responsibility to call fsync(), but it would be a good backup to have for safety’s sake.

  6. Well I’ve had my incredible eat itself and start force closing apps over and over and over due to corrupt yaffs file system from pulling the battery to swap out with another… Hope this doesn’t mean it’ll be more prevalent to data loss.

  7. Hi Ted.

    Thanks for exposing your toughts on filesystems. I noted that you didn’t mention fs like UBIFS, what’s your opinion about that, since they are designed to work on flash memory? I made some tests (not a scientific benchmark) and noted boot time of Android (2.2) running UBIFS are very quickly that worth mention.

    –tm

  8. How does ext4 compare against jffs2 or ubifs that are log structured file systems designed specifically for flash ?

  9. I’m still a bit surprised they didn’t include you earlier. If I had the ext4 developer working in my company, I guess I would ask him about that as soon as I would have the idea to use it, not after rewriting mk2efs :)

    About sync(), another good idea would be to call it when the accelerometer detects a drop. Good laptops already have that feature embedded in the hard disks or in the ec to park the heads when they detect a fall (but for mechanical reasons). There’s no problem with head ripping on the disk surface when you use flash, but it’s still a good idea to sync() at that time.

  10. Interesting, does fsync get down to the mmc layer? When we we’re working on the g2 (#g2root) we ran into issues where sync might not actually force data to be written, as mmc doesn’t report errors that aren’t protocol errors (this is actually how the write protection on the G2 worked, the eMMC is write protected and the kernel block layer just caches data). We are also working with cjb on this issue. — tmzt

  11. > About sync(), another good idea would be to call it when the accelerometer detects a drop.

    That would be the contrary of a good idea: if you detect a drop, an actual problem (e.g. battery escape) is very likely to happen within a second. sync() is likely to take more than that, possibly cutting power *during* sync(), which is IMHO worse than not sync()ing at all.

  12. Actually all this is ONLY due to the fact that the Nexus S has a so called FTL, aka cannot be directly addressed via YAFFS for example.

    So a normal FS had to be chosen, and EXT4 is the most logical choice – period.

    Has nothing to see with this whole “wow its so cool they change for EXT4 cause its better and stuff”

    Nope. They use EXT4 for the same reason Ubuntu uses EXT4, simple as that.

  13. @16: Zu

    … and since the Nexus S is using a fixed storage, they’re also using ext4 for the data partition, instead of FAT. And yes, it’s no news that ext4 is a superior filesystem to FAT. But I still think it’s cool that ext4 is now being used from devices as small as cell phones and as large as Google data centers. I’m not asking for anyone else to think it’s cool; but I still think it’s cool. That is all. :-)

  14. Personally, I’d be quite happy if I had the option of formatting the external storage as ext4 (or any other non-patent-encumbered, non-FAT filesystem for that matter).

    Someone would need to write a “present the filesystem as though it were FAT” layer so that Windows users could read and write to it, but it’d be nice anyway.

  15. Vs. UBIFS: take a look here: http://linux-mtd.infradead.org/doc/ubifs.html#L_rednote

    Ted put wrong emphasis I think: “YAFFS is single threaded, …”. But I guess they are switching from raw NAND to something like eMMC, and they simply cannot use YAFFS or UBIFS any more, so they had to choose a block-based FS, and selected ext4.

    IOW, YAFFS, JFFS2, UBIFS are FSes “from different world”, and questions like “why they selected ext4 but not UBIFS” are incorrect.

  16. Yup, our main motivation here was to select a filesystem to work on top of emmc which has built-in block rotation and translation and looks like a regular block device (similar to ssds on desktop systems). At this point ext4 seems to our eyes to be the most stable and performant option of the standard linux filesystems that support full unix permissions and semantics.

    We use ext4 for the system image (read only), and the local user data and cache partitions (read-write). A separate FAT partition is used for media storage, since that’s still the least-common-denominator needed for USB Mass Storage to “Just Work” with PCs and Macs.

    For post-gingerbread we’re investigating alternatives to UMS which would allow us to stop using FAT for media/external storage entirely.

  17. “some of our handset and carrier partners are scared of GPL’ed licensed userspace, or so I’ve been told. It doesn’t make any sense, but sometimes it’s easier to fix a perception issue with engineering than it is to try to convince an executive that no really, the GPL isn’t all that scary, especially since the kernel is GPL’ed).”
    To be more precise, the GPLv2. I have wished that Google would make Android GPLv3, particularly after this blog article:
    http://android-developers.blogspot.com/2010/12/its-not-rooting-its-openness.html

  18. Bad election from the Android team.
    The right election would be to migrate Android to FAT32… hahaha
    just kidding… :)
    i think its a great decision.
    i will migrate my home file server from XFS to ext4 in a couple of weeks…
    waiting to Squeeze prior this migration
    cheers!

  19. Isn’t ext4 for FLASH based media kind of a stupid choice ?

    Why not use NILFS2 for example ? EXT4 is not optimized for FLASH, not to mention the fact that it tends to write some infrastructure blocks much more frequently than others.

    I had Gentoo on cheap usb stick and while ext3 was unuseable and soonish destroyed the key, NILFS2 simply flied.

    Its snapshotting capabilities come as an extra bonus…

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>