Extremely Slow SD Card Access on Android 5 (Lollipop)

In Android 4.4 (KitKat) they decided to restrict apps from writing into arbitrary folders on the external SD card. This decision causes a lot of pain for users who updated their expensive smartphones to the new firmware only to find out the SD card, possibly the main reason why they chose this device model and not some others, becomes practically useless. Understandably many users are quick to blame the lazy app developers who don’t bother to update their apps for the new OS version. I myself spent countless hours replying to angry user emails. Adding a message into the app itself to warn users about this problem and link to a detailed document explaining what happened is not enough to calm them down. Frankly I am not surprised. When was the last time you did system update after being asked by the device manufacturer and ended up losing an important feature that worked perfectly well when you bought the device merely two months earlier?

Android 5 (Lollipop) adds a new API for apps to write to the SD card. It’s unbelievably complicated API to achieve something developers could do pre-KitKat:

OutputStream os = new BuferredOutputStream(new FileOutputStream("/storage/extSdCard/foo/bar/baz.txt");
os.write(....);
os.close();

The implementation on some popular devices is buggy. The I/O sematics is much poorer than what normal random access file can do. No seek, no append. But at least something.

To write to the folder subtree app presents a system chooser dialog for the user to select the folder (e.g. /storage/extSdCard/foo). The system returns fooTreeUri. This corresponds to folder foo. To get to baz.txt app needs to traverse down, at each level asking for the correspoinding URI. There is a helper class DocumentFile in android support v4 library to make things a bit simpler and to ease transition from older Android versions:

DocumentFile fooDir = DocumentFile.fromTreeUri(context, fooTreeUri);
DocumentFile barDir = fooDir.findFile("bar");
DocumentFile bazFile = barDir.findFile("baz.txt");
OutputStream os = context.getContentResolver().openOutputStream(bazFile.getUri());
os.write(...);
os.close();

The catch? DocumentFile.findFile() is slow. It loads the list of all entries in folder foo into RAM, then iterates through the list to find the entry with a matching displayName. (Bonus point: it’s case-sensitive matching, equals not equalsIgnoreCase even though everywhere else filenames in Android are case-insensitive. Fun, isn’t it?)

If folder foo contains media files, say JPEG photos, findFile becomes extremely extremely slow. It delegates to DocumentContracts.buildChildDocumentsUri which in turn queries the concrete document provider. It seems this provider brings up the whole media framework machinery, re-scans and re-indexes all those photos. On my LG G3 with a good SD card a single findFile call takes close to 10 seconds if folder foo contains 500 photos. It’s 30 seconds for a folder with 1000 photos. It’s almost as slow as listing files from a cassette tape on Commodore in the eighties.

I poked around to see if it’s possible to let go DocumentFile and use DocumentsContract directly in a more efficient way. Nope. Nothing there.

I made a fix for this problem in the new OneSync v2 app (Google Play link). The new version 2.0.5 will be live on Google Play in a few hours. If you use OneSync to sync a folder on SD card in your Android 5.0 device with Microsoft OneDrive, you should update right away. Depending on how many files are in that folder the speedup can easily be several orders of magnitude faster (50x-100x faster). In other words it was that slow. The fix is: avoid the new Lollipop API for SD card unless you ABSOLUTELY have no other choice.

I’ll update Dropsync, Autosync for Google Drive, and Auto Box Sync to include this fix in the next few days.