I knew Firefox was getting bloated, but that’s a bit excessive…

(If the kernel can’t implement fbarrier() for a particular scenario, it should return an error. In practice, it would probably need to take an array of fds, and it should be possible to tell in advance whether fbarrier() will work with a particular set of FDs, to select an appropriate writing strategy. Anyhow, we’re not here to design an API that will probably never be implemented, but none of this is very difficult to define sensibly.)

I accept that an fbarrier() api would be convenient and would substantially improve things for many usage profiles. But, what is the fallback strategy when running in an environment where the fbarrier() api is missing, ineffective, or where the api knows it cannot provide the expected guarantees? Merely having the fbarrier execute flush and sync type operations will result in worse behavior, because the app writer just blindly called fbarrier considering that it would do the right thing.

I generally find that hiding complexity from the developer is a bad thing. It leads to him being unaware of what the machine is actually doing, and often leads to wildly inaccurate assumptions about performance and safety.

From #16 above (I think)
> If we are peppering our code with fsync’s, even if it doesn’t hurt “that much”, we are violating the abstraction that says the kernel is supposed to take care of buffering, caching, and writing things out to disk in a sane way.

The problem is that the kernel cannot know what each developer means by “a sane way”, and kernel behavior that is correct for one usage case is totally wrong for another. The behavior that is correct for a high load webserver is almost certainly wrong for a critical log path, and neither behavior is really quite right for a responsive medium-importance gui app. Adapting the kernel and libs to assume any one of these jobs is going to break more things than it fixes. This suggests to me that the expected kernel abstraction has gotten a little too abstract.

Perhaps what is truly needed is to document correct recipes for all the different intended-behavior cases then make sure the kernel doesn’t suddenly regress any of the expected behaviors. It seems to me that a lot of people are counting on behavior that may or may not be in but that certainly are NOT in ext2. When something other than their favorite assumed fs behaves differently, they go around crying “Bug!”. I agree that there is a bug, but perhaps we disagree as to which code contains it

Adding an fbarrier()-like API would not inherently make anything less suitable for any other task. It might in practice, because implementing it may be difficult and cause internal design changes that could have adverse effects; but there’s nothing *inherent* about it that would do that.

Right now, we have a drill, and the hammer hasn’t been invented. The only means we have available to bang nails (safely write files) is to hit them with a drill (call fsync). There are no hammers (fbarrier).

I’d argue that SQLite is (when correctly configured) quite smart and robust. However, I’d also argue that it is not the correct tool for the job the Firefox developers had in mind. It is, however, quite simple to use and that apparently makes it the best choice for the task.

Part of the problem here is that everyone seems to assume that there is only one valid kind of problem, and that only the solutions that trade other things to maximize capability for *that* kind of problem are valid ones.

Another problem is that plenty of app developers seem willing to assume that the product will always run on Linux (by which they really mean “Always run on a linux box using a given filesystem, configured in a given way”). Many of them also seem to assume that “Linux” (see above) should return the favor by becoming perfectly optimal for their task at the expense of every other problemspace that might potentially ALSO be using the Linux kernel and libraries.

It is frustrating to watch someone take a perfectly good electric drill, use it as a hammer, complain vociferously that it’s awkward and far too complex for the job, and proceed to re-engineer it permanently into a bad hammer. Some of us occasionally need to drill holes or install drywall, and would really have liked to use the drill as a drill.

There’s a similar issue using Vim on a busy system. Writing a file can block the editor for several seconds, because it–correctly–uses a usual safe write sequence (a different one, since it’s overwriting the whole file). With write barriers, it could get safe writes without blocking, so I wouldn’t have to wait for several seconds, breaking my train of thought, as Vim freezes up on fsync.

> I wish the firefox developers would design their data I/O in a more robust and less braindead way overall

So now SQLite is braindead and unrobust. Right.

Maybe an eventual solution is to migrate /home to a log-based FS (say, nilfs2, now in 2.6.30) which internally does all this “snapshotting” as part of normal operations. Alternatively, one could write a userland library which does this for configuration files. Really, this means “open config files in append mode, and use a re-playable format”.