Comparing OLS and Usenix
Yesterday was the last day of
href=”http://linuxsymposium.org/”>Ottawa Linux Symposium, and I
can’t help comparing it with the
href=”http://www.usenix.org/publications/library/proceedings/usenix04/”>Usenix
Annual Technical Conference, which was only a few weeks earlier. The
difference in the quality and relevance (at least as far as my interests
are concerned) of the papers, the energy of the attendees, and the
overall atmosphere of the two conferences were distinctly different.
A look at this year’s
href=”http://www.finux.org/proceedings”>conference proceedings for
OLS is enlightening; two volumes, with a total of 600 pages and 50
papers. In the past, some academics had sneered at OLS and
other Linux conferences because the papers were not well written, at
least by academic standards. This criticism is now much less
valid — if it ever was valid at all. As Clem Cole has pointed out, the NFS
paper, which is now considered seminal, is by academic standards a lousy
paper, and it is doubtful it would have been accepted by today’s program
committees under the guise of “raising quality of Usenix’s
papers”. More importantly for OLS however, beyond the
quality of the writing, was how vital and relevant the topics of the
papers.
Two people (one from IBM, and one from EMC) independently commented
to me that they had thought they stepped into a time-warp; the energy,
exchange of ideas, and excitement that they felt at OLS was reminiscent
of a Usenix conference from the early 1980’s. Two others (both previous
Usenix or Freenix program committee members and/or chairs) agreed with
these sentiments and said that they had pretty much given up any hopes
of “salvaging” Usenix and encouraged me to skip the
href=”http://www.usenix.org/events/usenix05″>2005 Usenix ATC and go
to Linux.conf.au instead. Yet
another old-timer stated that Usenix ’04 was a great place to meet old
friends, but OLS and LCA is where the action is at.
That’s one possible solution, I suppose — after all, if all I care
about is Linux and Open Source Software (OSS) development, OLS (and to a
lesser extent, the [Linux
Kongress]2 and linux.conf.au) is the primiere place to submit papers
and meet with colleagues doing the leading-edge development work in
Linux, X Windows, and other OSS packages.
Unfortunately, there are a number of reasons why this might not be a
satisfactory or complete answer:
-
As others such as Bryan Cantrill have pointed out in his
href=”http://blogs.sun.com/roller/comments/bmc/Weblog/whither_usenix”>web
log, there is a need for a place for industry work to be published,
and increasingly Usenix is not fulfilling this role. So
this is a problem that goes beyond that of just Linux and the OSS
world.
-
Even if we ignore the problems of the proprietary Unix
vendors, the Linux and OSS community needs to exchange ideas with other
industry practitioners. This kind of discussion and cross-breeding
is in the long-term critically important. And even if AIX, HPUX,
and Solaris end up going the way of the dodo, who knows what insights
and new ideas might come from EMC, or Google, or even Microsoft?
-
I (perhaps stupidly ) agreed to serve on the board of Usenix, so I
have a duty to Usenix’s long-term health as an organization and to its
mission. So I can’t just abandon Usenix’s ATC as a hopeless cause,
as some have urged me to do. As a result, some have accused me as having
a too-finely honed
sense of duty….
So, what can be done? I spent
quite a few hours brain-storming with Val Henson, Paul McKenney, and a
few other interested parties during the course of OLS. Val recorded the
results of some of these brain-storms in her
href=”http://blogs.sun.com/roller/page/val/20040722#the_freenix_track_has_it”>web
log. As we had discussed it, the “Product Track” (which I jokingly called
“Realnix”) would contain technologies that either are in a shipping
product, or is intended to be integrated into a shipping product, where
for the purposes of this criteria, the mainline Linux, NetBSD, GNOME,
KDE, X Windows systems, would be considered “shipping product”.
Some might argue that this separation of “academic papers” and
“industry papers” might not be such a great idea. The argument has been
made that the academic track might be viewed as a “Golgafrincham B ark” where papers
with extensive bibliographies, meticulously enumerated related work
sections, and results describing 3% improvements on microbenchmarks
would be relegated. Perhaps such a perception is unfair, but whether or
not Rob Pike’s description of
href=”http://www.cs.bell-labs.com/who/rob/utah2000.pdf”>Systems Software
Research as Being Irrelevant is accurate is a topic for another day
and time. If Rob’s observations have any validity, however, then it is
up to those in the academic world to address it, and the health and
vitality of the “academic track” will ultimately be their
responsibility, for good or for ill.
If we don’t create separate tracks for academic and product papers,
it’s going to be important to make sure the program committees are
balanced with representatives not just from universities and research
labs, but also from product groups. Bryan Cantrill has put together
href=”http://blogs.sun.com/roller/page/bmc/20040708#check_this_out_a_img”>some
data showing how poorly the computer industry in general, and
product groups specifically have been representated on the Usenix
program committees. My concern is that the academic and
product-group communities, as well as their standards for papers, have
diverged too much already, and trying to manage this tension may prove
to be too much for many program committees. This may result in a
lurching back and forth of the acceptance criteria and overall balance of
the Usenix ATC, which would not be a good thing. Consistency
and predictability is extremely important to potential authors who are
deciding whether or not to submit a paper proposal to a conference or
track.
From a practical perspective, given how strong the OLS conference has
been this year, it will be harder and harder for Freenix to attract
papers about Linux and Open Source Software; OLS has proven itself as
the place for the Linux/OSS community to meet, and I predict that this
will if not starve Freenix submissions, cause it to be extremely
lopsided with very few relevant and interesting Linux papers. Hence,
expanding the scope of Freenix to include product papers may make a lot
of sense.
Of course, making room for papers from product groups (no matter how
we do it) isn’t going to be enough by itself. We will also need to find
ambassadors who work at each of various companies in the computer
industry to market to engineers and to managers why submitting papers
and serving on program committees is a good and useful thing to do —
both for the potential paper author as well as their employer. Val
Henson has put together a set of
href=”http://blogs.sun.com/roller/resources/val/slides.pdf”>slides
for doing just that; see her
href=”http://blogs.sun.com/roller/page/val/20040713#program_committees_superpages_and_compare1″>web
log entry for more discussion and the LaTeX sources to the
slides. (She has given permission for others to adapt her slides
for their company; you should credit her if you use them, though!)
Is this going to be enough to keep Usenix relevant and
interesting? To be honest, I don’t know. I sure hope
so. The good news is that there are more and more people thinking
about the problem, as some of the links to various web logs have
demonstrated. So even if we don’t have the complete solution to
the problem right now, we do have a growing consensus that there is a
problem, and what the scope of the problem is likely to be, and that’s
always a better place to be than being unwilling to acknowledge that
there is a problem in the first place.
Update (7/27): I’ve since found the following
[
blog entry]3 which presents
an opposite, or at least slightly different, point of view.
In it, Werner Vogel argues that the problem is that
program committee members have tried, but failed to get industry (product group)
participation — either on program committees or via submissions. I think he makes some
good points, although I think I differ about which is the chicken and
which is the egg here. It may be that having a separate track just for industry/product-group
representatives may make it easier to both recruit program committee members, and
to have enough people to actively solicit and arm-twist paper submissions. Also, perhaps
a program committee will have to use more creative ways of soliciting papers that
might not occur to a group of academics — for example, such as going through
the product marketing group to arm-twist the engineers to write and submit real papers (not just white papers!)
describing the sexy new technology in a recently-released product. Anyway, Werner’s comments are thoughtful
and should be carefully considered.