Theory of Computing Blog Aggregator

And More Fun News....

2010-03-13T21:59:00Z

Right after Stuart Shieber sent me news on CS undergrad earnings, Harry Lewis sent me two additional links. The first is a nice Boston Globe piece on the increase on undergraduate majors in the sciences at Harvard.

The second is much more amusing. Apparently, the Harvard Robobees project has made #1 (that's right, we're number 1!) on Sean Hannity's list of the 102 worst ways the government is spending your tax dollars. Now, I try to stay apolitical on this blog, but I have to say, I'm impressed by Sean Hannity's lack (well, actually, more like a complete absence) of acumen in understanding the nature of scientific research. A look at the Robobees home page, I would think, would certainly suggest that there's important scientific and engineering questions underlying the long-term challenge of building a robotic bee. Of course, maybe the Hannity camp just objects to the government spending money on science generally, I don't know. I'll go on record as suggesting that nobody from the Hannity camp bothered to look at the Robobee home page.

Subscribe to comments | Post a comment

Good News for Computer Science Majors

2010-03-13T20:37:00Z

Undergraduate computer science majors, according to the National Association of Colleges and Employers, are still getting paid well. If you don't want to be an engineer for an oil or chemical products company, we're still apparently the best way to go. (I'd like to think careers in CS are more interesting than in these areas as well, but I really don't have the experience to say.)

Thanks to Stuart Shieber for the link.

Subscribe to comments | Post a comment

Non-archival conferences?

2010-03-13T19:50:37Z

A comment on my recent post mentions that ”COMSOC is non-archival so it should not have an impact on the number of submissions to EC”. Frankly, I wasn’t really aware of this fact (despite being on the COMSOC PC), but indeed looking at the COMSOC website I see that:

Accepted papers will be collected in informal workshop notes, printed copies of which will be available at the workshop. To accomodate the publishing needs of different scientific communities, we stress that authors will retain the copyright of their papers and that submitting to COMSOC-2010 does not preclude publication of the same material in a journal or in archival conference proceedings.

Submission and reviewing standards seem like those of usual “archival conferences” except that “recently published” submissions are OK too:

Submissions of papers describing original or recently published work on all aspects of computational social choice are invited….. All submitted papers will be reviewed by the program committee.

I wonder what does this really mean. The issue of copyright is pretty clear but seems scientifically irrelevant, especially since one would hope that most papers will be available freely on the web (preferably on arXiv and findable from the COMSOC website). The fact that written proceedings will not be made available using “normal” print venues also seems clear but, again, who cares? Many “archival conferences” don’t have printed proceedings either, mostly since these seem pretty useless given the web.

The fact that papers appearing in COMSOC supposedly can be published in economics journals unlike those in “archival conferences” is pure “voodoo”: you change a few scientifically irrelevant technicalities (like copyright) and lo and behold your new conference paper suddenly becomes un-published and acceptable to journals. In fact, EC takes this to a logical conclusion, allowing the authors to chose the archival/non-archival tag of their paper:

To accommodate the publishing traditions of different fields, authors may instead submit working papers that are under review or nearly ready for journal review. These submissions will be subject to review and considered for presentation at the conference but only a one page abstract will appear in the proceedings with a URL that points to the full paper and that will be reliable for at least two years.

One may ask whether a paper is “really” published if it appeared in a non-archival conference. My point is that the meaning of “really” is not very real. There is the question of to what extent was the paper evaluated and refereed, but this does not seem related to the “archival” nature of the conference. As an example, EC papers are judges in the same way whether or not they choose the non-archival track; I also doubt that the “non-archival” status of COMSOC will have much bearing on how the PC evaluates papers. There is the question of whether one writes it on their CV — but different people may do different things, and even the same person has different versions of their CV according to the standards of those that request it.

Finally, there is the all important question how much weight and prestige is attached to the conference publication, in hiring, promotion, or grant decisions (as is uniquely done in CS). This again widely differs between institutions, as well as the issue in question. For example, from my limited experience, more weight is given to conference publications in hiring decisions than in tenure decisions, and more weight is give to conference publications by top ranked departments than by lower ranked ones. Is this prestige a function of the “archival” status of the conference? I doubt it. The bottom line is that at first approximation any conference will gain or lose prestige according to whether appearing in it is indeed an indication of scientific quality. In the case of COMSOC, as for any other young conference, this still remains to be seen.

So, what’s my bottom line: I like the notion of a “non-archival” competitive conference that does not expect exclusivity from submitted papers. I certainly see no reason to “respect” such conference publications less than “archival” ones — respect should just be a function of quality. Maybe having high-quality highly-competitive “non-archival” conferences can get the best of both worlds in the CS journal-conference debate? (See also my view.)

NRC/CRA/ISI ranking trouble

2010-03-13T19:27:56Z

Much of U.S. academia has been anxiously awaiting the overdue National Research Council's rankings of academic programs, but I've been hearing rumors of a brewing fight concerning those rankings between the NRC and the Computing Research Association. The specific issue is how to measure the research productivity of computer science faculty: the story goes that the NRC will use only ISI-listed journal publications for this measurement, despite very clear statements from major professional bodies that ISI data should not be used in computer science due to its omission of most of the leading publication venues. Due to this omission, it seems likely that rankings based on ISI data would lead to the false appearance of lower productivity in CS than in other subjects, and to distortions when researchers whose specialties happen to fall into ISI-listed publications are inaccurately shown as being highly productive.

As far as I can tell, the options seem to be:

NRC backs down and uses some other source of publication data: very unlikely.

NRC doesn't rank computer science programs at all: possible.

NRC doesn't include research productivity measures when ranking computer science programs: also possible.

NRC continues to use ISI data and releases an inaccurate set of rankings, leading to a much more public fight with CRA and other computer science research bodies, as well as to fights within the administrations of a lot of universities over the interpretation of the rankings: most likely.

It's our own damn fault for publishing in conferences instead of journals and we should conform to the rest of academia or suffer the consequences: maybe, but the NRC rankings should be descriptive not prescriptive.

None of this is true and I shouldn't listen to unfounded rumors: also possible.

Is anyone else in possession of more hard information concerning this issue and willing to share?

Breaking Provably Secure Systems

2010-03-13T15:08:13Z

Provably secure crypto-systems can and have been broken

Neal Koblitz is a strong number theorist who is famous, among other things, for inventing elliptic curve cryptography. This was also independently invented by Victor Miller at the same time—1985. Neal, during his career, has also worked on critical reviews of applications of mathematics to other areas. Mathematics is a perfect fit for the physical sciences, but is less clear how it fits with the social sciences. Neal was, for instance, involved in a potential misuse of mathematical methods to social science—see my discussion earlier about Serge Lang and his book “The File.”

Today I plan on talking about a recent article he wrote with Alfred Menezes for the AMS Notices entitled:

The Brave New World of Bodacious Assumptions in Cryptography.

Not the usual math title—I have never seen a title like this in an AMS publication, or in any other publication. According to Wikipedia bodacious means:

Remarkable, courageous, audacious, spirited;
In CB radio jargon, a general-purpose word of praise;
A variety of iris (plant);
Voluptuous, attractive, “hot,” appealing to the eye;
Bo-Day-Shus!!!, an album by Mojo Nixon and Skid Roper.

Neal has been critical of theory’s approach to cryptography, and has written some very critical pieces. I do not agree with him, but I do think he raises some important issues.

Early Days Of Cryptography

I have two background stories I wish to share—hopefully they will help explain my views on security, and why I think Neal has raised some interesting issues. It will also show how stupid I was, but I will get to this in a moment.

Cheating At Mental Poker: Adi Shamir, Ronald Rivest and Leonard Adleman (SRA) in 1979 wrote a paper on how to play “mental poker.” They described a protocol for playing two-person poker over a communication channel, without using any trusted party. One player acted as the dealer and handed out the cards, and both then played poker.

When I got the actual paper—this is way before Internet and pdf files—I immediately realized there was a problem with their protocol. Their method used a family of maps

where is a large prime and was odd. The key to their method was the simple observation:

I sent Rivest a letter outlining an attack based on getting at least one bit of information: I used the observation that their maps preserved whether a number is a quadratic residue or a non residue. This allows the player who is “dealing” the cards to cheat and control who gets, for example, the aces.

Ron replied quickly, by return mail, with a fix: this stopped my attack, the dealer no longer could decide who got the aces. However, it still had a problem, the non-dealer could still see a few bits of information about the dealers hand. This is plenty of information to win at poker—in the long run. I should have challenged Ron to actually play for money, but instead wrote a paper on how to cheat at mental poker. The paper was not widely read, even though it was published as part of an AMS workshop. At least one paper cites me as “Ron Lipton”—oh well.

Shortly after this Shafi Goldwasser and Silvio Micali wrote a beautiful paper on how to use probabilistic methods to construct a protocol that avoided the problems I had pointed out. I was stupid, since I did not try to find such a provable protocol. At the time, I was working on complexity theory, and did not follow up on my crypto ideas. Too bad.

A few years ago, at a very pleasant lunch in San Francisco with Dick Karp, Shafi and Silvio pointed out to Dick how much they were influenced by my cheating paper. Even though I usually get little—no credit?—for my work, they were quite adamant how important it was to their thinking about semantic security. I thanked them.

In the SRA paper, they did not realize their protocol leaked at least one bit—more than enough to win at poker. This simple example, in the hands of Shafi and Silvio, led eventually to important new ideas in cryptography. The second story is also about how hard it is define what is security.

Proving An OS Kernel is Secure:

In the summer of 1976, Anita Jones and I were part of a RAND summer study group organized by Stockton Gaines on security. Anita and I eventually wrote a fun paper during the summer: a paper with a long, complex, and crazy story; I will share it with you another day.

During the workshop, Gerry Popek visited us one day, and gave a talk on his then current project to prove a certain OS kernel was secure. This was a large project, with lots of funding, lots of programmers, and he said they hoped in two years to have a proof of the OS’s correctness. What struck me during his talk was he could write down on the board, a blackboard, a relatively simple formula from set theory. The formula, he stated, captured the notion of data security: if a certain function had this property, then he would be able to assert his OS could not leak any information. Very neat.

At the end of his talk I asked him if he wanted a proof now that his function satisfied the formula. He looked at me puzzled, as did everyone else. He pointed out his was defined by his OS, so how could I possibly prove it satisfied his formula—the was thousands of lines of code. He added they were working hard on proving this formula, and hoped to have a full proof in the next 24 months.

I asked again would he like a proof today? Eventually, I explained: the formula he claimed captured the notion of security was a theorem of set theory—any function had the property. Any. He said this was impossible, since his formula meant his system had no leaks. I walked to the board and wrote out a short set theory proof to back up my claim—any had his property. The proof was not trivial, but was not hard either.

The point of the story is: the formula captured nothing at all about his OS. It was a tautology. What surprised me was his response: I thought he would be shocked. I thought he might be upset, or even embarrassed his formula was meaningless. He was not at all. Gerry just said they would have to find another formula to prove. Oh well.

Provable Security

I hope I do not lose my theory membership card, but I think Neal has made some serious points in his recent paper. But, first let’s look at the foundations of modern cryptography. This is based on turning “lemons” into “lemonade”: turning hardness of certain computations into security of certain crypto-protocols. You probably all know the framework, but here is my view of it:

Let be a protocol between Alice and Bob. This is nothing more than a fancy way of saying Alice and Bob have an algorithm—almost always random—they plan to execute together. The goal of the algorithm is to send a message, to share a secret, to agree on a value, to do a computation together, and in general to do some interesting information processing. Of course, Alice and Bob are using the protocol to avoid others from learning all or even parts of their information.
Let be some computational problem we believe is “hard.” The assumption that requires a great deal of computation can vary greatly: it can mean worst case cost, average cost, and can, today, mean the cost on a quantum computer.
Let be the class of “attacks” allowed against the protocol. Usually these have to be explicitly stated and carefully defined. Sometimes, we allow simple attacks, sometimes we allow more powerful attacks. For example, allowing the attacker to influence Alice or Bob is a much more powerful attack, than an attacker who passively watches their protocol.
Finally, the proof of security is a mathematical theorem of the form:

Theorem: Any efficient attack of type against the protocol , implies the problem is not hard.

Of course, such theorems would have more quantitative bounds, but this is their general form.

Provable Security?

The framework is very pretty, but can fail in multiple ways—as Neal points out in his papers. Let’s take a look at what can go wrong. It is convenient to do this in reverse order of the parts of the framework: the theorem first, then the attacks, then the hardness assumption, and finally the protocol.

The Theorem: Any theorem, especially a complex theorem, can have an incorrect proof. I have talked about this is several previous discussions—see here. Even great mathematicians make mistakes, so it should come as no surprise if cryptographers make mistakes.

There are two reasons errors here are perhaps more likely than in some other areas of mathematics. First, there may be much less intuition about the behavior of a complex protocol. No matter how precise you are in your reasoning in any proof, intuition is a great guide. Sometimes, I have thought I had proved something, but it did not fit with my intuition. Usually, this led, after more careful thought, to the discovery of an error in the proof.

Second, as Rich DeMillo, Alan Perlis, and I have pointed out in our famous—infamous?—paper on social processes, the social network of mathematicians is a requirement for confidence in proofs. Without others checking your theorem, teaching it in their class, using it in their own work, finding new proofs of your theorem, we would be flooded with errors. The trouble with crypto theorems is they do not always have this social network to back them up. Many crypto papers prove theorems used by others, but many do not.

The Attacks: This is in, in my opinion, the main issue: do the attacks include all the possible attacks? My examples on mental poker and OS kernels show two simple examples where attacks were not properly defined. There are many attacks on crypto protocols that were not envisioned initially. These include: timing attacks, fault based attacks, composition of protocols, and many others.

The Hardness Assumption: There are hardness assumptions, and there are hardness assumptions. Clearly, if the problem is not as hard as you thought, then even if the theorem and attacks are correct, the theorem is meaningless.

The failure of a hardness assumption to really be hard has happened over and over in modern cryptography. Famous examples include:

The assumption about hardness of knapsack problems;
The assumption in the original RSA paper on the cost of factoring;
The assumptions about solving certain Diophantine equations—especially the work of John Pollard and Claus Schnorr on solving binary quadratic forms, and Don Coppersmith on low exponent RSA.

The Protocol: Finally, the protocol itself is a potential failure point. If the protocol is implemented incorrectly, then all the other parts of the framework are useless.

Neal’s Main Attack

Neal’s main attack is on a paper written by of one of own faculty at Georgia Tech: Sasha Boldyreva. She with Craig Gentry, Adam O’Neil, and Dae Hyun Yum wrote a paper on a certain crypto protocol in 2007. In order to prove their protocol was secure they needed to assume a certain technical problem was intractable. They argued their assumption was reasonable: they even could prove it was hard provided the groups involved were generic.

I will not explain exactly what this means, but in my view it is equivalent to really restricting the attacks allowed. Roughly, if a group is generic, then there are very limited operations an attacker can perform on the group. Neal quotes them as saying: This has become a standard way of building confidence in the hardness of computational problems in groups equipped with bilinear maps.

The problem is the attacker can use “burning arrows:” the attacker can use the structure of the group in question, and is not limited to treat it as a generic group. The attacker can do what ever they want, as long as the computations they perform are efficient. This is exactly what happened two years later: Jung Hwang, Dong Lee, and Moti Yung showed Sasha’s protocol could be easily broken. Neal—to hammer home his point—gives the attack in full, since it is so simple. Not simple to find, but simple since it involves only solving a few equations over the group.

The lesson here is a valid one I believe: we must be very careful in making assumptions about what an attacker can do. An equivalent way to look at this situation is: the hardness assumption was not true, if sufficiently general attacks against it are allowed. Neal then adds a comment:

The 4-page proof is presented in a style that is distressingly common in the provable security literature, with cumbersome notation and turgid formalism

This is unfair—I always have felt whether a proof is clear is a subjective statement. But, there is a lesson here. Just as SRA did not allow a simple attack that defeated their mental poker protocol, Sasha and her colleagues did not allow non-generic attacks against her protocol.

Open Problems

Rich DeMillo, Alan Perlis, and I, in our social process paper, were most concerned with implementations of complex software. The verification community idea, at the time, was to use mathematics to proof correctness of such systems.

But, perhaps one way to summarize is this: the whole framework of crypto-provable systems needs the same social processes we outline in our old paper. The exact specification of attacks, and the proof of the security of a protocol are just as vulnerable to problems as the implementation of any large software.

An open problem is to see more proofs of the completeness of attacks. I am unsure how we can do this, but I think this would be very confidence building for the field.

Choosing the number of clusters II: Diminishing Returns and the ROC curve

2010-03-13T08:57:00Z

(this is part of an occasional series of essays on clustering: for all posts in this topic, click here)

In the last post, we looked at the elbow method for determining the "right" number of clusters in data. Today, we'll look at generalizations of the elbow method, all still based on the idea of examining the quality-compression tradeoff curve.

For the purpose of discussion, let's imagine that we have a plot in which the quality of the clustering (measured anyway you please, as long as 0 means all items in one cluster, and 1 means all items in separate clusters) is measured along the y-axis, and the representation complexity (i.e the space needed) is measured on the x axis, where again 0 corresponds to all items in a single cluster (least representation cost), and 1 corresponds to all items in separate clusters.

The ideal cluster is located at (0,1): cheapest representation and perfect quality. In general, as we use more and more clusters to organize the data, we can trace out a curve that starts at (0,0) and ends at (1,1). For most sane functions, this cuve is concave, and lives above the diagonal x=y.

This curve contains lots of useful information about the behavior of the data as the clustering evolves. It's often called the ROC curve, in reference to the curve used to capture the tradeoff between false positive and false negatives in classification. But what can we glean from it ?

We can ask what the curve would look like for "unclusterable" data, or data that has no definite sense of the "right number" of clusters. Such data would look self-similar: you could keep zooming in and not find any clear groupings that stood out. It's not too hard to see that data that looked like this would have a ROC curve that hugs the diagonal x=y, because there's a relatively smooth tradeoff between the quality and compressibility (so no elbow).

Conversely, data that does appear to have a definite set of clusters would try to get closer to the (0,1) point before veering off towards (1,1). This suggests a number of criteria, each trying to quantify this deviation from unclusterability.

you could measure the area between the curve and the diagonal. This is (essentially) the AUC (area-under-curve) measure.
You could measure the closest distance between (0,1) and the curve. This also gives you a specific point on the curve, which you could then associate with the "best" clustering.
You could also find the point of diminishing returns - the point of slope 1, where the clustering starts costing more to write down, but yields less quality. I've used this as the start point of a more involved procedure for finding a good clustering - more on that later.

The ROC curve gives a slightly more general perspective on the elbow method. It's still fairly heuristicy, but at least you're trying to quantify the transition from bad clustering to good in somewhat more general terms.

Ultimately, what both the ROC curve and the elbow method itself are trying to find is some kind of crucial transition (a phase transition almost) where the data "collapses" into its right state. If this sounds like simulated annealing and bifurcation theory to you, you're on the right track. In the next part, I'll talk about annealing and phase-transition-based ideas for finding the right clustering - all fairly ingenious in their own right.

(ed. note: comments on the last post have convinced me to attempt a post on the nonparametric approaches to clustering, which all appear to involve eating ethnic food. Stay tuned...)

Subscribe to comments | Post a comment

The Impagliazzo Hard-Core Lemma for the Mathematician

2010-03-13T00:23:49Z

At the conference I was at last week, I was asked by a mathematician about the exact statement of the Impagliazzo Hard-Core Lemma, and whether I could state it without saying “algorithm.”

So I gave the following statement:

Theorem 1 (Impagliazzo Hard-Core Lemma — Basic Version) Let be a class of boolean functions , let be an arbitrary function, and let 0}" class="latex" src="http://l.wordpress.com/latex.php?latex=%7B%5Cepsilon%2C%5Cdelta%3E0%7D&bg=ffffff&fg=000000&s=0" title="{\epsilon,\delta>0}"/> be arbitrary parameters.

Then at least one of the following conditions is true:

is globally well approximated by a simple composition of a few functions from .
That is, there are functions , and a “simple” function such that

is locally almost orthogonal to .
That is there is a subset of density at least such that for every ,

The theorem has been strengthened in a couple of ways in work of Klivans and Servedio and Holenstein. The “complexity” parameter needs to be only which is conjectured (or known?) to be tight, and the density of the set can be made , which is tight. In all proofs is really simple, being more or less a threshold function applied to the sum of the .

Stating the result in this way (which is, of course, standard) raises a few questions that I find rather interesting.

Lower Bounds on ?

I don’t know if a lower bound on is known, but I think it’s helpful to see why the statement would be false if we tried to approximate with one function from in the first case.

Take to be a random function, pick an arbitrary partition of into three subsets , and define such that on and on . Pick . Now we see that there is no that agrees with on a fraction of inputs. On the other hand, no matter how we pick a subset of density of , there is at least a function that agrees with on on at least a fraction of inputs. (Note that the sets are a partition of , and at least one of the three must have size less than .)

If I am not mistaken, this argument yields a lower bound for constant . How does one argue an lower bound?

From Boolean Functions to Bounded Functions

When one abstracts complexity theory results as we have just done, as results about classes of boolean functions, usually the same proofs also give more general results about bounded functions, which correspond to probabilistic algorithms.

A scaling argument suggests that the analog of the Lemma for bounded functions would be that for every set of of functions of , every function , and every parameters 0}" class="latex" src="http://l.wordpress.com/latex.php?latex=%7B%5Cepsilon%2C+%5Cdelta+%3E0%7D&bg=ffffff&fg=000000&s=0" title="{\epsilon, \delta >0}"/>, at least one of the following holds:

there are functions and a combining function such that
there is a subset of density at least such that for every ,

But this turns out to be false. Take to be a random function and take to be any class of functions and to be parameters such that (1) is not true even for large (for example, take to be class of functions computable by size circuits and to be inverse polynomial in ); now note that taking has inner product at least with the constant-one function on every subset .

Is there a nice way to generalize the Hard-Core Lemma to bounded functions? (Partial answer in the next section.)

Explicit

When formulating complexity-theoretic results in this abstract form, I often find it useful to see what happens if I instantiate the class of functions that are meant to represent efficient algorithms with the class of Fourier characters of . Typically, one obtains a basic fact from Fourier analysis, but with an unusual proof that makes use of no properties of the characters other than they are a family of functions, which can be nice to see. Sometimes, one obtains a statement that is not trivial, and presenting the proof in this special case can be a nice way to introduce a complexity-theoretic result to mathematicians.

Here, if you try this experiment, you get something that is not standard, but also not very meaningful: that a boolean functions is either well approximating by a function of a few characters (that is, a function that is constant on the cosets of a rather large subspace), or else there is a large subset on which the function has very low correlation with all characters. The first case is an interesting one, but the second one sounds strange, because we don’t know anything about the set.

It would be nicer if we could say: either is well approximated by a function of few characters, or there is a subspace such that is nearly orthogonal to all characters when restricted to . This would correspond to a version of the Hard-Core Lemma in which the characteristic function of the set is itself a simple combinations of functions from . Unfortunately, this stronger statement is false in general, and it is even false in the special case of the Fourier analysis application: take to be 1 with probability and with probability , and you see that it is not approximable, but also that the constant-one character has large correlation in all large subspaces.

Maybe we could say that either is globally approximable, or else there is a large subspace such that restricted to has low correlation with all non-constant characters.

In fact, I think that the following statement is true via the same density-increment argument we have in this paper with Reingold, Tulsiani and Vadhan, where we prove a slightly different version:

Theorem 2 (Impagliazzo Hard-Core Lemma — Constructive ) Let be a class of boolean functions , let be an arbitrary function, and let 0}" class="latex" src="http://l.wordpress.com/latex.php?latex=%7B%5Cepsilon%2C%5Cdelta%3E0%7D&bg=ffffff&fg=000000&s=0" title="{\epsilon,\delta>0}"/> be arbitrary parameters.

Then at least one of the following conditions is true:

is globally well approximated by a simple composition of a few functions from .
That is, there are functions , and a function such that

is locally almost orthogonal to after a shift.
That is there is a subset of density at least , such that for , , and for function , such that for every ,

The only difference between the above statement and the one in the RTTV paper, Lemma 3.1 is that there Case (2) has a weaker form equivalent to

but, unless I made a mistake in the calculations I just did, the same proof works for the Case (2) in the theorem stated above.

In fact I think that this statement would also work for general bounded functions, reformulating Case (1) to require distance at most instead of agreement at least .

While is polynomial in and , the function might have exponential complexity in in the only proof I know of of Theorem 2, so the results discussed in this section have limited applicability in complexity theory. Is it possible to prove a constructive hard-core lemma in which the combining function has polynomial circuit complexity in and ?

Holenstein proves a constructive hard-core lemma in which is samplable if the distribution is samplable, and in a paper with Tulsiani and Vadhan we show that the characteristic function of can be made efficiently computable given oracle access to , but the question I would like to raise is whether, as in the above statement, one can have fully polynomially computable for an arbitrary to which we have no access of any kind, but with the “shifted inner product” condition in Case (2).

Ad Auctions Workshop 2010

2010-03-12T23:37:00Z

The 6th Ad Auctions Workshop will take place on June 8th, collocated with the EC Conference. The call for participation is now available. Deadline April 14. CFP says "The workshop will bring together researchers and practitioners from academia and industry to discuss the latest developments in advertisement auctions and exchanges." I love how ad exchanges are now first class objects.

The organizing committee comprises: Moshe Babaioff, Ben Edelman, Jon Feldman, Sebastien Lahaie and Kamesh Munagala.

As Jon says, "Historically this is always a great workshop, more focused than the typical academic conference. It brings together folks from both industry and academia, and lots of different areas (algorithms, economics, optimization, machine learning)." And as CFP says, "The workshop’s proceedings can be considered non-archival, meaning contributors are free to publish their results later in archival journals or conferences." In other words, it is a true workshop.

Subscribe to comments | Post a comment

Privacy strikes (again!)

2010-03-12T22:44:00Z

Events unfold thus:

Netflix releases data for a stunning contest that captures the researchers' imagination.
Researchers correlate the data with publically available data to break anonymity. Netflix is sued.
Legalese, yaada, yaada, yaada ... Netflix cancels the version 2 of the contest.

Subscribe to comments | Post a comment

TR10-044 | Affine Dispersers from Subspace Polynomials | Eli Ben-Sasson, Swastik Kopparty

2010-03-12T14:39:36Z

{\em Dispersers} and {\em extractors} for affine sources of dimension $d$ in $\mathbb F_p^n$ --- where $\mathbb F_p$ denotes the finite field of prime size $p$ --- are functions $f: \mathbb F_p^n \rightarrow \mathbb F_p$ that behave pseudorandomly when their domain is restricted to any particular affine space $S \subseteq \F_p^n$ of dimension at least $d$. For dispersers, ``pseudorandom behavior'' means that $f$ is nonconstant over $S$, i.e., $\left|\{f(s) \mid s\in S\}\right|>1$. For extractors, it means that $f(s)$ is distributed almost uniformly over $\mathbb F_p$ when $s$ is distributed uniformly over $S$. Dispersers and extractors for affine sources have been considered in the context of deterministic extraction of randomness from structured sources of imperfect randomness. Previously, explicit constructions of affine dispersers were known for every $d = \Omega(n)$, due to Barak, Kindler, Shaltiel, Sudakov and Wigderson (2005) and explicit affine extractors for the same dimension were obtained by Bourgain (2007). The main result of this paper is an efficient deterministic construction of affine dispersers for {\em sublinear} dimension $d = \Omega(n^{4/5})$. Additional results include a new and simple affine extractor for dimension $d>2n/5$, and a simple disperser for multiple independent affine sources. The main novelty in this paper lies in the method of proof, which makes use of classical algebraic objects called {\em subspace polynomials}. In contrast, the papers mentioned above relied on the sum-product theorem for finite fields and other recent results from additive combinatorics.

TR10-043 | Interval Graphs: Canonical Representation in Logspace | Johannes Köbler, Sebastian Kuhnert, Bastian Laubner, Oleg Verbitsky

2010-03-12T11:11:49Z

We present a logspace algorithm for computing a canonical interval representation and a canonical labeling of interval graphs. As a consequence, the isomorphism and automorphism problems for interval graphs are solvable in logspace.

Six-to-one perspective

2010-03-12T05:16:27Z

The drawing below is in "six-to-one perspective": three-dimensional axis-parallel rays in each of the six possible directions are drawn as circular arcs that all converge to the same one point. Axis-parallel line segments become circular arcs, right angles in space become 60 or 120-degree angles in the plane, and (in exchange for some fisheye distortion) the whole 360-degree field of view fits into a plane. If a corner polyhedron (such as a cube, below) is drawn using this projection, we can see all of its vertices, even the hidden back vertex.

By comparison, axonometric projection uses nice straight lines and has the same 60- or 120- degree angles, but can't show the points of convergence.

Three-point perspective again has nice straight lines, and (when used less excessively than the view below) looks the most realistic of any of these methods, but has nonuniform angles and can only show three of the six convergence points.

And six-point perspective uses circular arcs and represents all six convergence points, but again has nonuniform angles.

In choosing the radii for the guide circles in the six-to-one perspective drawing, it turned out to work well to pick smooth numbers: the radii are in the proportion 3:4:6:8:12, leading to lots of triple junctions on which the vertices of the cube can be placed. Maybe for a more complex drawing I'd need to step up to a larger number of prime factors?

If anyone knows any references on this type of perspective, I'd appreciate hearing about them.

TR10-042 | Relativized Worlds Without Worst-Case to Average-Case Reductions for NP | Thomas Watson

2010-03-12T00:07:51Z

We prove that relative to an oracle, there is no worst-case to errorless-average-case reduction for $\NP$. This result is the first progress on an open problem posed by Impagliazzo in 1995, namely to construct an oracle relative to which $\NP$ is worst-case hard but errorless-average-case easy. We also handle classes that are somewhat larger than $\NP$. In fact, we prove that relative to an oracle, there is no worst-case to errorless-average-case reduction from $\NP$ to $\BPPpath$. The latter class contains $\p^\NP_\Vert$ and captures the power of randomized computations conditioned on efficiently testable events. We also handle reductions from $\NP$ to the polynomial-time hierarchy and beyond, under restrictions on the number of queries the reductions can make.

Two Announced Breakthroughs

2010-03-11T23:34:43Z

I was at a conference last week, at which I heard two pieces of mathematical gossip.

One was that Arora, Barak and Steurer have developed an algorithm that, given a Unique Games in which a fraction of constraints are satisfiable, finds an assignment satisfying a constant fraction of constraints in time . This is now officially announced in an earlier (public) paper by Arora, Impagliazzo, Matthews and Steurer, which presented a slower algorithm running in time where .

I suppose that the next targets are now the approximation problems for which the only known hardness is via unique games. Is there a subexponential algorithm achieving approximation for sparsest cut or approximation for Vertex Cover?

The other news is on the Polynomial Ruzsa-Freiman conjecture, one of the main open problems in additive combinatorics. Apologies in advance to readers if I get some details wrong. In the special case of , the conjecture is that if is any function such that

then there is a matrix and a vector such that

where the probability in the conclusion is independent of and is polynomial in . Various proofs were known achieving a bound of . The first proof, due to Samorodnitsky achieves, I believe, a bound of , while the results from this paper should imply a bound of where we use the notation .

At the conference, Ben Green announced a result of Schoen implying a subexponential bound of .

The proof begins with the standard step of applying a theorem of Ruzsa to find a subset such that , and on is a “Freiman 16-homomorphism,” meaning that for every 32 elements of such that

we have

The choice of 16 is just whatever makes the rest of the proof work. The theorem of Ruzsa works for any arbitrarily large constant. Then we consider the set of all the elements that can written as with , and we define a function on by setting

which is a well-posed definition because is a Freiman 16-homomorphism. (It would have been sufficient if had been an 8-homomorphism.) Note that for every such that we have

Now the result of Schoen implies that if is a subset of of size , then there is a subspace of dimension such that is entirely contained in .

Previously, it was known how to get a subspace of dimension , by adapting a technique of Chang (see this survey paper).

Note that is now a linear map on , and that it agrees with on . (I cannot reconstruct how this last step follows, however — maybe that claim is that agrees with on a fraction of elements of ? ) It now just remains to extend, arbitrarily, to a linear map over the entire space.

TR10-041 | Improved Algorithms for Unique Games via Divide and Conquer | Sanjeev Arora, Russell Impagliazzo, William Matthews, David Steurer

2010-03-11T20:03:19Z

We present two new approximation algorithms for Unique Games. The first generalizes the results of Arora, Khot, Kolla, Steurer, Tulsiani, and Vishnoi who give polynomial time approximation algorithms for graphs with high conductance. We give a polynomial time algorithm assuming only good local conductance, i.e. high conductance for small subgraphs. The second algorithm runs in mildly exponential time, $e^{\alpha n}$, but makes no assumptions about the underlying constraint graph. As the completeness approaches $1$ (completeness $1-\epsilon$), the constant $\alpha$ in the running time rapidly approaches $0$ ($\alpha = exp(-\Omega(1/\epsilon))$.) The value of the solutions returned by these algorithms depend only on the completeness of the Unique Game and either the local conductance or the allowed running time respectively. In particular, the performance of these algorithms does not depend on the number of labels in the Unique Game. Both algorithms are based on new methods for partitioning graphs by cutting small fractions of edges when the graph can be embedded in a suitable metric space.

Dagstuhl 10101

2010-03-11T18:08:00Z

I am at Dagstuhl seminar 10101 (schedule) on computational social choice; it ends tomorrow. Much of the chatter in the evenings has been about the AAAI conference, due to author feedback to reviews being made available a couple of days ago; to a lesser extent ACM-EC accepted papers, also announced a couple of days ago. I will not try to do a complete overview; let me try a more vignette-like approach.

A nice talk by Jérome Lang was in the context of conducting a collection of yes/no votes where there is preferential dependencies between attributes, meaning that a voter's support for one issue may depend on the outcome of the vote on one or more of the other issues. The example used was voting to build a swimming pool and voting to build a tennis court, where some voters would like one or the other, but not both (too expensive). Each voter is represented by a preference relation on the 4 possible outcomes (for n yes/no issues, he uses a more concise representation, "CB-nets"). The question is, how hard is it for the "chair" (who chooses which order the issues are voted on) to control the outcome. Commonly NP-hard, which is taken to be good news, although it is noted that it raises the question of easier manipulation in typical or average cases. That issue is analogous to the hardness of a voter choosing a strategic vote (a ranking of the candidates that is not his true ranking) so as to get a better outcome. While that is NP-hard for some voting schemes, it may often be easy in practical settings. Indeed, Edith Hemaspaandra's talk was about polynomial-time manipulability when there is single-peaked preferences over the candidates.

The rump session (not covered in the above schedule) was 11 talks each of 5 minutes, really aimed at stating problems where no results have been obtained. I gave an introduction to the "chairman's paradox" (to ask about related computational issues) -- it was identified by Farquharson in 1969 and goes as follows. Say you have a committee of 3 voters {1,2,3} who have to choose one of 3 outcomes {A,B,C}. Let voter 1 be the "chair" and voters 2 and 3 be the ordinary members. The special role of the chair is that if all 3 outcomes get one vote each, then the one supported by the chair is the winner. The paradox is that if the voters' preference lists are generated at random, and your solution concept is pure Nash equilibrium, then the chair gets what he wants less often than the other members. For example, consider the (Condorcet cycle) preferences where 1 has preferences ABC (in descending order), 2 has preferences BCA and 3 has preferences CAB. Then, voter 2 will vote for C since that results in C rather than A winning (2 and 3 supporting C). Voters 1 and 3 continue to support A and C respectively, having no incentive to switch. That is the only pure Nash equilibrium that results from iterative removal of weakly dominated strategies, and notice that 1 (the chair) gets his worst outcome C. I will note that the last time I had to chair a committee of this nature, I felt disadvantaged, although not quite for this reason. The session contained an interesting talk by Kóczy on a method for ranking economics journals (someone has to do it I suppose; see this article on the ranking obsession factor). Finally I should surely mention Felix Brandt's entertaining talk on the "kicker theorem".

Edith Elkind's talk on plurality voting with abstentions was related to the above in using pure Nash equilibrium as the solution concept with a bunch of voters and alternatives (not just 3). An interesting open problem she raised is: Suppose you have sets of voters and alternatives, and for each voter/alternative pair there's a numerical utility of that outcome to that voter. Assume that in the event of a tie, the voter's utility is the average (furthermore, let's assume that all numbers and averages are distinct). Suppose voters cast their votes in some prescribed order, and consider the solution concept of subgame perfect equilibrium. What is the complexity of computing their votes? (On a less formal note, Aviv Zohar told me about a "taxicab game" he had played rather poorly, in which a sequence of (say) 10 people board a minibus with 11 seats, and you aim to end up next to the vacant seat, or failing that, maybe some seats/neighbours are better than others. OK, it needs to be specified more precisely.)

Here is another recent blog post on computational social choice, just to prove I have been paying attention.

Subscribe to comments | Post a comment

Theorems that you simply don't believe

2010-03-11T15:40:00Z

There are some theorems that are surprising. I've already blogged on that (I can't seem to find the link). However, there are some theorems that some people simply do not believe. I mean people who understand the proofs and still don't believe them. Let me give you a contrast- I DO believe that NSPACE(n) is closed under complementation because, while surprising, the proof really does tell you why its true. For the following surprising results the proof does not help. Or at least does not help the people who were surprised by it.

Barrington's theorem. I've read it, talked to Barrington about it, and even taught it. I still don't believe that (say) the set of strings that have the number of 1's equivalent to 0 mod 101 can be done by a width 5 branching program.
Banach Tarski Paradox A CS grad students who knows some math says that it shows that mathematics is broken. I would prefer to say it casts doubt on the axiom of choice.
The classification of finite simple groups. Does any one person even know the proof? Couldn't they have missed some group? Counter argument: the list is on Wikipedia so it has to be correct.
The rationals and naturals are the same size. I know someone who knows the proof and is happy to say they are the same cardinality but refuses to say they are the same size. (I think they are wrong and this is important- using the term size DOES matter.)
A well known theorist told me that he used to believe both P ≠ BPP and there were problems in DTIME(2^O(n)) that require circuits of size 2^Ω(n);. Oh well.
Lance Fortnow tells me he has a hard time believing the Recursion Theorem. Perhaps because the proof is completely uninformative. (Ted Slaman, a well known recursion theorists, agrees that the proof is uninformative. Bob Soare thinks the proof is quite intuitive- a failed diag argument.)
Probability has a few of these: The Central Limit Theorem says that stuff is all normal. That can't be true! I've done the calculations for Birthday Paradox but it still seems suspect to me. And don't get me started on The Monty Hall Paradox.
Local Lovasz Lemma has gone from being something I didn't believe to something I now understand and believe. The original proof just looked like symbols being pushed around, but Moser's and later Moser-Tardos's constructive versions makes sense to me.
We all know that Godel's theorem surprised people- but were there people who did not believe it? This theorem does not surprise Generation Xers who are not at all surprised to find out certain problems cannot be solved. Their response: Whatever.
The existence of Geometries that are as valid as Euclidean but not Euclidean. Again, this surprised people, but were there those who did not believe it? In this age of moral relativism people have no problem with different geometries that are all valid.

How about you? Are there any theorems that you simply don't believe?

Subscribe to comments | Post a comment

Asynchronous Bounded Expected Delay Networks

2010-03-11T00:00:00Z

Authors: Rena Bakhshi, Jörg Endrullis, Wan Fokkink, Jun Pang
Download: PDF
Abstract: The commonly used asynchronous bounded delay (ABD) network models assume a fixed bound on message delay. We propose a probabilistic network model, called asynchronous bounded expected delay (ABE) model. Instead of a strict bound, the ABE model requires only a bound on the expected message delay. While the conditions of ABD networks restrict the set of possible executions, in ABE networks all asynchronous executions are possible, but executions with extremely long delays are less probable. In contrast to ABD networks, ABE networks cannot be synchronised efficiently. At the example of an election algorithm, we show that the minimal assumptions of ABE networks are sufficient for the development of efficient algorithms. For anonymous, unidirectional ABE rings of known size N we devise a probabilistic leader election algorithm having average message and time complexity O(N).

The zero exemplar distance problem

2010-03-11T01:30:14Z

Authors: Minghui Jiang
Download: PDF
Abstract: Blin, Fertin, Sikora, and Vialette recently proved that deciding whether the exemplar distance between two genomes with duplicate genes is zero is NP-hard even if each gene appears at most two times in each genome. We give an alternative proof of this remarkable result.

March 12 center meeting

2010-03-10T22:58:58Z

Mar

The schedule for Friday March 12 center meeting is below. The meeting will be at the usual place (room 402 in Princeton CS).

10:00 – PI meeting

10:45 – Alexandra Kolla: Spectral Algorithms for Unique Games

11:30 – Lunch + talk – Steven Myers : One-bit Encryption is Complete

12:30 – Coffee break

13:30 – David Steurer: An -time algorithm for -satisfiable unique games.

TR10-040 | Relationless completeness and separations | Pavel Hrubes, Avi Wigderson, Amir Yehudayoff

2010-03-10T19:15:11Z

This paper extends Valiant's work on $\vp$ and $\vnp$ to the settings in which variables are not multiplicatively commutative and/or associative. Our main result is a theory of completeness for these algebraic worlds. We define analogs of Valiant's classes $\vp$ and $\vnp$, as well as of the polynomials permanent and determinant, in these worlds. We then prove that even in a completely relationless world which assumes no commutativity nor associativity, permanent remains $\vnp$-complete, and determinant can polynomially simulate any arithmetic formula, just as in the standard commutative, associative world of Valiant. In the absence of associativity, the completeness proof gives rise to the following combinatorial problem: what is the smallest binary tree which contains as minors {\em all} binary trees with $n$ leaves. We give an explicit construction of such a universal tree of polynomial size, a result of possibly independent interest. Given that such non-trivial reductions are possible even without commutativity and associativity, we turn to lower bounds. In the non-associative, commutative world we prove exponential circuit lower bounds on explicit polynomials, separating the non-associative commutative analogs of $\vp$ and $\vnp$. Obtaining such lower bounds and a separation in the complementary associative, non-commutative world has been open for about 30 years.

A Conflict Question

2010-03-10T18:05:00Z

I was recently asked the following question:

Suppose you're the PC chair, and someone who has submitted a paper asks you NOT to have the paper reviewed by a specific person on the PC. Do you honor that request?

It's an interesting question -- that I hope others will comment on -- though as a default my answer would be yes. I certainly have had run-ins of sufficient severity with various people through the years that I would not want (and would likely ask) for them not to review my papers if the issue came up. Looking at it from the other end, if I am on a PC and those people submit a paper, I make sure not to review them. (Usually it is sufficient simply to rank them low on my list of desired papers, but I have also told PC chairs in advance I would not review certain papers if they seemed likely to head my way.) It is not that I actually think I couldn't give a fair review; it's that I think it's inappropriate, in such a situation, for me to give a review in the first place. If as a PC member I have the right (actually, I would say, a responsibility) to refuse to review a paper under such circumstances, it seems fair that a submitter can ask for a specific PC member to not review a paper as well.

Context does matter, though. In the networking conferences I have served on, this is standard -- PC members and submitters are expected to list their conflicts. Indeed, one issue that seems to have arisen lately is that there is suspicion that some people submitting papers are abusing this right, listing people as conflicts when they are not because they are known to be "challenging" reviewers. While I'm skeptical this sort of gamesmanship gains anything (challenging reviewers are usually calibrated appropriately at the PC meeting), it is a concern that once you open the door to such requests, you may need to make sure the privilege isn't abused.

For theory conferences, where many people seem painfully unclear on what "conflict of interest" even means, I'd grant such a request as a matter of course.

Subscribe to comments | Post a comment

Turing Award and Waterman Award and the variety of our field

2010-03-10T17:01:00Z

As Lance tweeted:

The Turing Award for 2009 was given recently to Chuck Thacker LINK. See here. He developed the first modern PC.
The Alan T. Waterman award was given to Subhash Khot. See here. He formulated the Unique Game Conjecture and has proven many consequences of it.

These two award recipients demonstrate the vast variety there is within computer science. I suspect that these two people, one very practical, one very theoretical, have very different mindsets. The most striking is that in theory we have PROOF as our... proof that something is true (I can't even escape using the word!). In practical things the proof is in the pudding.

There is much less variety within Mathematics. All (well... most) mathematicians have proof as their criteria of truth. They may not understand each others problems and interests but they understand the type of problems each other works on.
Physics has two campus- theorists and experimentalists. But I get the impression they talk to each other and understand each other. While this is true in some parts of computer science (crypto and bio-comp come to mind) it is also often not true. (If I am wrong about Physicists let me know.)

Consider the following statements, both probably exaggerated.

In a math department any professor can teach any undergraduate class.
In a computer science department it is NOT the case that every professor could PASS every undergraduate class.

Subscribe to comments | Post a comment

TR10-039 | On the degree of symmetric functions on the Boolean cube | Gil Cohen, Amir Shpilka

2010-03-10T11:04:21Z

In this paper we study the degree of non-constant symmetric functions $f:\{0,1\}^n \to \{0,1,\ldots,c\}$, where $c\in \mathbb{N}$, when represented as polynomials over the real numbers. We show that as long as $c < n$ it holds that deg$(f)=\Omega(n)$. As we can have deg$(f)=1$ when $c=n$, our result shows a surprising threshold phenomenon. The question of lower bounding the degree of symmetric functions on the Boolean cube was previously studied by von zur Gathen and Roche who showed the lower bound deg$(f)\geq \frac{n+1}{c+1}$ and so our result greatly improves this bound. When $c=1$, namely the function maps the Boolean cube to $\{0,1\}$, we show that if $n=p^2$, when $p$ is a prime, then deg$(f)\geq n-\sqrt{n}$. This slightly improves the previous bound of von zur Gathen and Roche for this case.

TR10-038 | Satisfiability Allows No Nontrivial Sparsification Unless The Polynomial-Time Hierarchy Collapses | Dieter van Melkebeek, Holger Dell

2010-03-10T00:13:21Z

Consider the following two-player communication process to decide a language $L$: The first player holds the entire input $x$ but is polynomially bounded; the second player is computationally unbounded but does not know any part of $x$; their goal is to cooperatively decide whether $x$ belongs to $L$ at small cost, where the cost measure is the number of bits of communication from the first player to the second player. For any integer $d \geq 3$ and positive real $\epsilon$ we show that if satisfiability for $n$-variable $d$-CNF formulas has a protocol of cost $O(n^{d-\epsilon})$ then coNP is in NP/poly, which implies that the polynomial-time hierarchy collapses to its third level. The result even holds when the first player is conondeterministic, and is tight as there exists a trivial protocol for $\epsilon = 0$. Under the hypothesis that coNP is not in NP/poly, our result implies tight lower bounds for parameters of interest in several areas, namely sparsification, kernelization in parameterized complexity, lossy compression, and probabilistically checkable proofs. By reduction, similar results hold for other NP-complete problems. For the vertex cover problem on $n$-vertex $d$-uniform hypergraphs, the above statement holds for any integer $d \geq 2$. The case $d=2$ implies that no NP-hard vertex deletion problem based on a graph property that is inherited by subgraphs can have kernels consisting of $O(k^{2-\epsilon})$ edges unless coNP is in NP/poly, where $k$ denotes the size of the deletion set. Kernels consisting of $O(k^2)$ edges are known for several problems in the class, including vertex cover, feedback vertex set, and bounded-degree deletion.

Efficient Parallel and Out of Core Algorithms for Constructing Large Bi-directed de Bruijn Graphs

2010-03-10T01:30:21Z

Authors: Vamsi Kundeti, Sanguthevar Rajasekaran, Hieu Dinh
Download: PDF
Abstract: Assembling genomic sequences from a set of overlapping reads is one of the most fundamental problems in computational biology. Algorithms addressing the assembly problem fall into two broad categories -- based on the data structures which they employ. The first class uses an overlap/string graph and the second type uses a de Bruijn graph. However with the recent advances in short read sequencing technology, de Bruijn graph based algorithms seem to play a vital role in practice.

Efficient algorithms for building these massive de Bruijn graphs are very essential in large sequencing projects based on short reads. In Jackson et. al. ICPP-2008, an $O(n/p)$ time parallel algorithm has been given for this problem. Here $n$ is the size of the input and $p$ is the number of processors. This algorithm enumerates all possible bi-directed edges which can overlap with a node and ends up generating $\Theta(n\Sigma)$ messages.

In this paper we present a $\Theta(n/p)$ time parallel algorithm with a communication complexity equal to that of parallel sorting and is not sensitive to $\Sigma$. The generality of our algorithm makes it very easy to extend it even to the out-of-core model and in this case it has an optimal I/O complexity of $\Theta(\frac{n\log(n/B)}{B\log(M/B)})$. We demonstrate the scalability of our parallel algorithm on a SGI/Altix computer. A comparison of our algorithm with that of Jackson et. al. ICPP-2008 reveals that our algorithm is faster. We also provide efficient algorithms for the bi-directed chain compaction problem.

Congratulations to Chuck Thacker

2010-03-09T16:19:00Z

The news has hit the wires -- Chuck Thacker is this year's Turing Award winner.

I had the great pleasure of getting to know Chuck while I worked at DEC SRC. He's a character, a tinkerer, and a great and curious mind. I think recognizing his work -- the Alto -- is a great choice.

Subscribe to comments | Post a comment

Home-baked graphics

2010-03-09T15:44:41Z

A couple of commenters have asked what software package I use to create the graphs that appear in bit-player posts–illustrations like the one below, which is a slightly improved version of something I posted last week. Let’s call it Figure 1.

Prompted by these inquiries, I immodestly ask myself: Why do my graphs look so darn good? I immodestly answer: It’s not because of any packaged software! I don’t need a cake mix, or even a recipe. These are home-baked graphs, made from scratch out of locally grown organic pixels.

I have strong opinions about the aesthetics of scientific illustrations, and I could certainly spout off about the design elements of Figure 1, such as that putty-colored background, just dark enough to allow drop-out white grid lines, yet neutral enough to avoid competing with the data curves, which also have a distinctive color scheme on which I could discourse at length. Yes, I can talk the Tufte talk. But I think the commenters were really asking how I create the graphs rather than why they’re so elegant, and so I’m going to focus here on the practical programming problem.

Most of my experience in drawing pictures with a computer comes from the world of print publishing, where the final product is ink on paper rather than pixels on a screen. Compared with the online environment, print has some advantages, notably higher resolution (up to 1,000 dots per centimeter) and precise control over typography and color. But print also has obvious limitations: On a magazine page, there are no mouseovers or clickable buttons, and you can’t make a square knot twirl in 3D.

Thirty years ago, the big challenge for computer-generated illustrations was not how to draw the picture but how to get it out of the computer and onto the printing press. You couldn’t just export a PDF and place it in a Quark or InDesign document; none of those things existed. The only practical option was to print out the artwork, photograph it, and “strip” the negative into the page-size film that would be used to make the press plate. Because of this emphasis on printouts, most of the effort went into programming the printer rather than the computer.

The figure below is the first published computer-generated illustration I had a hand in creating. It appeared in Scientific American in 1983.

The array of 28² tiny bar graphs was produced with an Epson MX-80 dot-matrix printer, using escape codes to fire combinations of the eight pins in the printhead. Of course the MX-80 was a black-and-white device. The two-color illustration was created from two separate printouts. Also, the Epson letterforms were replaced with typeset characters.

The world of computer-generated illustrations changed dramatically with the arrival of PostScript, the “page description language” created by John Warnock and his colleagues at Adobe Systems (based in part on earlier work at Evans and Sutherland and Xerox PARC). PostScript was designed as a complete programming language rather than just a file format or a set of drawing commands. And something else set it apart as well: attention to details of graphic design. With most earlier software (such as programs based on the Apple Quickdraw library), trying to create publishable figures was an exercise in frustration. For example, the apparent weight of a line would vary depending on its orientation: lighter when vertical or horizontal, heavier when diagonal. PostScript allows very precise control over such niceties of presentation. To take another example, where lines meet the edge of a graph, you don’t want to have to choose between falling short and overshooting; PostScript provides the tools needed to make it look right.

(The version in the rightmost panel is created by allowing the colored lines to extend outside the background box, and then applying a clipping mask that cuts off all objects at the boundary of the box.)

Obsessing over minute details like these may seem comically fussy, but I believe that neatness counts in these matters. To some extent, illustration is an art of illusion. Graphs and diagrams work best when you can look through them rather than at them. The viewer should be seeing the underlying information or abstraction–the array of correlation coefficients, the function y = f(x), or whatever–rather than noticing the mechanics of how the drawing was constructed. A ragged edge is the kind of distraction that destroys the illusion.

Although PostScript was a giant step forward from the MX-80 command set, in the early years it was still just another printer language, not a computer language. The only way I could execute a PostScript program was to send it to a laser printer and wait to see what came out. Sometimes it was a long wait. I had no way of running a PostScript program on the computer itself. (Ghostscript came later.)

My first PostScript illustrations were created as hand-written PostScript programs; the same language was used both for doing the computations and for presenting the results. The faces at right were created in this way. (They were inspired by the work of Herman Chernoff and drawn to illustrate an American Scientist article by Robert Levine in 1990.) The dual role of the language caused me a moment of disorientation just now when I went looking for my records of this project. I found an EPS (encapsulated PostScript) file, which I knew was the finished illustration, but where was the source code? And then I remembered: It’s the same file! Open it up in Ghostscript or Adobe Illustrator and you see those silly faces smiling or scowling at you; open the same file in a text editor, and you see procedures for drawing elements of the faces:

   /draweyes
     { newpath
       dx dy eyewidth eyeheight 0 360 ellipse stroke
       ex ey eyewidth eyeheight 0 360 ellipse stroke
     } bind def
   /drawpupils
     { fx fy pupilsize pupilsize 0 360 ellipse fill
       gx gy pupilsize pupilsize 0 360 ellipse fill
     } bind def

Bill Casselman, the graphics editor of the Notices of the American Mathematical Society, still favors this direct-to-PostScript methodology. He has written an excellent guidebook, taking you from the basics of PostScript through an elaborate library for rendering three-dimensional objects.

But here I part company from Casselman; I’d rather not do all my computing in PostScript. It’s not that I have anything against the language itself, but the development environment is not to my taste. I therefore adopted the modus operandi of writing a program in my language of choice (usually some flavor of Lisp) and having that program write a PostScript program as its output. After doing this on an ad hoc basis a few times, it became clear that I should abstract out all the graphics-generating routines into a separate module. The result was a program I named lips (for Lisp-to-PostScript).

Most of what lips does is trivial syntactic translation, converting the parenthesized prefix notation of Lisp to the bracketless postfix of PostScript. Thus when I write (lineto x y) in Lisp, it comes out x y lineto in PostScript. The lips routines also take care of chores such as opening and closing files and writing the header and trailer lines required of a well-formed PostScript program.

But the lips interface is low-level, confined to drawing individual dots, line segments, rectangles and the like. Assembling a complete graph out of these primitives is tedious. For example, the grid of white lines in Figure 1 would have to be drawn one line at a time, with each line specified by a sequence of commands such as

    (newpath)
    (moveto u v)
    (lineto x y)
    (stroke)

Before you can issue those commands, you have to calculate u, v, x and y. Clearly, a higher-level front end is needed; like everyone else, I call mine plot.

At the core of any plotting program is a simple operation: mapping points from an abstract user space to coordinates in a rectangular pane, the page space. In Figure 1, the y axis runs from 0 to 5000; values in this range have to be scaled to the dimensions of the graph, which is about 300 PostScript points, or 11 centimeters. Mathematically, the transformation is straightforward. Indeed, if I wished I could leave all the arithmetic to the PostScript interpreter, simply passing in the appropriate matrix elements for scaling and translation. This is an attractive option; it would allow plot to work entirely in user space. But a few niggling details get in the way. Consider the tick marks along the y axis in Figure 1. Their vertical positions are conveniently expressed in user coordinates: one tick every 500 units. But what about the length of the ticks–their horizontal extent? This dimension is purely concerned with the appearance of the graph and has nothing to do with the content; it ought to be expressed in unscaled units of points or pixels.

Here’s a possible solution: Let everything inside the rectangular frame of the graph–the area with the putty-colored background in Figure 1–go through the scaling engine, but define everything outside the frame, including the tick marks and the axis labels, directly in page coordinates. If you think this is the final answer, take a look at Figure 2:

In this nonsensical graph (constructed just for this occasion), data points are indicated by stars, crosses and diamonds. The positions of those glyphs ought to be defined in user space, but the drawing commands that create the shapes are properly defined in page coordinates. If we tried to draw the glyphs in user space, their size and shape would vary with position in the graph.

What’s the best way to deal with this messy situation? Is there some tidy solution that will reconcile the two coordinate systems and allow all dimensions to be treated uniformly? I don’t believe so; it’s just in the nature of graphs to mix up elements from these two disparate realms. We look through a window into a world of data or mathematical abstractions, but we also draw our own little doodles on the window itself.

Of course there are solutions; they’re just not as pretty as I would like. My own strategy for coping is to attach extra information to each geometric point, indicating whether or not the x and y coordinates are to go through the scaling transformation. This is less troublesome than it might seem; from the user’s point of view, it’s almost always invisible.

In writing the lips and plot programs, I walk a path that is already worn smooth by many earlier footsteps. I don’t know who wrote the first computer program for plotting data, but it probably came soon after the first program for producing data. Today we have hundreds of clever, comprehensive, well-designed and well-maintained programs for plotting and graphing. Gnuplot is very capable; Grace is one I’ve never used but I’ve heard good things about it; Mathematica, Sage, R, MATLAB, Octave and the like all have elaborate graphics facilities built in; the Python world, as usual, has an overabundance of options; there are a few libraries for my beloved Lisp; you can even do dataviz online.

All of which raises the question of why I bother to roll my own. I’ll never keep up–or even catch up–with the efforts of major software companies or the huge community of open-source developers. In my own program, if I want something new–treemaps? vector fields? the third dimension?–nobody is going to code it for me. And, conversely, anything useful I might come up with will never benefit anyone but me.

The trouble is, every time I try working with an external graphics package, I run into a terrible impedance mismatch that gives me a headache. Getting what I want out of other people’s code turns out to be more work than writing my own. No doubt this reveals a character flaw: Does not play well with others.

In any case, the time for change is coming. My way of working is woefully out of date and out of fashion. PostScript is a technology that even Adobe seems to regard as outmoded. And making ultraprecise PostScript graphs is quite silly when their destination is the web; before I can put them online, I have to convert them to low-res PNG images. Furthermore, a PostScript-based workflow loses out on all the interactive richness of the web. These are deathly still images. How can I expect to earn any web cred when my work is not even clickable, much less multitouch-enabled?

If I continue in my stubborn, do-it-yourself mode, I could replace the PostScript back end with one that generates SVG. This wouldn’t be a major undertaking. But is SVG the right answer? It’s been around for more than a decade and you still don’t see much of it in the wild. And there are horrid browser incompatibilities. I suspect that Javascript (and JQuery) has a brighter future. And if I can get over my abreaction to libraries, there are plenty of options. Advice anyone?

Update 2010-03-13: Many thanks for all the thoughtful comments. Herewith a few comments on comments:

Ron Renaud’s graphs using Javascript in a canvas element are really very pretty, and they give me renewed hope that web graphics can measure up to a print standard. But is the world quite ready for the canvas? This is a blog, after all. Lots of people get to it with an RSS reader, not a web browser.

Zvika requests a link to a higher-resolution PNG. I don’t know how to do that. I can make a larger PNG, but the resolution–the dots per centimeter–is really determined by the screen you’re looking at. Which is not to say that larger illustrations wouldn’t be a good idea. When I redesign these pages, I want to allow more room for bigger pictures.

Gary Reuben suggests Python Matplotlib and the ggplot2 package for R. The latter is new to me, and very impressive. I want to go read more about it.

Several other readers favor SVG. I’m okay with that. It looks easy to change my current software to generate SVG output instead of (or in addition to) PostScript. The question remaining for me is whether SVG on the web is something that browsers (and, again, RSS readers) can swallow without choking.

John Haugland mentions PDF as the successor to PostScript. I couldn’t possibly survive without PDF these days, but I don’t see it as an ideal medium for illustrations embedded in web pages. Reading PDFs within a browser requires a plugin, which some people refuse to install. (I’m one of those people.) Furthermore, because PDF is a binary format based on a directory of offsets to tables, it’s more trouble to write PDF files than either PostScript of SVG.

Nate mentions Processing, the graphics and animation language created by Casey Reas and Ben Fry. I’ve written about this before at bit-player and elsewhere. I’m a big fan, but Processing is essentially a front end to Java, and I have reservations about embedding Java applets in web pages. John Ressig’s reimplementation of the language in Javascript overcomes that problem, and one of these days I’ll get around to doing something serious with it.

Finally, Marc asks for a look at my Lisp code. I’m always shy about sharing such unfinished things, but here it is.

HW policies: PROS and CONS

2010-03-09T15:38:00Z

The last blog entry had lots of good comments about different HW policies. I enumerate them and say PROS and CONS

Hard Deadline. PRO- uniform, no favoritism, can post HW Solutions or go over HW in class as soon as it is handed in. CON- there could be legitimate reasons for lateness that are short of a doctors note. CON- you want the student to DO the HW even if it will be late. CON- you need to be TOUGH to say NO.
Moral Deadline (what I do, see last post). Same as Hard Deadline, but its a bit easier to say NO.
Penalty for lateness. PRO- the students will still do the HW. CON- delay in posting solution. CON- slackers are still slackers. ODDITY- the penalty is supposed to discourage lateness. But it may encourage it (gee, 10% off if I hand it in one day late. OKAY, its a deal)
Look at late HW only if they affect the final grade. PRO- less to look at likely, CON- Don't really want to keep track of these things. CON- student may not be discouraged from handing things in late.
Only count (say) 10 of the 12 HWs, and have HARD DEADLINES. PRO- same as HARD DEADLINE. CON- students will blow off 2 HW's, possibly the last two which may be important for the final. CAVEAT- raises the much bigger question of whether to treat students like adults or like ...students.
Students get x number of late days (this one was new to me). PRO- well defined rule, flexible but no favoritism. CON- delay in posting solutions. CON- keeping track of it.
If you miss a HW then the others will count more (up to some limit). PRO- uniform. CON- students may still miss some HW they should do.
HW are OPTIONAL. PRO- they sink or swim on their own. CON- they sink or swim on their own.

Diff topic- how much to COUNT HW? I often count it low (like 10-20 percent) so that I don't' have to worry too much about cheating. Actually I think its GOOD if students help each other but BAD if students copy each other, but it can be hard to tell.

Which of these work best? Depends alot on the school and the course and even the profs willingness to say NO.

Subscribe to comments | Post a comment

EC Papers Up

2010-03-09T14:16:00Z

The list of accepted papers for EC is up. I'm happy to say that our Swoopo paper made the list.

I was surprised in the acceptance letter to find that there were 45 acceptances out of 136 papers -- an acceptance rate of about 1/3. (Compare with WSDM.) This makes it one of the less "competitive" CS conferences I know of, although a little research shows this is a bit unusual -- last year the numbers were 40/160 or so, so they accepted more papers and had fewer submissions this year. Is that a trend in the making or an accident of timing this year? Also, while I'm an EC newbie, the list of papers looks very interesting, with plenty of top-tier names. "Competitive" or not, I'm expecting high quality. I'm really looking forward to it -- and not just because its location makes it remarkably convenient for those of us in the greater Boston area.

Subscribe to comments | Post a comment

EC accepted papers

2010-03-09T12:58:56Z

The list of accepted papers for EC 2010 has been posted.

A Class of Graph Properties Computable in Linear Time

2010-03-09T00:24:37Z

Monadic second order logic and treewidth

Ron Fagin is a great theorist who has made many important contributions to diverse aspects of computing. He is perhaps most famous for his brilliant categorization of polynomial time by second order logic—Fagin’s Theorem.

Today I plan on talking about a related theorem—Courcelle’s—proved in 1990. Yet the theorem is not, in my opinion, as well known as it should be. The truth is I did not know this pretty theorem until I ran across it the other day. So perhaps everyone else knows Courcelle’s result except for me. Oh well.

Courcelle’s Theorem

Bruno Courcelle’s theorem is quite striking, in my opinion:

Theorem: Let be any graph property that is definable in MSO logic. For any fixed , the there is a linear time algorithm for testing the property on any graph of treewidth at most .

I will explain the theorem and try to give a high level view of its proof. But, even if the concepts of MSO—Monadic Second Order—logic and treewidth are new to you, the theorem should still be striking. There are a few theorems of the form: any graph property expressible in a powerful logic is computable on a class of graphs in linear time.

Treewidth

The concept of treewidth is fairly widely known, and was used extensively in the famous work of Neil Robertson and Paul Seymour on graph minors. I will include a definition here for completeness. My intuition is that for most graph problems the case of trees is easy: for example, there is a linear time algorithm for tree isomorphism. The notion of small tree width is an attempt—a very successful attempt—to generalize trees to include a much larger class of graphs.

A tree decomposition of a graph is a composed of a family of sets of vertices, called bags, and a tree on the nodes with these properties:

Every vertex of is in at least one bag.
Every edge of has both endpoints in some bag.
For all vertices of , the set of bags that contain the vertex form a subtree of .

The last is more precisely: for all vertices of the set is a subtree of . The width of the decomposition is the maximum of over all . The treewidth of a graph , is the minimum such width over all valid tree decompositions. The is there to make the treewidth of a tree

Here is an example, from our friends at Wikipedia, of a graph and a decomposition:

MSO

Suppose that stands for the edge relationship of some simple undirected graph. Then, first order sentences allow variables only over vertices. For example, the sentence

means that the graph has no triangle. This is a first order sentence, since all the variables range over vertices of the graph.

First order sentences can capture some interesting properties—they can express the property of having a -clique for any fixed . However, first order is too weak to expressive many important graph properties such as the property: a graph is -colorable.

This leads to the idea to extend the first order logic to allow more powerful variables. A natural notion is to allow variables to range over arbitrary sets of tuples of vertices: this is called Second Order (SO) logic. This theory is very powerful, since Fagin’s Theorem is:

Theorem: Properties expressible by second-order logic existential sentences are precisely the complexity class NP.

An existential sentence is a sentence of the form:

where and and range over relations and is a first order formula.

Clearly, there are properties of this form requiring more than linear time—even on trees. This is the reason that Courcelle’s Theorem must restrict the properties to Monadic Second Order (MSO) logic. A sentence is a monadic one provided all the second order variables range over sets. Notice that being expressible in MSO logic is a stronger requirement than being expressible in Second Order logic: MSO is a subset of Second Order logic.

It is now easy to define -colorable. Here is how to express -colorable:

where uses first order quantifiers to check form a partition of the vertices, and that if then and are colored differently. Think of as meaning is colored “A,” and the same for the other sets.

The logic MSO is quite powerful, but is still not powerful enough to define some properties: for example, the property of having a Hamiltonian Circuit, is not definable in MSO.

Proof Idea

The proof of Courcelle’s theorem is based on two key ideas:

The ability to find a -treewidth decomposition in linear time.
The ability to check the MSO property on a tree decomposition in linear time.

The first is a result due to Hans Bodlaender, who improved the original algorithm of Robertson and Seymour from quadratic time; see this for a nice survey and discussion of algorithms for finding treewidth.

Once a tree decomposition is found the key is a tree automaton can be constructed to check the MSO sentence. Tree automata are a generalization of finite automata from linear words to finite trees. Luckily many of the same constructions and properties of finite automata carry over to tree automata, this allows the checking to be done in linear time.

Open Problems

The Courcelle theorem is, in my opinion, quite pretty. However, the hidden constants are huge, and an obvious question is when can “real” linear time algorithms be found for properties expressible in MSO? Another interesting open problem is what is the largest class of sentences of SO that lead to linear time algorithms on graphs of bounded treewidth?

Who pays for submissions ?

2010-03-08T22:34:00Z

Writing a paper takes a tremendous amount of time. So, one of the frequent complaints that authors make is when PC members submit half-baked, clearly below-threshold reviews on a paper just to get the resume bullet and claim to have done their reviewing duties. Personally, I feel intense anger when receiving crappy reviews that come with not the slightest bit of insight, and then am expected to rebut them or accept them. Not to mention the long-term psychological damage incurred by having papers rejected one after another.

The problem is that reviewing a paper for a conference is free: all it takes is a few clicks of the mouse to upload your PDF file. (Of course, I'm not accounting for the cost of doing the research (ha!) and actually reviewing the paper.)

Let's estimate the costs associated with doing research and submitting papers to conferences. I spend many months working, writing and submitting papers to conferences. A highly competitive conference will assign three reviewers to my paper, and with a lot of luck one of them might even tangentially be aware of my research area. After I make up a bunch of numbers, the cost of rejection of my paper amounts to over 3 gazillion dollars, none of which I can recoup. It's clear that conferences, which only survive if people submit, should be paying me to submit !

Of course, imposing this kind of a fee would no doubt drastically reduce the number of papers that are submitted. But this seems like a good thing: it would probably reduce the number of conferences, and remove the fiction that conferences actually do "quality control", leaving them with their original purpose of networking and creating a sense of community. Conferences could generate revenue by charging reviewers for the opportunity to preview the new works being submitted: this would potentially also improve the quality of the reviews as well. Although the financial incentive is not that great, getting paid should encourage TPC members to take the process more seriously.

The only downside I can see is people who submit a ton of papers everywhere and become "professional paper writers", but TPC chairs would clearly have to balance this against the research credentials of the people submitting papers. Note that many journals impose author fees for publication of the paper, so this provides a nice offset against that cost.

It just seems crazy to me that the research community provides this free paper previewing service for committees with no negative ramifications for writing totally bogus reviews.

Disclaimers for the sarcasm-challenged:

Yes, I am obviously aware of Matt Welsh's post on this topic
Yes, this is a total ripoff/parody of his post
Yes in fact, I disagree with his point.

Subscribe to comments | Post a comment

Asymptotics and usability

2010-03-08T21:06:25Z

In Google Wave, when a user is editing a wave, every keystroke entails sending messages back and forth between that user, the Wave servers, and every other person who has the same wave open at the same time, so that all users can see what everybody else is typing as they type it. But that's not so bad: it's just a small message, and everything is still pretty fast, at least on a new wave with not much in it yet. Despite all this communication the system can be very usable and responsive.

However, when a wave grows to a hundred or so messages and a few tens of thousands of characters, everything slows down to a crawl. Each keystroke takes a second or so to process. Typing directly into a wave can become so painful that it can be much easier to switch back and forth between a separate text editor and Wave and to copy and paste large chunks of text (each a single operation, as fast as a single keystroke) rather than editing directly within the wave. The only way to return to a responsive system seems to be to archive the wave and start a fresh one.

So far this is verifiable personal experience but the rest is speculation. My wild guess at the underlying cause for this lack of scalability: as each keystroke is processed, it entails an amount of work by the browser, or worse an amount of communication between the browser and the Wave servers, that's linear in the size of the wave. For instance, it's plausible to me (though I don't know the details of the communication protocol) that each update causes the server to communicate the entire new updated state of the wave to the browser. The net effect is that, if one creates a wave by typing a character at a time, the number of bytes communicated, over the course of the wave, is quadratic in the number of characters in the wave. And as we all (should) know, linear algorithms scale well to large amounts of data and quadratic algorithms don't.

The moral: asymptotic analysis matters for usability, even for problem sizes that are not massive (there is no problem fitting a whole wave into main memory) and even in situations such as this one where the designers are probably thinking less about algorithmic efficiency and more about functionality and user interface design. If your system doesn't scale, it will become slow and unusable when its users push its boundaries, and that will limit the uses people make of it.

Hiring at UMD, part 2

2010-03-08T16:33:30Z

As reported earlier, we had a very accelerated hiring season at UMD — all our candidates were brought in for interviews before the end of last semester(!)

I am pleased to announce that three faculty will be joining our department, including one “core theorist” and two other “theory-friendly” profs. These are:

Mohammad Taghi Hajiaghayi (Algorithms)
Hal Daume (Natural Language Processing/Machine Learning)
Hector Corrada Bravo (Computational Biology)

I look forward to seeing them all here!

A HW policy- MORAL due date.

2010-03-08T15:28:00Z

This semester I am using the following HW policy.

HW is due on Tuesday. However, your dog died! Hence you get an extension to Thursday. That is, for all people in the class I assume you have a quasi-legit reason to ask for an extension to Thursday. Hence you can hand it in Thursday for full credit. However, if you want an extension past that you will not get it since I already gave you an extension to Thursday. (There may be some severe exceptions which will have to be documented.)

This will save alot of time in terms of students asking permission to hand it in late since I will say I already have you an extension and you are asking for another one?
Some students will get into the habit of handing it in Thursday. This is okay so long as they do not ask for an extension past that.
Clyde tells me that this is really a cheat- the HW really is due Thursday. I may have a higher moral ground when telling them they can't hand it in later than Thursday, but they will still feel that they deserve an extension if their dog dies on Wednesday. My response: they do not.
I do make sure that they have enough knowledge to do the HW by Tuesday.
I am teaching one Junior-Senior class and one honors-class so these are already pretty good students. They (I hope) know what I mean when I say that they cannot ask for an extension past Thursday. Also they will likely not need them. I have not tried this in a Freshman class. I would like to but they are usually co-taught and large so it would be harder to manage.

Subscribe to comments | Post a comment

TR10-037 | Simulating Independence: New Constructions of Condensers, Ramsey Graphs, Dispersers, and Extractors | Boaz Barak, Guy Kindler, Ronen Shaltiel, Benny Sudakov, Avi Wigderson

2010-03-08T14:03:07Z

We present new explicit constructions of *deterministic* randomness extractors, dispersers and related objects. We say that a distribution $X$ on binary strings of length $n$ is a $\delta$-source if $X$ assigns probability at most $2^{-\delta n}$ to any string of length $n$. For every $\delta>0$ we construct the following poly($n$)-time computable functions: 1) (2-source disperser:) $D:(\{0,1\}^n)^2 \rightarrow \{0,1\}$ such that for any two independent $\delta$-sources $X_1,X_2$ we have that the support of $D(X_1,X_2)$ is $\{0,1\}$. 2) (Bipartite Ramsey graph:) Let $N=2^n$. A corollary is that the function $D$ is a 2-coloring of the edges of $K_{N,N}$ (the complete bipartite graph over two sets of $N$ vertices) such that any induced subgraph of size $N^{\delta}$ by $N^{\delta}$ is not monochromatic. (3) (3-source extractor:) $E:(\{0,1\}^n)^3 \rightarrow \{0,1\}$ such that for any three independent $\delta$-sources $X_1,X_2,X_3$ we have that $E(X_1,X_2,X_3)$ is $o(1)$-close to being an unbiased random bit. No previous explicit construction was known for either of these for any $\delta<1/2$, and these results constitute significant progress to long-standing open problems. A component in these results is a new construction of condensers that may be of independent interest: This is a function $C:\{0,1\}^n \rightarrow (\{0,1\}^{n/c})^d$ (where $c$ and $d$ are constants that depend only on $\delta$) such that for every $\delta$-source $X$ one of the output blocks of $C(X)$ is (exponentially close to) a $0.9$-source. (This result was obtained independently by Ran Raz). The constructions are quite involved and use as building blocks other new and known objects. A recurring theme in these constructions is that objects which were designed to work with independent inputs, sometimes perform well enough with correlated, high entropy inputs. Preliminary version of this paper appeared in STOC 2005.

TR10-036 | On the Relation between Polynomial Identity Testing and Finding Variable Disjoint Factors | Amir Shpilka, Ilya Volkovich

2010-03-08T12:05:34Z

We say that a polynomial $f(x_1,\ldots,x_n)$ is {\em indecomposable} if it cannot be written as a product of two polynomials that are defined over disjoint sets of variables. The {\em polynomial decomposition} problem is defined to be the task of finding the indecomposable factors of a given polynomial. Note that for multilinear polynomials, factorization is the same as decomposition, as any two different factors are variable disjoint. In this paper we show that the problem of derandomizing polynomial identity testing is essentially equivalent to the problem of derandomizing algorithms for polynomial decomposition. More accurately, we show that for any reasonable circuit class there is a deterministic polynomial time (black-box) algorithm for polynomial identity testing of that class if and only if there is a deterministic polynomial time (black-box) algorithm for factoring a polynomial, computed in the class, to its indecomposable components. An immediate corollary is that polynomial identity testing and polynomial factorization are equivalent (up to a polynomial overhead) for multilinear polynomials. In addition, we observe that derandomizing the polynomial decomposition problem is equivalent, in the sense of Kabanets and Impagliazzo, to proving arithmetic circuit lower bounds to NEXP. Our approach uses ideas from a previous work in which we showed that the polynomial identity testing problem for a circuit class $\mathcal M$ is essentially equivalent to the problem of deciding whether a circuit from $\mathcal M$ computes a polynomial that has a read-once arithmetic formula.

SAGT 2010

2010-03-08T09:49:36Z

The 3rd International Symposium on Algorithmic Game Theory, SAGT2010, will be held on October 18-20, 2010 in Athens, GREECE. Submission deadline is May 3rd.

Choosing the number of clusters I: The Elbow Method

2010-03-08T06:03:00Z

(this is part of an occasional series of essays on clustering: for all posts in this topic, click here)

It's time to take a brief break from the different clustering paradigms, and ponder probably THE most vexing question in all of clustering.

How do we choose k, the number of clusters ?

This topic is so annoying, that I'm going to devote more than one post to it. While choosing k has been the excuse for some of the most violent hacks in clustering, there are at least a few principled directions, and there's a lot of room for further development.

(ed. note. The Wikipedia page on this topic was written by John Meier as part of his assignment in my clustering seminar. I think he did a great job writing the page, and it was a good example of trying to contribute to the larger Wikipedia effort via classroom work)

With the exception of correlation clustering, all clustering formulations have an underconstrained optimization structure where the goal is to trade off quality for compactness of representation. Since it's always possible to go to one extreme, you always need a kind of "`regularizer"' to make a particular point on the tradeoff curve the most desirable one. The choice of $k$, the number of clusters, is one such regularizer - it fixes the complexity of the representation, and then asks you to optimize for quality.

Now one has to be careful to see whether 'choosing k' even makes sense. Case in point: mixture-model clustering. Rather than asking for a grouping of data, it asks for a classification. The distinction is this: in a classification, you usually assume that you know what your classes are ! Either they are positive and negative examples, or one of a set of groups describing intrinsic structures in the data, and so on. So it generally makes less sense to want to "`choose"' $k$ - $k$ usually arises from the nature of the domain and data.

But in general clustering, the choice of $k$ is often in the eyes of the beholder. After all, if you have three groups of objects, each of which can be further divided into three groups, is $k$ 3 or 9 ? Your answer usually depends on implicit assumptions about what it means for a clustering to be "`reasonable"' and I'll try to bring out these assumptions while reviewing different ways of determining $k$.

The Elbow Method

The oldest method for determining the true number of clusters in a data set is inelegantly called the elbow method. It's pure simplicity, and for that reason alone has probably been reinvented many times over (ed. note: This is a problem peculiar to clustering; since there are many intuitively plausible ways to cluster data, it's easy to reinvent techniques, and in fact one might argue that there are very few techniques in clustering that are complex enough to be 'owned' by any inventor). The idea is this:

Start with $k=1$, and keep increasing it, measuring the cost of the optimal quality solution. If at some point the cost of the solution drops dramatically, that's the true $k$.

The intuitive argument behind the elbow method is this: you're trying to shoehorn $k$ boxes of data into many fewer groups, so by the pigeonhole principle, at least one group will contain data from two different boxes, and the cost of this group will skyrocket. When you finally find the right number of groups, every box fits perfectly, and the cost drops.

Deceptively simple, no ? It has that aspect that I've mentioned earlier - it defines the desired outcome as a transition, rather than a state. In practice of course, "`optimal quality"' becomes "`whichever clustering algorithm you like to run"', and "`drops dramatically"' becomes one of those gigantic hacks that make Principle and Rigor run away crying and hide under their bed.

The Alphabet-Soup Criteria

So can we make the elbow method a little more rigorous ? There have been a few attempts that work by changing the quantity that we look for the elbow in. A series of "`information criteria"' (AIC, BIC, DIC, and so on) attempt to measure some kind of shift in information that happens as we increase $k$, rather than merely looking at the cost of the solution.

While they are all slightly different, they basically work the same way. They create a generative model with some kind of term that measures the complexity of the model, and another term that captures the likelihood of the given clustering. Combining these in a single measure yields a function that can be optimized as $k$ changes. This is not unlike the 'facility-location' formulation of the clustering problem, where each "`facility"' (or cluster) must be paid for with an 'opening cost'. The main advantage of the information criteria though is that the quantification is based on a principled cost function (log likelihood) and the two terms quantifying the complexity and the quality of the clustering have the same units.

Coming up next: Diminishing returns, the ROC curve, and phase transitions.

Subscribe to comments | Post a comment