We’d like to paginate with the sort order criteria “x ascending, y ascending”. For example, that index might look like this.

While paginating, we need to know whether there is a “next page”, because we don’t want to return an empty page unless we have no objects.

To do this, we’ll look-ahead for the next item in our search. So, if the page size is 2, we’ll just fetch 3 items. That extra object will give us the necessary (x, y) coordinates we need to resume the search, that is, to fetch the next page. This object is called the “bookmark”.

Given the (x, y) bookmark, how do we fetch the next page? In index terms, we might construct a prefix string “/$x/$y/”, and return the next rows after that in our index. Unfortunately App Engine doesn’t give us a low-level interface like that, so we have to use filters.

What kind of filters? We want all objects greater than or equal to our given “x”. But, for objects with the same value as “x”, we have to skip those less than our given “y”, because we have already seen them. Clearly we will need filters on both “x” and “y”, but woe, App Engine insists that no more than one distinct field can have a filter expression per query.[1]

Ryan Barrett observed that for a search with n sorted fields, you will need n queries to resume the search from a bookmark. In other words, you need one query for each field.

What does this look like? In our example, we have two sorted fields. To resume the search, the first query will fix “x” and filter on “y”, with “y ascending”. The second query will filter on “x” with “x ascending, y ascending”.

What would that look like on our example index?

Each shaded region is one query, that is, one trip to the datastore. The green objects are the current page. The orange object is the next bookmark. The blue objects are the old pages.

The first query is on a fixed “x” and filters “y”. The second query resumes where the first left off, and filters on “x”. Since the sort orders are consistent with the filters, we obtain a consistent view of the index.

On the fourth search, we happen to get lucky and find our bookmark on the first query, so we can skip the second query.

In general

If you’re sorting on n fields {f₀, f₂, … f_n-1} you will need n queries {q₀, q₁, … q_n-1}.

Query q_i will filter f_n-i and fix fields {f₀, … f_n-i-1}.

For n=2, query q₀ filters f₁ and fixes f₀. Query q₁ filters f₀.

Why don’t you use cursors you slob?

Cursors are great! Except, they don’t immediately answer the question, “Do we have a next page?” If you don’t care then by all means use cursors, they’re much easier to use than bookmarks.

Cursors in App Engine can’t look-ahead. Instead, you have to fetch one object with the query’s next cursor, rather than fetching an extra object in the original query.

On the other hand, cursors are allowed to perform queries which users cannot. Cursors can perform the “prefix string” trick described above. So, a cursor can do in one query what takes us two or more.

So which should you use? That probably depends on how many fields you are sorting on. For a single field, I prefer the look-ahead technique, because it only requires one trip to the datastore. For two fields, either technique requires (at most) two trips, although cursors are usually simpler. For more than two fields, cursors easily win.

[1] This requirement exists to prevent users from requesting filters which the datastore cannot satisfy by scanning consecutive rows in an index. For example, the filters “x > 2 and y > 3″ can specify non-consecutive rows of an index that sorts “x” and “y” ascendingly.

M and I performed a double-blind study of four kinds of water. Our preferences agreed, in order: purified water, filtered tap water, natural spring water, and unfiltered tap water.

(I know the labels here don’t make sense. They were the first four Greek letters I could recall.)

Progress updates to follow.

This model rules Yelp. A review on yelp is a measurement of the distance between the reviewer and the reviewed. The review will be favorable if and only if the reviewer is nearby the reviewed in the model.

A restaurant (e.g., Bite) gets good reviews because the people that review it know or can guess, before they go there, that they will like it. If they didn’t, the place wouldn’t have such stellar reviews.

So why do some places get terrible reviews? I see cases that would cause this. First, if a new nightclub (e.g., Universal) is still “figuring itself out,” it might attract a lot of people expecting one thing and getting another. In other words, they are not near the place in the model, but they visit it anyway because they think they might. Likely, because the place is so new, many people are testing it out, so you can expect the reviews to be unfavorable, even by the people who are supposed to like it, because the presence of outsiders ruins the whole experience.

In another case, a super-exclusive bar might only cater to a very narrow range of clientele. In other words, you have to be unusually close to this bar in the model in order to like it. Perhaps it only supports one or two cliques of regulars, and everyone else is shunned.

In this example, the radii correspond to the ability of a bar or restaurant to support a wide or narrow section of people.

Here are a few examples. A couple’s answers their friends’ questions. A researcher’s or expert’s revblog answers questions from others in her field. A CEO’s revblog answers his company’s employees. Changing the parameters around may allow anybody to ask and answer (a forum) or only the author to ask (a traditional blog), but keeping in the spirit of “questions” and “answers.”

For instance, imagine this were a reversed blog where I come up with silly ideas and you tell me if they suck or not. Would that be a good idea?

Update: As pointed out in the comments, this idea has now been implemented by Stack Overflow and its brethren: Super User, Server Fault, and Math Overflow.

At first I thought an extension was the way to go, but its turns out extensions are aimed at tweaking Firefox’s interface, not fussing with the page. Greasemonkey is perfect for the job, since it basically just executes a script on every page that matches a given pattern. Plus, users don’t have to go through the hassle of downloading crap and restarting the application.

This screenshot shows link underlining, headers, and boldface (i.e., embiggening). I haven’t figured out how to handle styling as the page is edited, though, but once that’s out of the way the rest should just be a matter of regex hacking.

Here is a quick WikiSyntax demonstration.

The idea is to get a bunch of players to represent business owners, and they collude against each other to gain the upper hand in a diverse marketplace based around neighborhoods and cities.

An excerpt from the wiki, dealing with law enforcement:

Workers and players may perform illegal activities that attract the attention of law enforcement. For instance, a player may own a mafia syndicate that extorts protection monies from the nearby businesses of other players in exchange for not sending soldiers over to permanently silence the other players’ workers. This is just business. The victim in this case may either pay the protection or pay the cops to protect him by bribes or pushing through legislation against organized crime. The mafia boss, of course, may also bribe law enforcement to look the other way. This is the mechanism by which illegitimate (under the table) transactions are taxed.