Solving decentralized discovery

If you ask me, the most interesting problem in decentralization right now is discovery — that is, the challenge of efficiently connecting users to resources when they're not quite sure what they're looking for.

If you want to create value, I suspect it's also one of the most important problems to solve. As a volume of data grows, the opportunity tends to lie in helping people find things — and there are an awful lot of software applications in which how you find the data is arguably just as important as where you store it — which means that merely distributing the data doesn't always correlate with distributing the power. One obvious example of this phenomenon is ride hailing. For cab drivers, there's simply no benefit to storing your contact info to a blockchain if there's no mechanism to help local people find it when they need a ride to the airport. Marketplace applications like Uber are just one example of the critical role that discovery plays in building a less centralized future.

The decentralization hype machine is largely ignorant to this idea. A staggering number of bloggers and Twitter celebrities who opine about DAOs erroneously reduce the marketplace problem to one that can be solved by smart contracts alone — imagining that Uber's only role as middleman is making sure that both passenger and driver agree on the fare, and grossly ignoring the question of how the two parties get introduced in the first place. This misunderstanding seems to result from a failure to grasp the complexity of most discovery problems — particularly with the added restriction that such marketplace apps are likely to run on resource-constrained mobile devices and suffer from intermittent connectivity issues. The path to a technologically decentralized driver cooperative is probably paved with advances in offline discovery, not advances in blockchain transaction speed.

Given the ubiquity of discovery problems, I often wonder if there will soon emerge a "discovery protocol" specialization among decentralization enthusiasts. Discovery problems often present highly overlapping use cases — that is, a single discovery problem is typically shared by a number of application types — which means that a generalized discovery protocol, decoupled from both infrastructure and application, could solve the problem for multiple apps simultaneously. One example of this principle in action is that of the geographic discovery function employed by Uber. The problem of discovering data based on some notion of geographic proximity is often solved by conducting range queries over a hierarchical database of latitude and longitude information, typically after it's been remapped to one dimension using a space filling curve. A composable protocol which solves this search problem in a decentralized way — with all the desirable fault tolerances, fully independent of any underlying storage mechanism — could also power a decentralized Tinder, a decentralized DoorDash, and a decentralized Airbnb. Protocol engineers could focus their energy on important new problems rather than duplicating each other's work.

Since discovery overlaps with search, the underlying computer science trends in the opposite direction of blockchains. (If your goal is to make data searchable, you can't really pick a worse solution than a blockchain.) Accordingly, developing next generation discovery protocols is probably a task for an adjacent set of programmers who bring a somewhat adjacent collection of knowledge. The academic literature in distributed data structures tends to formalize discovery as a distinct research practice, but the activity among hacker types hasn't caught up yet. I think it will soon, because it's an underexplored opportunity.