On Websites & APIs

A few weeks back, I attended Startup Weekend in Israel. Startup Weekend is a gathering of people of all sorts – coders, designers, marketers and the like – that join forces for one intensive weekend to create something out of nothing. While most groups spent most of their time discussing business plans and polishing presentations (which was a disappointment for some of the more talented developers in the bunch), our team spent almost all of our time developing a new internet service.

What’s In A Service?

Developing a new service in 48 hours is not an simple task, especially for a group of ten people who have only just met and have widely varying skill sets. We wanted to take one commercial area, which we felt was badly served by existing sites, and revamping it. Creating, in two days, the open source seed that could later be used to take over the category. In Israel, the worst served segment is of classified ads, so that is what we were aiming for.

Here’s a list of what we had in mind:

  • Insert a new item (for example, a car or a cellphone)
  • Different displays and properties for each kind of classified ad (cars are different than cellphones)
  • Filter many items (according to properties of the item)
  • Search
  • User registration and login (via existing services, such as Facebook Connect)

To make things more difficult, we had ambitious goals about creating the front end of the service as well:

  • Develop not only a website, but also mobile applications (mainly, iPhone and Android).
  • Translate to several languages.
  • Test several, completely different,  design concepts and user interfaces. We really wanted to make a radically better website. Aside from the graphic design, we had several ideas about comments and Facebook integration that were pretty cool. We also had ideas about mixing UI concepts from price comparison sites as well as classified ads sites.
  • Use real ads, taken from competing sites (which, in Israel, is probably legal since classified ads are considered too utilitarian to be protected by copyright law).

The Open Website API

Building several clients to the service required extreme separation of responsibilities. In most existing frameworks (it’s slightly better in Ruby on Rails than in Django), are built on three layers: the data model, the controller or view and the template.

The data model defines what data objects are used in the system. We’ve built a simple generic model for a classified ad, and a set of meta-models that define the properties expected for each type of item.

The controller or view is responsible for fetching the data required for this action. For example, in order to create a new ad, I need the list of expected properties for this type of item (a car). The controller is responsible of all the heavy lifting (such as fetching data, validating input correctness, etc.). We created the controllers for our main actions, quite simply.

The template defines the way the page layout. It is responsible for rendering the way the website looks like (the HTML). This is what we found most cumbersome. First, each template is tied to a URL. This meant that we couldn’t have two HTML clients without splitting a lot of code. Then, the mobile client needed an API, which required a third branch of controllers and templates to render the objects for the iPhone.

This just wouldn’t do. Instead, we decided to create the API once, and use it for everything else. That means that on the server side we only had to develop the API, and not even a single HTML page. Then, we could have written as many clients as we wanted, each with a completely different flow, look and feel. The web clients were just html with some ajax, that didn’t have to be on the server. The iPhone app was just as simple to develop. No code was duplicated. I was surprised at how fast we have iterated ideas.

What’s it good for:

  • Complete API – by default, there’s an API for everything. There isn’t anything on the website that can’t be easily implemented on any other client. It makes the website very open for developers, without requiring any special treatment.
  • Complete separation of functionality from design – the server side was responsible for authentication, data validation and simple access paths to the data. The client is responsible for the flow, and user interface. Each client can be completely different, tailored for the device it is used on (think: web vs. mobile). There were hardly any limitations on what a client could do, because the API was so basic.
  • DRY code – write once, use everywhere
  • Third party friendly – having an API is important for another reason. Think of the Twitter API and how it helped create the Twitter ecosystem (something they are fighting today). For an open source website, this is an only an advantage. You want as many people plugging in and creating something new on top of it. Over time, the best ideas will merge, and community will benefit from the competition, while not wasting resources duplicating the data layer.

Cons:

  • Slower loading times – pre-rendered HTML will always be faster to load. Do people care that gmail takes a few seconds to load, every time you open it? Not really, because it’s so useful. And with smart client side caching and some clever ajax pre-loading, you can cut this time down significantly.
  • Command & Control issues – does the project include a client? If so, which one? How do you chose which client is “official”, or best? If not, does it mean you need two packages to install the website? How do you manage a list of clients? Where’s the data? Is it free and portable?
  • Security and tampering – when you have a very open API, you are vulnerable. There’s a fine line between being open, and being so open that you endanger the data integrity.

Rapid Development, Distributed Development

There are two special cases where this method can show itself to be especially useful.

The Lean Startup:

A startup in its most early stages is an organization trying to build an unknown solution for an unknown problem. It is a team of people, trying to find both a business problem and a solution to such a problem. This is called the product/market fit. The lean startup mentality dictates that the early stages of the startup should focus on learning.

Having an open website API is a great way to learn. First, it’s a great way for A/B testing and iteration of ideas. Second, there’s a real chance for serendipity. Your users will create clients for themselves, thus telling you what they need, and why they love your service.

The Open Source Website:

We’ve all heard about open source code projects, but an open source website is a much rarer creature. I’ve talked a bit about the reasons why it’s so difficult last year. The open website API liberates the project in several ways. The biggest pain point of the open source API is the data. A website without data is useless. By having the data in one central place, there’s great opportunity for innovation on the client side, having several open source clients developed with ease. In any other way, you wouldn’t be able to fork the website’s look and feel without copying all the data as well (think: Wikipedia).

I’d love to hear what you think. What other pros and cons are there? Would you want to see more open source websites?

Be Sociable, Share!

    Also read...