Automated Testing with MongoDB and OffScale

Last friday, I attended MongoSF 2012 as a speaker. It was a great event, with about a thousand programmers showing up, and some fantastic and deeply technical talks about advanced topics in MongoDB. And in the midst of all that, 10gen were gracious enough to let me have my own thirty minute slot at the EcoSystem Track to talk about OffScale, and how it can help with automated testing.

As a first time speaker, it was a challenging and rewarding experience. It forced me to step out of my comfort zone and practice my public speaking skills. For OffScale it was a great opportunity to get in touch with more programmers and hear some feedback. As we are in beta, and working on improving our product, listening to people’s questions is sometimes more important than having good answers, as this helps us cristalize our priorities. To this, I want to give out special thanks to the MongoLab guys that attended the talk and asked some great questions about use cases we will definitely address in the future. This was exactly why I wanted to speak.

Here are the slides for the talk I gave. I’ll probably go into more details in future posts, but the basic theme is: do more tests, use real data, make your database a first class citizen in your environment & have lots of jokes in the slides if you are not a funny person.

Go download our beta and try it out.

Schema Management Under Source Control

A long time ago, I’ve written a post for Kaltura, the best video management platform out there, where I worked at as a data warehouse consultant and lead developer. I’ve spend a lot of my time there scaling the system as the company grew, writing ETL processes and code, migrating large databases from one schema to another, and building tools and processes to help make testing and deployment easier for such large databases.

My good friend, Roni Cohen, community manager extraordinaire, has brought to my attention that the post has finally been published:

Schema Management Under Source Control

Deploying new features that require changes to the database is a technical challenge in any application. To quote Jeff Atwood from Coding Horror: “When I ask development teams whether their database is under version control, I usually get blank stares.”

The database is a critical part of your application. If you deploy version 2.0 of your application against version 1.0 of your database, what do you get? A broken application. And that’s why your database should always be under source control right next to your application code. You deploy the app, and you deploy the database. Like peanut butter and chocolate, they are two great tastes that taste great together.

Not coincidentally, it is very relevant to what OffScale is doing today, trying to make the database an integral part of the development cycle and a first class citizen in a source controlled environment.

Learning Ruby, Using Koans

For the longest time I wanted to learn Ruby on Rails. It has been bugging me ever since I’ve built my first website, some five or six years ago, that I have failed to find the time to learn the leading web development framework. I’m familiar with RoR from afar, watching some of DHH’s lectures and reading some texts, but didn’t get to actually try it out.

Why I Chose Django Over RoR

At the time, I looked into two frameworks in languages that were then new to me: Django, the python web framework, and RoR. Since I haven’t worked with either, and haven’t written any code with any of these languages, I had a hard time choosing between them.

At the end, I had a table with pros and cons that looked something like this:

Ruby on Rails:
+ Massive adoption
+ Proven & more mature
– No accessible tutorials
– Ruby

Only to discover Ruby on Rails is meant to be magically simple to use, which means there is a lot to learn about conventions and things that happen in the background that really aren’t well defined in the API. Why does Cat turn into Cats automatically? In 2006-7, the tutorials for RoR were abysmal. And trying to learn Ruby wasn’t so easy. What’s up with this little tidbit of code?

$ 1.+(1)

I’m out. This freaks me out.

+ Best documentation ever
+ Python
+ Less magic
– Not even at 1.0

Django was in beta. That’s a killer, isn’t it? Well, the Django community seemed to have it together with great documentation and easy on boarding. The 0.97 version seemed fairly stable. And I was surrounded by python-loving hippies (no offence, guys).

Also, seriously, compare how nice Jacob Kaplan Moss to David Heinemeier Hansson:



Django, I love you, but you are bringing me down

I slowly lost touch with the Django community. Most of my professional life I’ve been the data and databases guy, so I have less stake in web development. I’ve worked on back-end analytic services in some largish PHP (bah!) company. I’ve worked on my startup. I haven’t developed any web site for a while now.

From the sidelines, the size of the community seems to matter. Look at how MongoDB is killing it compared to Couchbase. They don’t do it by having a better product. They do it by having a larger community. They do it by integrating well with Ruby on Rails and others.

RoR is the bigger community, and thus it is more vibrant and moves faster than it’s python counterpart.

Learning Ruby, Take II

I haven’t learned any new language for too long. I’ve only messed with Scala a little in 2009, failing to find a good project to really dig into it. I’ve messed with JS a little in 2010. And some PHP (bah!) in 2011. 2012 belongs to Ruby.

And how the community changed!

I’m going through the Ruby Koans to get used to the feeling of the language. Unit tests are such a perfect learning tool. I’m stumped at how easy and fun it is to learn this way!

Go ahead, give it a try.

I’ll then go more deeply into Rails, and come back to tell you what I think. I still don’t like all the magic they’ve stuck inside, but I’m willing to suspend my discomfort for a later stage, until I can have a more educated opinion.

On and away.

One More Time, From The Top

Last time I’ve demonstrated a way to create mosaics from images. I wanted to do a little bit of image processing using javascript and HTML5’s new canvas abilities. As I mentioned, this was only the first of an idea I had about turning social experience into a beautiful, expressive and personal graphic.

Mosaics, Revisited

Canvas is an extremely powerful concept, allowing web developers to create more expressive web pages. The web will soon be filled with rich user experience and we will be dancing like it’s 1995. This has been a long haul, but we are finally WINNING. I just hope we have better designers on the web than we did on the desktop. I never (ever!) want to see the Word 6.0 toolbar sprawl on a web page.

My favorite web sites focus on content. They try to let everything else blend into the background. On the best sites content is king. Google, YouTube and Facebook all use a clean, white background, but there are other examples for beautiful design. TheSixtyOne is both artfully designed, while the functionality just melts away into an incredible user experience.

So, I took the mosaic idea one step further, creating a (very simple) tiling jQuery plugin that allows to recreate a large image from small images. Here’s a demo.

Continue reading →

Messing Around With Images

The Side Dish

If you know me at all, you know that I always have a side project. I mean, I love the main course as it is full of challenges, but sometimes I need to do something else. I need a Gedankenvacation.

Dude, you may ask, why do your vacations include hours of code in front of a small screen? Why not go out, get some fresh air and enjoy the sun? That’s actually a good question. Maybe it’s because I love to code. Maybe I’m just a bit lazy. Maybe I’m a vampire. Who knows?

All I know is that when I work too hard on one problem, I can tap out of resources pretty quickly and then I turn into a Facebook Zombie. Learning a new skill or working on a different problem usually helps me get out of the rut, and allows me to go on for much longer stretches.

My side projects usually have some sort of visual or artistic nature. I’m hardly any good at it, but I love doing it, and I don’t mind failing. One year I have written detective stories in a blog format (think of how a blog of a private investigator would look like). One year I’ve founded (and terminated) an online T-shirt store. One year I have worked on a massive multi-player online game (that we abandoned unfinished). And one year I helped organize reality-hacking events, where I got to meet interesting people and edit lots of videos. This year I want to work on a combination of visualization and social themes, simply because it’s the complete opposite of what I’m working on during my “day job”.

Messing Around With Images

It’s been almost a decade since I last did anything related to image processing. I was in the second year of my computer science degree, and I was looking for any practical course I could find. I never felt engaged by highly abstract mathematical courses such as Computational Complexity Theory. Call me a heretic, but in my mind Turing machines aren’t very practical mental models for computers, and I don’t understand why they are still being taught.

Anyway, digital image processing was exceptionally suited for my practical needs. First, you could actually see the results of your work. Second, it included just enough math so it wasn’t completely technical and boring (unlike, unfortunately, most of the programming tasks in Data Structures) and just enough code as to be useful.  Third, did I mention you could actually see results with your own eyes?

Continue reading →