What owning your personal cloud means for the open source movement

No readers like this yet.
A person holding on to clouds that look like balloons


Sandstorm.io is a radically easier way to run personal instances of web apps. It allows you to have your own personal server and install apps to it through an app store interface as easily as you would install apps on a phone. Here at Sandstorm, we talk a lot about the need for privacy, security, and control of your cloud data. Let me let you in on a secret: these aren't the reasons for Sandstorm, our personal cloud platform (currently accepting funding on Indiegogo.) They are pleasant side effects.

The real motivation for Sandstorm is, and always has been, making it possible for open source and indie developers to build successful web apps.

In today's popular software-as-a-service model, indie development simply is not viable. People do it anyway, but their software is not accessible to the masses. In order for low-budget software to succeed, and in order for open source to make any sense at all, users must be able to run their own instances of the software, at no cost to the developer. We've always had that on desktop and mobile. When it comes to server-side apps, hosting must be decentralized.

But today, personal hosting is only accessible to those with the time, money, and expertise necessary to maintain a server. Even most techies don't bother, because it's a pain. Sandstorm exists to fix that, making personal hosting easily accessible to everyone.

"The only solution is to make sure everyone has a server where they can install any software they want."

Open source has worked on the desktop and mobile

On my desktop, I run Debian Linux. My system is composed of several thousand packages. I have browsers, text editors, IDEs, chat clients, office suites, development tools, photo editors, and media players. Remarkably, every single one of these is open source. Even more remarkably, I can't remember the last time I felt any need to use a non-open-source desktop app.

I'm no zealot. I don't impose open source on myself, nor would I do so on others. I'll use proprietary software when it gets the job done, and I have spent plenty of time as a Windows user and as a Mac user in my past, but these days I'm simply happiest on a Linux system. That's my personal preference and it's not for everyone, but the fact that this choice is available to me and that I can run an all-open-source desktop without pain is pretty amazing considering how things looked fifteen years ago.

Even Windows and Mac users these days use lots of open source software. By some measures, a majority of people now use an open source browser. VLC, BitTorrent, and other "indie" open source desktop apps are widely used even among non-technical people. Mobile seems even more full of open source and low-budget indie apps, as the various mobile app stores make it extremely easy for small developers to reach large audiences.

Yet, somehow, the web today is nearly completely devoid of open source software. Every day I use apps like Gmail, Facebook, Twitter, Feedly, and others. None of these are open source. Granted, these apps often run on open source infrastructure, but that's different. Most proprietary desktop apps use open source components and tools. But web apps, as the users see them, are almost invariably proprietary.

Why are all my web apps proprietary?

Open source web apps exist. For example, webmail apps like SquirrelMail and RoundCube have been around for a while. If you look hard enough, you can find open source online document editors, RSS readers, and even a few social networks. But even among techies, hardly anyone seems to use these, probably because they all require running your own server, and few people have the time, patience, and expertise for that.

There are a few success stories. WordPress is open source and widely-used for blogging. But this seems to be the exception rather than the rule. And it's questionable at that: most people who use WordPress are not actually able to edit the code, because they are using it through a WordPress hosting service and can only use the version of WordPress that that host provides. Naturally, all hosts will likely only use the "official" version. So in practice, it's almost not really "open source" but "visible source"—you can see the code and request changes, but you only get to use your changes if they're officially adopted.

Why does no one use "indie" web apps?

Even on Windows, people commonly install little open source apps to get things done. Need to tag some mp3s? Want to connect to multiple chat networks with one client? Need to unpack a weird archive format? You'll probably use an open source app. Sometimes it's hard to build a business case around a niche purpose, yet little apps written by random people in their spare time are abundant. But on the web, it doesn't seem to work this way. Any significant service with a server-side component can really only be run by a funded corporation.

Let me describe a case in point: I know a certain prolific coder by the name of Brad Fitzpatrick. You may know him as the author of LiveJournal, Camlistore, memcached, OpenID, and other things. But I want to talk about a project of his you probably haven't heard of.

scanningcabinet is a little web app that helps you organize your paper mail. You drop your mail into your scanner, and the app scans it and uploads it to "the cloud", where you can access, label, and search it later. Brad wrote this on a weekend several years ago and threw the code up on GitHub.

This app could be useful to just about anyone. But sadly, no one can really use it. To set it up, you have to configure a server (App Engine, in this case) and deploy the code to it. Even for someone like me, who knows how to do that, it's not something I really want to do.

By today's model, if Brad wanted to make this app accessible to the masses, he'd have to run it as a service. He'd have to build in multi-user support, make sure it's secure, deploy it, and monitor it. Worse, he'd have to pay for it, which means he'd have to monetize it, which probably means he'd have to start analyzing people's mail to build advertising profiles, or set up billing. Brad obviously doesn't want to do any of that.

And even if he did, who would use it? Do you want to upload your paper mail to servers run by some guy on the Internet? I'm certainly not going to trust my personal data to any service that isn't at least backed by an identifiable organization with something to lose if it screws up.

The problem is hosting.

By this point the problem is becoming clear: for open source software to make sense, the user has to be running their own instance. Software-as-a-Service and open source just don't make sense together. It's not really open source if you can't run modified code, and the high barrier to entry shuts out hobby projects or anything unwilling to be monetized.

The only solution is to make sure everyone has a server where they can install any software they want. They don't necessarily have to administer that server—it could be run by a friend, or a service—but each user must be able to install arbitrary software. And that software must be securely sandboxed to prevent buggy or malicious software from harming the rest of the server.

Today, this doesn't exist in any practical form. Servers require time and technical expertise to set up, while turnkey hosting services only allow you to run a fixed set of software.

There is no place for open source web apps to run.

We're making decentralized hosting viable

Sandstorm is a web app hosting platform that enables non-technical end users to install and run arbitrary software. Apps may be downloaded from an app store and installed with one click, like installing apps on your phone. Each app server runs in a secure sandbox, where it cannot interfere with other apps without permission.

We talk a lot about privacy, security, and control, but to me these have always been pleasant side-effects of Sandstorm's model. My main motivation for starting this project has always been to enable open source software, hobby projects, niche applications, and indie developers. Even if each individual app in this category ultimately has a small impact compared to Gmail or Facebook, the collective value lost by not giving these apps a platform is enormous. We need open source software to fill the niches that big companies aren't interested in and to push the boundaries they find too risky. We need software that can be tweaked without permission to try new things without starting from scratch. It honestly seems absurd to me that we don't really have this on the web today.

Help us get there

We've already come a long way. We have a demo that does most of what we're talking about already. But we're coming up on the limits of what we can self-fund. We can get Sandstorm into production, but we need your help.

Please check out our campaign on Indiegogo, and spread the word.

Originally posted on the Sandstorm news blog. Reposted with permission.

User profile image.
Kenton Varda | I write open source software. I worked for Google for 7.5 years, during which I worked on infrastructure and security but was best-known for open sourcing Protocol Buffers, Google's internal data interchange protocol.


I tried the demo, and is it ever slick. Sandstorm seems like an elegant solution to a pressing (as you compellingly note here) problem. I wish you the best as you move forward with it.

This is very slick indeed! In less than a minute I installed mailpile app on the demo instance and sent mail successfully

In reply to by bbehrens


I just had a quick poke at your shell, and your ghost demos. Seems to work, whch is a good start.

A few technical points:

* Your demo landing page infinitely loops without cookies, and the landing page requires javascript for no clear reason.
* It looks like you are tied to google's ajax code, so if they yank that, yes your site goes down.

From a more social perspective, I'm not certain that your solution really solves anything you claim to solve. Having servers that users can use doesn't overcome the network effect that gmail and others leverage. Nor does it solve the problem of having to supply a domain in the first place (which is enough of a hassle for anyone). I don't see the clear path from the problem you are stating to the solution you have developed.

IMHO, a good distributed networking library, with a hosted backup (a 'la trackers in torrents) would have been a clearer solution to the issues you describe (everyone loves the feel of native code on their phones/computers/whatever). Better scalability, and no need to invoke domain names.

On the other hand, this seems like a serious attempt at online hosting. If you can clarify why you are doing it, this would be great!

> Your demo landing page infinitely loops without cookies,

Sorry, I think that must be CloudFlare's doing, as I don't actually use cookies myself but I know CloudFlare introduces one. But I would think it that case that all CloudFlare-enabled sites would be affected, which is a pretty significant chunk of the internet. Do you find a lot of sites have this problem?

Or did you also disable localStorage along with cookies? Sandstorm does in fact use that.

> and the landing page requires javascript for no clear reason.

The landing site is part of a Meteor app. Meteor apps are pretty much entirely Javascript. The design makes it possible to browse between pages of the app without waiting for slow page reloads; apps that avoid Javascript feel awfully slow and dated these days.

> It looks like you are tied to google's ajax code, so if they yank that, yes your site goes down.

Huh? Sandstorm does not rely on any external resources to my knowledge.

The Sandstorm web site (not Sandstorm itself) uses analytics and social buttons but they are all optional; it still works if they fail.

> Having servers that users can use doesn't overcome the network effect

Not all apps have a network effect. E.g. a document editor doesn't have much network effect. I would argue e-mail apps don't have much network effect either, since e-mail is an open standard. Obviously network effect is still a problem for federated social networks, but at least by removing a separate major problem (hosting) we are getting closer.

> IMHO, a good distributed networking library...

No. Native clients are great (and Sandstorm will support them) but for anything social or collaborative you still need a server. E.g. in order to have a real-time collaborative document editor, you need a server that hosts the canonical copy and mediates edits.

Moreover, what you describe would require rewriting all our apps. Being able to seed Sandstorm with existing apps is really important for bootstrapping.

In reply to by amoose (not verified)

This is definitely an interesting project. It's nice to see a one-click installer that contains 1) only FLOSS, and 2) applications that I'd actually use.

Best of luck with your Indiegogo campaign!

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.