This is a partial transcription of the two keynotes from day 1 at the All Things Open conference in Raleigh, NC held on October 22nd and 23rd.
Keynote from Jeffrey Hammond, VP & Principal Analyst at Forrester Research
If you are a developer, I cannot think of a better time in the history of our industry to be a developer. It is a golden age if you write code. If you have people work for you that write code, there is another part of that message. You need to understand how open source is part of that process or you risk being consumed by generational changes happening in our industry.
When I built my first company in 1999, it cost $2.5M in infrastructure just to get started and another $2.5M in team costs to code, launch, manage, market, and sell our software. So, it's not surprising that typical "A rounds" of venture capital were $5-10M.
If you look at where we are today, ideas cost about 90% less than what they cost when I got started in this space. Today, omni-channel clients, deployed on elastic infrastructure, aggregate discrete services, use managed APIs, integrate open source software, employ devops techniques, and focus on measurable feedback.
Open source is so ubiquitous. Anyone want to go talk to a purchasing officer when they want to spin up another node? That is driving the adoption systemically. We're seeing the evolution of DevOps, and open source is a driving force for modern application development.
We ran a developer survey at Forrester; 700 developers across multiple countries. We asked: "What classes of open source software have you used in the last 12 months?"
- 41% open source databases
- 38% operating systems
- 34% web servers
When we look at developers that built cloud, mobile, or big data, the response rates are at near-unanimous levels:
- 93% cloud
- 92% mobile
- 78% big data
1 in 5 developers have not used open source software. Even the folks using Microsoft and Oracle are using open source software in some way.
In the past, we've seen servers being the big one. Now it is the open source databases. We're starting to see a shift in the DB domain. We've seen a tail off in app servers. As we see more modern development, they don't use app servers but change how they run a backend.
When we look at modern applications, we see differences from previous generation. We see APIs everywhere. Developers today look for services and APIs first. There are more services now to use and call asynchronous communications. For the last 12 years, we've been locked into a MVC (model, view, controller) world. Tightly coupled MVC architectures. That doesn't work as well now. You see more event-driven frameworks now. Many of the old frameworks are as well suited for today, and the new ones, are open source software.
Lightweight process communication frameworks such as Socket.io, Nginx, and Node.js are replacing previous open source software and tech. It isn't just open source versus proprietary, we're seeing open source versus open source, and substitution happening. (i.e. Subversion versus Git). In memory, databases are becoming more and more popular and have broad adoption. Elastic infrastructure is the norm. No more "max" licenses.
We are seeing an emerging dominance of sharded SQL or NoSQL databases for open source options in that space.
Behind modern apps, we see "modern engagement architecture." Systems that are built on top of systems, going to employees or customers. Web apps are building in a very different way; fourth tier architectures. In between, there is an aggregation tier, that gathers and ingests data in real-time, from the IoT (Internet of Things), and other sources, and predicts what is the next best step via context. After that, a delivery tier, with things like Amazon Web Services.
Engagement platforms such as Netflix, can be decomposed, you can see all the pieces—client tier, delivery tier, aggregation tier, and services tier—all with open source sprinkled throughout. The same is true of Evernote, Instagram, and Untappd... you can see the pervasive use of open source in modern application architectures.
Increasingly, that innovation is driven by open source software communities. Collaborative collectives set to drive innovation in the industry forward. Those of us looking to hire developers have to understand how to restructure to work with and build on top of the work of those collectives, to attract talent.
I've been asking the question: "Do you write code outside of your day job?" And, 70-75% of developers say "Yes." Some only a couple of hours. Some as much as 11-20+ hours a week off the clock coding. That desire to write code on your own time is driven by a number of motives—learning, starting a company—but those motivations are intrinsic, it makes the developer feel good and feel happy. 1 in 4 developers tell us they contribute to open source software as part of their own time. Those developers are some of the most talented and creative developers out there. If you think of development as a creative field, then you know it is widely distributed. If you are looking to hire talented developers, productive developers, there is a correlation with those that do FOSS (Free and open source software), and those you want in your organization.
There is such a lack of top-talent in our space, everyone is looking for folks who understand modern frameworks and NoSQL, and can build on top of the cloud. If you have those skills, and know how to use the frameworks to build apps, there is a very bright future ahead for you. In the US, we expect developer growth of over 28% until 2020. In 2014, they had a $92K average salary, and in 2013 it was $70K. This shows there is a demand/supply imbalance.
We are in a generational tech shift. Modern tech is different than client/server applications. We gotta understand how to use this tech, and elastic architectures that allow us to innovate cheaply. The cheapness of open source is a perfect fit for modern platforms. 4 out of 5 use open source, and it works.
Open source software projects drive the collaborative collectives, whether they hang on GitHub, or in Drupal, or in foundations like Eclipse or Apache. These are the centers of gravity of development moving forward into the next decade, and the center of gravity grows.
Talent is a seller's market, and we are in a golden age.
Keynote from Dwight Merriman, Executive Director and Co-founder of MongoDB
Hi everyone. I'm Dwight. I work at MongoDB, as mentioned, and I've been there from the start. I wrote a lot of code in the version 1.0 days. I wanted to talk this morning about something consistent with what Jeff was talking about.
Pre-object oriented programming. Pre-relational databases. It goes back a long way. I'm going to focus on the data layer. I do like that term "modern application" because we're not building the same things. It's not "we need a new version of inventory system" but an entirely new class of apps, B2B, and B2C that didn't exist before. The way we build them has completely changed. The schedule this morning, you see all the talks, and they are peppered throughout with the names of projects and products, and lots of open source things. One thing I was thinking about was how Jeff talked about elasticity of licenses. In my mind, it is a big deal for "granularity" of software. It is easy today to mash up third party code that is from completely separate products. You can imagine using a dozen third party components or more. It can become a very long list.
If it was closed source, it would be hard to have that many different things to go buy, evaluate, and develop. That granularity aspect is very real and a big part of how we write software today. We don't want one big monolithic system, we want to break up what we can. Different specialized pieces is what we want, and that isn't as easy if it's not open source.
In that context of modern apps, I wanna talk about data. "Big data" and "NoSQL"—these are weird and imprecise words. There is something big happening. We're in the midst of the biggest change in the data layer of IT (information technology) in 25 years. There is something big going on. Big data is a bunch of new technologies in the data layer that are happening right now; and unusually large amounts change. A boring but accurate way to describe this is, you can divide out the buckets. NoSQL, scalable and great for building modern apps. Hadoop, more on the analytics side.
All this is happening, and it is a big change. The types of apps we write is changing too. In regards to that, the new apps and usecases we work on, the shape of the data is different. The shape of the data. The data is unstructured, polymorphic. It isn't just tabular. Accounting data in the real world is tabular. 1-to-1 mapping. Real-world usecase data, is all varieties of shapes. Unstructured on one side, and complex, or plastic and evolving on the other. The new tools are great at dealing with this. JSON, and the document-oriented notion of the database, or message passing in that format. This is significant in my mind. Saying "unstructured" data is inaccurate. Usually it has structure, but it's dynamic. It is like the age of dynamic/static-typed programming languages. The data we are dealing with is dynamic, and I like to think of it that way.
I like to think about the size too. Massive scale. How to deal with that. Computers I have today are cheap, and not very tall. They scale horizontally. In 1999, we spent $100M on hardware... we were serving lots of ads, but the computers were 1000 times slower, and they cost more money back then. You could get a "big" computer back then, but they were just taller. Today, you can't get that much faster processor by buying a "bigger" computer. You gotta scale horizontally. Parallelism. That dovetails perfectly with cloud computing, and the imperative to scale out, and this is really the only way to do it.
The main thing I think about is latency. The notion of the application and use of nightly or daily reports in your mailbox... that is old-school at this point. People are used to real-time interactions in the services they use on their phones. As developers, as we build systems, we gotta start from a real-time mentality. I would tell my team, of the 100 apps we build, default to real-time. As a CIO, that may sound obvious, but that is a big deal! Think back to Computer Science education in the 1980s. You are supposed to default to "batch" in 1980, batch algorithm complexity was lower, and the way to go. Much faster than real-time. Today, things are so much faster, and can handle it.
If my delivery mechanism was the mail, or FedEx, then batch processing wouldn't matter. One of the properties of modern applications, and something we gotta do. Anytime I do a new app, I default to real-time, and not only when there is a reason not to. Many of the new tools, NoSQL, facilitate that too.
Shape, size, speed, approach
Approach is "how we write code," and that has changed a lot. It is no longer architect, code, design, test, deploy. There is just constant iteration. On the developer side, it is helpful. And for the business side, it is great to be able to say: "Is this what you want?" and it can be changed rapidly to converge on needs if it isn't. It has always been hard to spec things. This opens doors to creativity. Even if I could write a perfect spec, I can have ideas as soon as a month in! You get a better outcome. That is a big part of apps today.
Look at Facebook. Everyday the site changes. Daily. For years. Iteration. That is a great real-world example.
Applications
I'll give some specific examples. The usage of MongoDB, when I'm asked is "general purpose database" or "operational database." We are trying to take a new cut at what is an operational database in the context and methodology of today. IoT is going on, and this is one place where people use NoSQL, but people are doing lots of things. John Deere is doing some neat things with data these days.
One of John Deere's messages is: "Feeding the planet"
Imagine it is harvest time, and we're running the combine, and as we go around, you can measure yield. You can measure how many pounds of stuff went into the bin in the last 1-5 seconds. You know where you are by GPS. We can have a yield map, topologically, and very precise. Of course, that is a lot of data. What can we do? We can look at it, and visual representations of it, like heatmaps. You can see problems like "the river is over here." You can imagine a very granular—something the size of this podium—being low yield, and using point-by-point fertilizer! That is the kind of stuff they are working on, and these are real-world tangible things. Candy crush is "cool" but it is intangible. This isn't just in the mental domain.
One of Bosch's messages is: "Connecting The Planet"
They have a business unit that does lots of software for automation and industrial management. They have drills or riveters—a power tool like the one in your garage—except these cost $15K and put rivets in airplanes. Safety is critical in airplanes. There must be quality control. Are they going in right? We can look at it, rivet by rivet? You can see if there are three mediocre rivets in a foot, which is no good, but over 100 feet, that could be ok. The gun records all that data, and it becomes part of the quality control of the manufacturer. In these cases, you gotta store a lot of data. That is what makes this interesting, the shape of the data. The timestamps that have nuggets of info. The polymorphism. All the sensors, not all of them the same, and how well you can deal with those differences.
One of Edeva's messages is: "Safeguarding The Planet"
Think about self-driving cars, which generate tons of data. You can use that data to do analysis to improve traffic flow, and figure out where to change something, or add a road, and what is the problem. You can use it for safety analysis. They've built a technology when you are driving across a bridge in Sweden, they see your speed. If you are going the speed limit, it does nothing. If you are going too fast, the system creates speedbumps just for you! It creates depressions in the road! If there is an accident on the bridge, you can ruin traffic for half-a-million people in a day. It has been a very successful project for them. I have mixed feelings about not being able to speed... but that is good for them.
(audience laughter)
Looking at these examples, you can get broader than that, but the question I would put out is: "What are you working on?" Maybe not this second, but over the next year. Are there intersections, or creative things? We've seen great things come from startups, like Uber. But I've also give you some examples here from more "classical" organizations. There is more to do in the "old" domains and fields, and we have the tools now to adopt the new mentality. Be ambitious, try to tackle these things.
Comments are closed.