Get the highlights in your inbox every week.
Exploring the open development method
Scrum is dead: Breaking down the new open development method
Image by :
Opensource.com
"Snake oil salesman" is a phrase that conjures up images of stuffy old men with bowler hats, bushy mustaches, and tailored pinstripe suits peddling bottles filled with "cure-all" liquids. Although prevalent a few centuries ago, the snake oil profession is now lost to the annals of history. These days the snake oil description has expanded beyond human health care. In the tech world, for example, a snake oil solution can be used to describe anti-virus software or a favorite software development process.
Among the most "oversold as a cure" methodologies introduced to business development teams today is Scrum, which is one of several agile approaches to software development and introduced as a way to streamline the process. Scrum has become something of an intractable method, complete with its own holy text, the Manifesto for Agile Software Development, and daily devotions (a.k.a., Scrum meetings).
Although Scrum may have made more sense when it was being developed in the early '90s, much has changed over the years. Startups and businesses have work forces spread over many countries and time zones, making sharing offices more difficult for employees. As our workforce world evolves, our software development methods should evolve, too.
Open technologies today
Startups and businesses are being founded on the ideals of open technologies, and even more established corporate entities, including Microsoft, are embracing all things open. Matt Asay does a good job of describing the current state of affairs:
Ten years ago, a new open source company or project was news. Not anymore. Open source dominates mobile, with Android displacing the seemingly unbeatable iOS in both smartphones and tablets. Open source also dominates cloud, with every significant cloud platform except Azure built using open source. And even Azure treats open source technologies as first-class citizens on its platform. And open source dominates Big Data, with Hadoop and NoSQL technologies the major forces used for managing the world's data explosion.
Open source in 2015 is ubiquitous. Potentially every person benefiting from modern technology is also benefiting from open source solutions in some fashion. When you consider the international pool of developers contributing to the world's most popular open source projects, you have to wonder how they've been able to succeed without the benefit of managers, meetings, and code sprints. When I began researching how open source projects have succeeded, I realized they share a set of principles that could be considered the tenets of the open development method.
The tenets of the open development method
Code quality
As part of an open development method, code quality is king. You should be asking key questions every time you write code:
- Is this code legible?
- Is this code testable?
- Is this code modular?
- Is this code economical?
Every question asked benefits not only you, but your team. When you write code in a such a way that another developer half a world away can sit down and start working on it immediately, without needing to ask any questions, you're helping improve your team's efficiency. Likewise, when you ensure your code is testable, you drastically cut down on the number of roadblocks your team may encounter. With modularity, you present code to your team that is both easily maintained and potentially recyclable for another project. And finally, economical code can save everyone—from your team and future contributors, to clients and end-users—both time and money.
Documentation
Developers do not necessarily enjoy documenting their code—that is no secret. Moreover, when they are subjected to the emergency room processes of a system such as Scrum, even if they wanted to document their code, chances are they will not have the time because priorities determined by the business often take precedence. The most significant piece of documentation you will write is your high-quality code, but that is not enough. Consider those who do not know the code, and those who have neither the time nor the desire to read it all. When your team focuses on documenting your projects appropriately, you are doing for your code the same thing that the popular TV show How It's Made does for its viewers by explaining away the complexities of your project and bringing it to a wider audience in an easier-to-digest fashion.
Testing
On a geographically diverse team in which you may not be able to receive help in real time, taking a test-first approach with code allows you to work with greater independence and fewer roadblocks. With testable code there are many benefits to consider. Let's take a look at Tim King's list of 12 tips from the Perl Shop:
- Unit tests prove that your code actually works.
- You get a low-level regression-test suite.
- You can improve the design without breaking it.
- It's more fun to code with them than without.
- They demonstrate concrete progress.
- Unit tests are a form of sample code.
- It forces you to plan before you code.
- It reduces the cost of bugs.
- It's even better than code inspections.
- It virtually eliminates coder's block.
- Unit tests make better designs.
- It's faster than writing code without tests.
Discussions
Everyone should have a say when it comes to discussing your project and direction. Never discount the marketing coordinator, the account manager, and non-coding designers. Everything is open to discussion, and when everyone is on has access to one another without needing to go through a chain of command, some of the most ingenious ideas and solutions can be discovered.
Although a veteran developer may dominate discussions due to their expertise, never disregard the new developer who has different, fresh ideas. The TODO Group Open Source Code of Conduct outlines general expectations of the community of a project that we can also consider when participating in team discussions:
- Be friendly and patient.
- Be welcoming.
- Be considerate.
- Be respectful.
- Be careful in the words that we choose.
- Try to understand why we disagree.
Transparency
Engendering user trust, promoting engagement with the community, and improving the security and stability of your project are traditionally considered separate goals on a roadmap for a project's success. But when your team embraces the open development method of transparency, all three goals can be achieved simultaneously.
From releasing a project's source code to making public the issue tracker and providing insights into internal processes and communications, the passion behind a project is apparent to anyone who comes in contact with it. When users and developers have access to these resources, your team will not only find many willing contributors, but greater respect and acceptance from the community. Transparency is a powerful marketing and working tool.
Asynchronicity
With startups, businesses, and open source projects distributing the workload to developers around the world, maintaining a certain level of synchronicity that a software development process like Scrum expects becomes difficult. For some groups, maintaining a daily Scrum meeting is unrealistic, and when you have five developers in five different countries, you can discount the possibility of paired programming. The software world today thrives on asynchronicity, and teams following the open development method can, too.
If someone has a question, he or she may send an email or post on a developer board with the expectation that their message will not get responded to for a while. Does that questioning developer stop their work? Not necessarily. With the need to be self-reliant and adaptable, developers can modify their expectations, focus on code quality, enhance their documentation, and improve their skills.
Democracy
In addition to open discussions and sharing ideas within a project community, community members should be involved with the decision-making process. When decisions get made by closed-door board meetings, executives, or managers (what I would call single points of failure), it is usually with a narrow and limited point of view. When everyone is involved in the decision process, including the end-user or client, potential issues that would otherwise fail to get identified have a greater chance of being resolved before they become problems.
Building the open development method together
Rather than allow only myself or a small number of people to develop this methodology, I believe the process should be as open and inclusive as any other open source project.
There may be several priorities that I have missed or points that need to be smoothed out, so please join me on GitHub, where I am continuing the discussion and working on creating a beautiful set of guidelines that will enable all development teams of all sizes to succeed.
About the author
49 Comments
this is interesting, but I am failing to see where any of what you describe is incompatible with scrum.
Hello Ahmad. Thank you for writing this article. My company practices Agile in a relatively strict manner (ironic, isn't it?),, but I find myself leaning toward more of your open source methodologies in my day-to-day operations and team interactions. As a traditional Agile Product Owner with development teams spread out on three continents across 15 time zones, sticking to traditional Agile methodologies is extremely challenging.
Do you have any insight how to bridge the gap between traditional Agile and open-source methodologies, to try to gain the best of both worlds?
Hi, snake-oil salesman here.
I'm not sure how this landed in my news feed and there's probably a whole lot of context I'm missing, but alas, I thought I chime in.
You're making some rather large statements. I do appreciate that you don't have all the time in the world to refine your argument. So let me also address it very broadly.
I'll agree that there are a lot of bad implementations of Scrum out there. Much like there are tons of projects that don't respect "the tenets of the open development method".
But that's beside the point. You're comparing a methodology that focuses on customer value to one that focus on code quality. Again, I don't have enough context into the open source world (this is my first time in this site, actually), but my inner-Freud tells me that throughout your many projects you might have seen tons of Agile Coaches. Mostly bad ones, it sounds like. That might have made you bitter towards Agile.
Scrum is not the silver bullet for anything and there are certainly environments where it doesn't apply. But I would like to ask you to reconsider your opinion on Scrum. Maybe you've got some friends colleagues who have worked on a 'good' Scrum team?
I would have to agree with the author about the snake-oil salesman comment. I have worked two places now where someone from the outside came in preaching about all the benefits of Scrum. They never stopped to see if it was really a good fit for the environment (not open Source). Management sucked up the snake oil. Sprint after Sprint, Stories were not fully finished. Users not available to test. Stories to big to do in one sprint. Not enough stories in sprint. Too many outside interruptions. And so on and so on. Management remains on the snake oil and cannot see that it is not working.
Yes, I do believe that Scrum does work in the right environments, but I have yet to see it.
Usually Waterfall is a better bet when done as originally designed (iterative).
I could not agree more.
I have been in IT a lot of years and have seen and heard a lot of wild claims that things like Agile/Scrum are the ultimate solution, but my experience is that the claims always fall short of reality.
If you ask a customer they will always opt for the cheapest (no insurance) option, however when everything goes wrong, they want all the documentation and five-start support, so lettiing them choose is a huge risk.
The other problem is that you empower people by trusting them, and not by bullying them by standing a circle like bullies in a playground.
An interesting rebut, which hopefully could have response from the author too to further enrich the discussion.
As one of my colleagues has pointed out, this article on improvements and silver bullets is relevant. http://seir.sei.cmu.edu/sheard/Life%20Cycle%20Silver%20Bullet.pdf It's easy to blame the method for its bad implementations, but that does not get you far in the long run.
I also don't see the how having best practices in software development is contradictions running scrum in your team.
If you read the scrum manifesto, it's all about having short feedback cycle while you are writing your software. How you write and what practices you choose is up to you!
Except your subtitle lists Scrum as "dead" which would make Open Dev contradictive. If I currently employ a hybrid Scrum approach successfully in all my projects, I would not list Scrum as dead. To be clear, I do not hold to a strict adherence nor do I expect any methodology to be so prescriptive that I cannot flex the art of software management. This is software after all. Strict-anything is almost a sure indicator something will come up short.
So I think many of these sour grapes with Scrum is due to process for the sake of a process or inexperienced operators (or as some have said, application when not a good fit for the dynamic).
I agree with almost every positive statement, and disagree with most of the negative comments about Scrum. Code Quality, testing, some documentation, transparency, flat structures, equal airtime for all team members, all these are fundamental to building a good product, and even though you might see them as a response to Scrum, those were all things covered 3y ago in my Scrum Master course. Don't see in any of these things any novelty, or any opposition to Scrum. Closing off the list of things we agree on, there's the fact that there are a lot of bad things being done around the world in bastardizations of scrum, trying to force in some elements of scrum, typically just the morning stand-up into full waterfall PMOs and even getting that one thing wrong, and still claim to be doing Scrum. Now, to the things I don't agree with you, apart from all your other criticism of Scrum, but that should be obvious by now. The way I read your paragraph on asynchrony, it reminds me of the days when IT thought they were so important, the business couldn't survive without them. Those were the days when projects would run for months without delivering a single feature while groups of developers had intellectual debates about grand frameworks that would save weeks of effort on an hypothetical future. That's why there's a product owner. If the developer needs time to think, wait for a reply, rethink some bigger design, that's fine. But it can't be an excuse to go doing whatever they want. That is discussed with the customer, or with the product owner, and with the development team. And the product owner, if relevant, should discuss it with the other stakeholders,either at the sprint review, or whenever he hears of the problem. That's how all the marketing managers and all others get their input in the product. At reviews or via the PO. Open source communities and startups can pivot the product's purpose, When the team has a specific problem to solve, someone needs to remind the team that there's a mission to,accomplish. It's up to the entire product team to decide how to build the product, but it's up to the Product Owner to decide what to build.A daily planning session, at the beginning of the day, ir at the end of the previous day, is a good practice in every domain. Construction workers have been doing toolbox talks for decades. If the team is split, is good to do follow the sun style handovers, but in each time zone it doesn't work to communicate and plan the day ahead. Not to be confused with reporting and finger pointing public executions some teams do
This is turning into its own blog post (srsly: http://zxq9.com/archives/1110) so here are the cliff's notes since I won't be done revising it for a while:
Scrum/agile was developed in the context of code written for a client that is expert in the problem domain but ignorant of software by a development team inexpert in the problem domain but expert in software.
Most foss projects are infrastructure, utilities and middleware where the end-users are usually tech people (and often the developers themselves are the initial users). Scrum can never fit here.
Software development spans several different worlds, and none of them really fit together. Embedded, web, mobile apps, application backend services, runtimes, client application code, single-release games, rolling-release/content games, etc. None of these share the same set of needs, so obviously there is no one-size-fits-all methodology that can ever hope to work.
We have to select methodologies and evolve our own team-sized development workflows based on the nature and context of the projects we are working on -- in the same way that we (really should) select our development tools based on the nature of the job they are intended to do.
Making anything religion is stupid, be it a development methodology, TDD, or even a particular licensing strategy (though honestly so far I haven't run into a case where releasing closed source actually rendered any benefits -- but there are certainly cases where the GPL isn't the right fit).
Oh here we go, the first reply being from a defender of the faith touting the usual excuse, "your not doing it right."
Agile and scrum is crap. I have personally watched it wreck the former team environment in my own company when the new owners shoved agile dogma up our asses.
Agile development is just business theory forcibly applied to software development. And like most MBA theories, it does not work.
Thank you for this article. While I do not agree that “scrum is dead” I welcome every opinion which questions scrum as best option for each and everyone. Scrum once proposed as a certain agile methodology became a religion to may people. Instead of asking themselves “does it work for us?” they ask “Is it proper scrum implementation?”. They put equality sign between being agile and implementing scrum, making anyone who does not follow their religion “not agile” (read: “bad”). Your observations are very interesting. Actually asynchronous development team, space and time distance – these are real challenges we face more and more frequently in a software development – not necessarily even in open source. I would also add specialization – which renders paradigm that “any team member takes any task from sprint” completely invalid, and puts question mark on the whole idea of story points and team velocity. I agree with you that perhaps it is time to start thinking of a better development methodology for today's world. BTW – you may be interested in visiting this website: http://growsmethod.com/.
Seems to me like you state your requirements for a Definition of Done.
You are basing your article and your comment on wrong assumptions. And the Agile Consultants did not point out your findings because they are partly not true! For instance,
Scrum is not a process for project management.
Scrum does not define " a single piece of working software operating on a single environment "
Scrum is not just a process, the process in scrum is just the most obvious part
Scrum 2.0 is something that is already iteratively and incrementally happening all the time. The Scrum Guide is being renewed quite frequently.
It seems that you should get more deeply into scrum before you try to criticise things that are just not true!
Arguments aside Scrum it's a process and Agile started as projement management methodology. It was adopted as software methods later.
I feel a bit mislead by the title of this article. It starts out by talking about how "scrum is dead" and we should "embrace the new...model" (Yes, "open" left out purposely). After reading a few paragraphs into the "embrace the new..." part, I realized that this article is really about how today's business embraces open source. It has very little, if anything, to do with how scrum is dead. What gives!?
I'm finding the comments more useful and entertaining than the original article. Full disclosure, I'm neither an Agile fanboy or Agile detractor and my team has experimented with many different development models. The best comment by far was from the dude who pointed out that open source development and Agile development should be used on different problems. Agile seems (to me) best suited for situations where clients have all the domain knowledge and developers have relatively little while open source development seems to be driven by developers with extreme domain knowledge. It makes sense to use two different approaches
Keep up the holy war rhetoric though, it's entertaining and a good parse and filter of the material provides some useful information.
Ahmad, I see few contradictions between your emphases on Quality, Documentation, Testing, Discussions, Transparency, Synchronicity, and Democracy and the values of Scrum. (I think you know this anyway since you write "I don't feel the Agile Manifesto is at odds with The Open Development Method . . .") But remember, the Agile Manifesto is not Scrum, though Scrum aligns with it. I am aware that "synchronicity" is in conflict with agile's preference for colocation, but most agile practitioners don't forbid a distributed team: They just have reasons for thinking that some projects work better that way -- no doubt partly this has to do with the weakness of collaboration tools when agile was getting off the ground.
In matter of detail, it is important remember that Scrum is not a methodology for software engineering only -- it is a framework for developing and sustaining complex products (emphases on "complex" and "products"). That's a big and fundamental distinction. Scrum is rather silent on expecting testing, story scoring, etc. It doesn't specify these things, nor does Scrum forbid it, but the focus of Scrum is elsewhere.
I'm not sure about the "snake oil" labeling. The problem with labeling anything as "snake oil" is that the fundamental issue is likely more with gullible people who are given distorted versions of whatever is being "sold."
I still do not get why Open Methodology and Scrum (or other Agile tools) could not work together. Code quality can be important for a company/project and in this case related tasks can get priority and get into the sprints. Also you can create a workflow and tasks can go through development, documentation, testing (in whatever order).
Excellent article. Thank you for sharing.
In response to your comment "We live in a world where Products need to span Microservices, Mobile Platforms, and Multi-Screen and No-Screen devices. the skills and tools of Software Developers, Product Designers, and Architects no longer fit this multi-dimensional world, and there for teams get bigger, more diverse, and split responsibilities to a place where Scrum comes out short."
The Windows 10 development team at Microsoft just transitioned fully to Agile. It is a product that could not better apply to your description of something that spans "Microservices, Mobile Platforms, and Multi-Screen and No-Screen devices".
Yes, Microsoft is embracing open source, but reception for Win10 was highly positive, especially for their insider program and iterative build/release program-- all of which is driven by the agile manifesto. I think that is powerful evidence of the strength of Agile.
There is nothing in Scrum that prohibits most of what you say you want in the original article. The single exception being the asynchronicity part. You can certainly have high code quality, unit testing, automation and all of the other "code quality" (and other quality issues) that you want within the Scrum framework. What part of the Scrum Guide says otherwise?
Scrum is meant as a starting framework to get people thinking in the right way and gives a small set of practices that seems to help most organizations start down the right path. It is NEVER intended as an end-point of the journey. If you want to have cargo cult Scrum I suppose it could look terrible, but I hope we are past the point where organizations simply do A, B and C without thinking and reflecting on the results and from that, creating meaningful change.
I think equating bad Scrum implementations to all Scrum implementations is a huge overstatement. There are many Scrum implementations that do every single thing on your list except the asynchronicity of the daily Scrum and the reason they don't do that is because they also believe in the Agile Manifesto principle that says "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation." I'm not sure how you can be all for "Agile" and then in the same breath say that asynchronous communication is acceptable. Asynchronous communication may be an impediment in some fashion, but instead of just accepting it, a Scrum team would explore alternatives.
Scrum says to be transparent, inspect and adapt. That is at the core of it and in fact those 3 words are often called the "pillars" of Scrum. If something isn't working, then fix it. Which is why I fail to see how any of your listed wants/needs can't be addressed within the Scrum framework.
I completely agree with your comment. I've seen scrum work beautifully--even in a large/scaled environment--and the only thing that would have posed a challenge would have been the asynchronous aspect.
It's disappointing to see all the negative comments about scrum because it sounds like the authors of those comments have only been part of snake oil implementations, and have never experienced it truly working. I used to be frustrated with scrum until I landed at a shop that truly "got it". Then everything clicked. It's truly amazing to see it in action.
I agree with all the tenets. The tenets Can easily fit into the Scrum methodology. I see people moving from Scrum to Kanban for a simple and more practical reason: There just too many periodic meetings in Scrum!
Thanks Ahmad for your article and triggering this useful discussions. I do agree on the end point you want to promote, and all the positive benefits and good principles of Open Development.
But the article posts some other side ideas which needs reconsideration, in my opinion. I have written this blog to discuss these points.
http://amr-noaman.blogspot.com.eg/2015/11/is-scrum-and-probably-agile-de...
So many people use the terms Agile and/or Scrum without having a clue as to what either of them really are. Some of these people are "snake oil salesmen" but there is also an equally large group of people within business who state they are taking these approaches, but are merely paying lip service.
Then there are articles (such as this one) that clearly have fundamental misconceptions about one (or both) and then use these errors in their own knowledge to make negative claims.
Scrum may or may not be the most effective framework for your organization. If it is not, then congratulations because that means that your organization in in the <20% where Scrum has not been demonstrated to deliver benefit [i.e. Scrum is an improvement over current frameworks for over 80% of development teams].
Agile [as defined by the Manifesto] is a set of 4 Value statements and 12 Principles. If there is *anyone* who believes that they have a scenario where these do not apply, I would welcome a discussion as I have not found this to be the case for *any* of the clients I have supported in the 31 years of running Dynamic Concepts Development Corp.
I appreciated the article and understand where you're coming from as far as managers thinking that Scrum is the end all be all of development world methodology and having nothing but sloppy code as a result. The most successful methodology I have been a part of is when the team has the freedom to use more than one method depending upon the project. Scrum can work really well for short term goals like defect corrections. 1-2 week sprints make sense. But for long term projects, I find that waterfall is a better way to make sure that what you're producing is quality as well as quantity. You can segment things out in development phases and roll outs, applying different goals to each version. I think that an approach where methodology is adaptable in a project by project basis makes a lot more sense, particularly in a virtual global environment.
thank You for sharing