Surfing the open data wave

Surfing the open data wave
Image credits: Jason Hibbets
submit to reddit
 
(4 votes)

During hurricane season, my attention turns to the Atlantic Ocean. I keep an eye on swell-producing storms and use a variety of forecasting tools to get to the right spot for the best surf. But I never really thought about the data behind the tools I use.

There are many different stats brought together by today's forecasting technology:  swell size, direction, and period; wind speed and direction; and tide predictions—all important factors if you want to score good surf. Where does that information come from?

Answering that simple question demonstrated the importance open data has on a sport that I've enjoyed for almost 20 years. There is probably open data that you take for granted, too. Consider your daily weather report or stock market conditions. What if the source of that data wasn't properly funded? What if it suddenly disappeared?

Open data from the NOAA

When you live a minimum of two hours from the coast, making the right call to go surfing is important. It's not a quick drive to the next surf spot to see what conditions look like, it's a dedicated road trip with the sole hope of scoring good waves. Being able to pinpoint the premier surf spot based on predicted conditions using forecasting tools is a huge advantage—and it's all thanks to open data.

When I began researching the data that powers surf forecasting websites, I quickly realized that many–if not all–sites use the same information from NOAA (National Oceanic and Atmospheric Administration). Because NOAA is a government-funded agency, the data is available to the public at no cost. (See their open access to data policy [PDF].) Similar to how other open source projects use open data, the real value is in the interpretation of the data and presentation to the user. I looked at the way three different services use and credit that data.

Surfline

Surfline.comOne of the first surf forecasting sites I used was surfline.com. They developed a proprietary system of wave forecasting using NOAA and other data sources. They provide free three-day forecasting, but charge a fee for premium services like extended forecasts and high definition streaming video of current surf conditions.

I used Surfline a lot in the past, but my preferences changed over time. I learned how to understand the raw forecast data. Since I had cultivated local knowledge of surf conditions, I began to prefer a simplified interface that gave me data I could interpret myself. And I found the limitations on their free report to also be a deterrent. I sometimes still use Surfline, but not as my first source of information. People are willing to pay for the premium services—in fact, I was a subscriber at one time.

Swellinfo

Swellinfo.comOne of the sites I frequently use for surf forecasting is swellinfo.com. Like other surf and weather sites, they use NOAA data. Because this data is freely accessible, it allows them to provide value to the end user at no cost.

Micah Sklut, M.S. Lead Forecaster, CEO of Swellinfo, provided the following statement about the value of open source data and the benefits that both service providers and end users receive from and open data source such as NOAA:

"Without this open source data, most private weather agencies in the country would be out of business. In other countries, such as Australia, you can pay a service fee to have access to weather agency data.

Allowing the NOAA data to be free allows for greater research and advancement in technology, and a plethora of business opportunities. If NOAA cut off access to their data, then we would have to solely rely on NOAA to provide the services of thousands of companies that now use this data.

I think NOAA is such a great example of a publicly funded project that truly helps the whole system and not just little parts of the system. We all benefit from weather and environmental information."

Open standards and accessibility are critical to Swellinfo.com's ability to build their business. Without open data from NOAA, they would not exist. It's amazing how many opportunities come from the same stream of data--the next example shows how different it can be.

Magicseaweed

magicseaweed.comThe site I use the most is magicseaweed.com. The value they provide is in the arrangement and presentation of the data. And while the data is open, the presentation is where they have determined their intellectual property lies. I also like it because it's free.

When I contacted the site maintainers for comments, they wouldn't reveal the source of their data, but did confirm it is open data. I wonder why they're reluctant to share? That's not very open source of them.

The disclaimer on their site confirms the data "exists copyright free in the public domain." This leads me to believe that it is NOAA -based.

As an experienced surfer, I can scan a tool like magicseaweed.com and understand most of the data they present. For example, I can interpret what an eight-foot swell with a 12-second period would look like with a light wind. (The answer: chest- to head-high waves and clean surf.) At that point, it's time to get on the coconut wireless and start making surf plans.

Their five-star rating system makes these determinations more obvious—and indicates to novice users if conditions are generally good or bad. But knowing the coast and having experience definitely influences how accurate the surf model is.

This site does more than just present the different elements in numerical format with a few icons. Magicseaweed compiles this data into infographics that show the data moving over a certain period of time. You can simultaneously see the predicted swell size, swell period, wind speed, and wind direction in three graphics that are side-by-side. Again, the value is in presenting that data to an end user in an easy-to-consume format.

Which tool is right for you?

There are many surf forecasting sites on the web. The one you choose will be based on your preferences. For me, Magicseaweed is my primary source. At a glance, it allows me to see the metrics I'm interested in. As I start to drill down into the data, I'll use Swellinfo to confirm my interpretation. I might log into Surfline to compare forecasts and view real-time video (though it's limited to 60 seconds for free users).

One thing all off these sites have in common: community. I think each of these sites recognizes the power and usefulness of the end user. Like many companies today, they include various features on their site intended to drive a community.  When surfers aren't in the water, forecasters want them surfing online. Building in community functionality is one way to keep people coming back—and keep developing the tools and views they want to see.

At the end of the day, nothing beats a real live surf check. As you scan the horizon for swells and watch them crash onto shore, you can see if all the time spent analyzing the data was worth it. The real question then becomes: Do you paddle out where you're checking the surf or scope out a different spot?

Open today, closed tomorrow?

All of these tools use open data from NOAA. They take this complicated, hard-to-interpret information and transform it into user-friendly surf predictions the same way a meteorologist gives us the forecast. Can you imagine trying to forecast tomorrow's temperature and wind direction based on satellite imagery and wind vectors?

NOAA data goes beyond surf forecasting--just like predicting the weather goes far beyond just telling us if it's going to be sunny tomorrow. The same sorts of climate and satellite information is used by seismologists to predict earthquakes, by other meteorologists to foretell the path of a hurricane or of tornadoes. What if downstream consumers like local news companies had to pay for this data? Or, worse, what if some of the data wasn't available at all?

Entire ecosystems have been established around data provided by NOAA. There are many free choices for people that are unwilling or unable to pay for services. But at the same time, there are a variety of business models that can be applied on top of this open data. The best part is, the user gets to choose.

Recently, the openness of this data has been threatened by the U.S. House of Representatives, as they explore privatization of NOAA services. Recent cuts to NOAA's budget may limit their abilities to accurately predict hurricanes. And that's one wave that we can't afford to wipeout on.

If you've made it this far, you deserve a surf video.

""
Creative Commons License

5 Comments

Ben Freeston's picture

Sorry if we weren't forthcoming on our data sources when you contacted us, we get 100's of emails a year from people asking the same question and don't always have the time to reply in detail - particularly as a good half of these enquiries start "I'm thinking of building a surf forecasting business like yours can you tell me how exactly you've built MSW" (I'm not kidding!). However it's no secret that our data is primarily but not exclusively provided by the NOAA and other agencies sharing under WMO obligations (but most often actually distributed publicly by the NOAA), however we're increasingly running our own models built upon this data. It's beyond the scope of smaller private businesses like ours to run the global atmospheric models that provide the wind data used for forcing the various swell models (and there are compelling reasons, outside of the value that private enterprise can add, related to the nature of gathering input data globally that necessitate open sharing), but we can and do run our own swell models and interpolated wind models to enhance the range of data we have available, although these models themselves are also open source so again the value we can create is thanks both to shared data and software. So while you're absolutely right that our primary IP is built on open data we do increasingly also create proprietary data from it - although in the spirit of openness we do share this proprietary data free of charge with anyone interested in using it for research and of course with the wider public via our website! Very glad you find our website useful and thanks for including us in the article.

jhibbets's picture
Open Source Sensei

Ben,
Thanks for sharing that info. I'm not surprised that you get those type of inquiries. And I'm almost positive that about 95% or more of the surf population never really gives this type of thing that much thought.

How do you feel about the potential of having to pay NOAA for the data that you use? I didn't go in-depth too much in the article about that, but I'd be curious on how that would impact businesses who consumer this data now (for free). Would businesses like MagicSeaWeed be willing to pay for that data? If so, would you just pass it onto your consumers?

Thanks for the reply,
Jason

Ben Freeston's picture

To be absolutely honest right now having to pay for data would be a benefit to us (although that would depend hugely on the exact price and licensing model). It would build a wall between us and potential competition. Of course track back a decade and we wouldn't be here if that data hadn't been free. So selfishly I'd benefit from chargeable data right now - but my vote (and the widest benefit) will always be for open data. This sector is an absolutely perfect example of how open data creates businesses, value and revenue, which I'm sure if someone took the time to investigate could be show to be generating more return in taxation than licensing the data ever could.

Bob McWhirter's picture

In the past, I've used and been amazed at the quantity and quality of data coming from both NOAA and the USGS. The National Atlas (http://www.nationalatlas.gov/) is also a treasure trove of data for GIS geeks, easily mixed with the weather data. Unlike the National Atlas, though, I've found digging around the NOAA website to locate data to be more difficult than it seems like it should in this day and age.
But big ups to the guys in the national weather data center (in Asheville, NC, I think) for making it all available.

jhibbets's picture
Open Source Sensei

Bob,

It might be interesting to do a story about the National Atlas with a little more detail. Are you up for contributing a blog post about that--focusing on open data and your experience with it?

Thanks,
Jason