Plug-and-play server provides access to millions of digital documents in Africa

No readers like this yet.
Penguins on beach

Original photo by Rikki Endsley. CC BY-SA 4.0

The WiderNet project, which is affiliated with WiderNet@UNC at the University of North Carolina at Chapel Hill, provides resources, coaching, training, computers, and educational materials to schools, clinics, libraries, and homes in underserved areas of the world. In this interview, Cliff Missen, the Director of the WiderNet Project, explains how the non-profit helps improve digital education and communications for international communities.

Cliff was a TED Fellow in 2007 and a Senior Fulbright Scholar in Nigeria in 1999. He and his team are heavy users of open source and proponents of open knowledge and open access.…" title="" typeof="foaf:Image" width="520">

How did the WiderNet Project get started?

I spent a year in Nigeria in 1999, teaching at the University of Jos and visiting universities around the country. Everywhere I went I was struck by inequities in information technology and information access. Academic leaders would tell me, "The only people who want to talk to us about technology are those who have something to sell." I realized that my colleagues needed to hear another point of view from reliable IT people who know the total costs of setting up and supporting systems as well as understand the vast economic disparity between universities in the West and those in developing countries.

Don't get me started about Internet connectivity. Bottom line today: it's rare, it's expensive, it's slow, and it's unreliable. So one solution we've developed is the eGranary Digital Library, a plug-and-play server that provides instant access to millions of digital documents without the need of a connection to the Internet.

I know it's probably hard to do, but can you quantify what open source and open knowledge has meant to the people you serve?

Our off-line education server, the eGranary Digital Library, is chock full of open source tools. We use open source whenever possible in the back-end services. We include open source software installers on the eGranary. And more than half of the 3,000 Web sites on the eGranary are Creative Commons, public domain, or some other open license.

How do you use open source software at WiderNet? What's in your software stack?

Cliff: We use HTTrack as our primary site mirroring tool, and fall back to MGET and other specialty utilities for hard-to-scrape sites. On the eGranary, we start with a modified Apache server that serves up over 3,000 sites. Our search service is built on Lucene and Solr. VuFind powers our catalog. We use OpenDS for login services. Although the eGranary works with any browser, we've developed an optimized version of FireFox to improve the patron experience. MySQL, Moodle, PHP are available to the end users to develop their own apps.

Finally, we have the Community Information Platform (CIP), a set of integrated tools that allow users to make their own web sites and upload local content. OpenLDAP provides the logins for unlimited users. Each user can be assigned unlimited web sites on the Apache server. The editing and design tools, a mix of CKEditor and a half-dozen image-manipulation tools, are all bundled into a single interface based on a web file file browser.

Beyond the software stack, what open knowledge data sources do you use?

First and foremost: Wikipedia. And Wikimedia. The whole thing—no abridged versions here. We have Khan Academy, MedLinePlus, the Gutenberg Project, MIT BLOSSOMS. Really too many to mention without dissing a thousand other authors.

What technology do your users usually have?

It's a mixed bag. Some institutions have computer labs—usually 20-30 computers for an entire school with 1,000+ students. Some places, like medical schools and hospitals, many staff and students have a laptop or tablet. Smart phones show up a lot, but only about 15% of the phones in developing countries are smart phones, so only the better-off have them.

Android devices are extremely popular, as are Windows computers and laptops—very few Apple products, except for the wealthier folks.

Electricity is a huge problem. Some universities experience frequent outages, sometimes daily. Even if they have a backup generator, there's usually a 10-minute down time. Oftentimes the UPSs don't have time to recharge fully and they are spoiled within a few months. This means the servers and workstations get shut off unexpectedly at a rate that's hard to comprehend in wealthier countries.

What's a typical day/week look like for a WiderNet user?

Over 3-million people have access to an eGranary at their site. Some of these use it constantly; many are only beginning to adapt to digital information. Remember: It took most of us 10 years or more to develop our digital skill sets. I think the most important impact comes when teachers or practitioners find information on the eGranary and pass it along to others. Despite all our enthusiasm for information technology, we have to remember that today—just as for thousands of years— nothing teaches like a teacher.

For many people, significant hurdles in language and information literacy makes it hard for them to adjust to the abundance of information—over 32-million documents—in the eGranary. In some cases, people are simply shell-shocked by the vast quantity of information. In many situations, people can't afford books, their nearest library is several towns away, and there's no book-lending. Students lack books and they rely on their notes and each other to recall what they've learned in class. Sometimes the instructors lack books and are teaching from their own notes on makeshift blackboards. To suddenly be exposed to millions of resources on every topic under the sun, most of which they have never before conceived, is like drinking from a fire hose. So it falls to us to go out of our way to understand teachers' needs and make information readily findable for them.

Given you are often deployed in no/low-bandwidth areas, how do you get feedback on your system? I imagine just the simple act of troubleshooting is a lengthy and time-consuming process.

Our first mission is to make the eGranary as bullet-proof as possible. For many of our subscribers, it's a plug-and-play situation. But FUD—fear, uncertainty, and doubt—hampers some installations. Or in a few cases, merging our server into an existing network gets more complicated. Or, in some cases, the local area network is mal-configured and the "newcomer" eGranary is held to blame. Either way, we try to be very responsive, using email, Skype, and phone calls to help our clients through the process at the least possible cost to them.

The best solution, we've learned, is to contract with technically savvy young entrepreneurs in the deployment country. We have Field Associates in nine countries who assist hundreds of customers with installation, training, and support.

We get Apache logs from our better customers, which help us to understand usage patterns. Occasionally we do a survey, but reaching many of our customers is cumbersome. It's hard to do an online survey when your respondents aren't online.

My favorite bit of feedback: Last year more 80% of new eGranary installations were done by someone who had already installed one and came to back to get more.

What has the more prevalent access of mobile technologies meant for you and your team? How are you using mobile to reach more people?

More people have access to telecommunications and the Internet through mobile technologies, but we have to be realistic. While people may live in an area served by 3G or 4G cellular technology, it's still very expensive for most people. In most developing countries people pay per minute or per byte for their services. In every country there's a cadre of wealthier people who can afford decent Internet services, but most have to be very careful how much they use the the Internet. Of the few people who use Internet services on their phones, most of their use is transactional—texting, electronic payments, or civic services, such as school registration, health monitoring, etc. Of course, many people avoid large files, such as PDFs, audio, and video.

The eGranary education server can be used by any WiFi device, including smartphones, which encourages subscribers to add more access points to their networks to leverage privately owned devices. As well, we're developing smaller collections aimed at specific audiences—such as health professionals and school children—that fit into hand-held devices.

Your solution requires quite a bit on the hardware side of things as well. Have you looked at all into any open hardware solutions?

We've experimented with devices like the Raspberry Pi. Once we get past the "Wow" factor, we haven't found a good niche. They are woefully under-powered to provide a host of services to dozens or hundreds of users running searches and building websites.

For smaller group situations, once we add the external devices to make the things usable—like keyboards, screens, significant memory and storage, and serious WiFi—we wind up spending more money than if we'd just used a tablet or laptop in the first place. So either we dumb down the eGranary to the processor, or we upgrade the processor to the eGranary.

In most cases, a decent used Windows PC can be found for the price of a fully pimped-out Pi. And most often, the person providing technical support is more familiar with the Windows environment. Despite our best efforts to introduce an open operating system, we learned that Windows is far better understood by our subscribers. Few have a dedicated technical staff, and it would take years of training and practice to make them as savvy in an open OS than the one they've used for years.

What are ways the broader open source community can help?

Our mission is to deliver a world-class off-line education solution to people who are currently underserved by the Internet. That means we're constantly polishing our product, adding new features, updating code, and seeking new content.

We can use help adapting off-the-shelf solutions, developing the code that cements the bricks together, creating apps to meet end-user needs, and testing these things so they work as seamlessly as possible in the field. Someone experienced with WiFi and GSM might help us develop hybrid off-line/on-line tools that deliver the greatest benefit for the least amount of bandwidth. Someone with design skills can help with interfaces for our many customized portals. Someone who understands streaming video can help us adapt or create tools for better video capture.

Every time I take a new version of the eGranary into the field to introduce the newest features to our partners, they say, "Brilliant! Now we just need ..." So no matter how much progress we've made, I always have a list of about 150 new features we'd like to add.

Looking forward, what does the next year or two look like for the WiderNet Project?

With over 1,100 installations worldwide and growing, including 12 correctional facilities in the U.S., we're just going to get better and more broadly deployed. By listening closely and patiently to our partners, we've learned what people generally need. Now we're launching into collaborations with information science programs and educators to do content development and curation in-country, integrating local content and languages so we create off-line libraries that meet their specific needs as well.

Today, two thirds of the world's population lives beyond the reach of the Internet. Very few of them have public libraries to fall back on. Few of them have first-hand access to information resources of any kind. It's not because they've made bad choices—they are simply poor. Empowering the poor through education has been humanity's challenge for millennia.

When I launched this project 15 years ago, many around me said it was impossible. A major foundation actually commissioned a study to confirm this. Even today, on a shoestring budget, it's still impossible. Fifteen years from now it will also be impossible. But we cannot let the perfect be the enemy of the good. We lean in. We do the hard work. We serve as robustly as we can. Only by flying into the teeth of the impossible do we come to understand what is truly possible.

Billions of people cannot wait for the Internet. We need more solutions like these to deliver information and education today.

More important than any physical technology, the social contract we've crafted with our partners is the key. The eGranary Digital Library represents the best intentions of a host of software and content contributors. Whether it's millions of open license documents, over 2,500 authors and publishers who have granted permission to copy their works, dozens of open source software titles that help us create and deliver the eGranary, or the over 30,000 hours that volunteers have put it over the years, all of this creates a wonderful mix of information and education tools that serves millions—and someday billions—of people eager to improve their lives.

Follow WiderNet on Twitter (@widernetproject) and Facebook.

User profile image.
Grant is the CTO and co-founder of Lucidworks, co-author of “Taming Text” from Manning Publications, co-founder of Apache Mahout and a long-standing committer on the Apache Lucene and Solr open source projects. Grant’s experience includes engineering a variety of search, question answering and natural language processing applications for a variety of domains and languages. He earned his B.S.

Comments are closed.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.