Accelerating OpenStack cloud authentication with Keystone tokens

Register or Login to like
putting the pieces together

This July, the OpenStack community celebrated five years of open cloud collaboration. It's now a global community of more than 500 organizations and 30,000 individual members across 166 countries. OpenStack has nearly four million lines of code and powers the clouds of some of world's largest brands, including AT&T, Disney, PayPal, and Walmart.

Open source components to build clouds—that's basically the problem that OpenStack solves. OpenStack is providing the building blocks to build both private and public clouds in an open source way that's standards based, collaborative, and non-proprietary. What kind of clouds? OpenStack is mainly helping open source developers deliver Infrastructure as a Service (IaaS) clouds with compute resources, virtual machines, and virtual networks that plug together well. It's almost all written in Python with common libraries you can use, virtual machine template storage, and authentication.

OpenStack Summit Tokyo October 27-30 will host thousands of developers, users, and administrators of OpenStack cloud software and feature one of the foremost Keystone experts and OpenStack pioneers in the Operations Track, Priti Desai.

When building your open source cloud, identity and access management are keys to safer and successful authentication for your users. Keystone is the OpenStack identity solution to manage user databases and service catalogs, integrating with backend directory services like LDAP. You can use Keystone to implement your own user single sign-on (SSO) for the cloud. There are a number of Keystone Authentication Token formats to choose from.

Priti is an advisory software engineer at IBM and part of various OpenStack projects including Keystone and OpenStack security projects (formally known as OpenStack Security Group). She has also worked as a principle software engineer at Symantec where she lead the development of identity management architecture for Symantec's private cloud, providing infrastructure and platform services for next generation Symantec products. This year at OpenStack Summit Tokyo, Priti's session will explore Fernet and other token formats.



When she's not working on Keystone implementations or hosting OpenStack meetups, hackathons, or lunch and learns, you may find Priti in a studio practicing the traditional Indian art form of Madhubani painting.

Who is your upcoming session aimed toward? What should they expect?

OpenStack summits are working sessions. They are a chance for the engineers working on different aspects of OpenStack to come together, collaborate on designs, and also to share best practices with their peers in different sessions. Summits are generally for developers, infrastructure engineers, and operations and support engineers. My session is for everyone from these categories.

Identity services are becoming increasingly important in a growing world of highly distributed clouds hosting the web applications we've all grown to depend on, potentially expanding the attack surface. So when we build our private and public clouds, we want to talk about authentication strategies that are simple for user, while remaining as secure as possible.

My session on Keystone tokens will give a deep dive covering pros and cons of the four different token formats to help decide which format is suitable for your cloud. It's for developers looking to learn how the Keystone tokens are implemented and utilized by others in the OpenStack world, infrastructure engineers wanting to learn how to deploy and make configurations changes, and operations and support folks interested in how to troubleshoot and understand the common checklist when you receive a ticket saying there's an authentication failure.

We already have UUID, PKI, and PKIZ token formats. What makes Fernet tokens interesting for multi-site cloud deployments?

We have had quite stable Keystone Token formats running in production deployments for several years now. UUID, PKI, and PKIZ are established and implemented in many OpenStack clouds. They are pretty well vetted.

UUID tokens are the smallest and most commonly used. The main disadvantage is that OpenStack services have to communicate back to Keystone to validate a UUID token.

By comparison, a PKI token is large in size. It exceeds even the HTTP header size. In an implementation for Symantec private cloud, for instance, PKI token size was 18KB. For what PKI tokens lack in brevity, they make up for in efficiency. Keystone middleware can decode a PKI token and validate it without going back to Keystone services for every token validation. The main disadvantage here is that it generates lots of traffic over the wire.

PKIZ is the compressed form of PKI. The size is reduced quite a bit with its optimized compression algorithm, though not close to a 128-bit UUID token. So even though PKIZ is smaller than PKI, it's still quite large for cloud infrastructures that need to transmit data from US-East to US-West, for example, consuming lots of bandwidth.

When you want to grow your OpenStack deployments across the globe, none of these formats are ideal. There is a replication issue. Whenever a user is authenticated in one region, s/he cannot use the same token in another region or data center. You have to replicate the tokens for multi-site deployment, which requires persistence.

These tokens have to be stored somewhere. That storage can be a MemCache or SQL backend. Without replicating the token backends, tokens generated in one data center cannot be utilized in another data center.

With Fernet it's different. It's not as small as UUID, but notably smaller than PKI and PKIZ. More importantly, it's non-persistent. You don't have to cache it. Instead, Keystone rebuilds Fernet tokens based on the metadata in the token itself. When you authenticate across multiple regions, you can use the same token in US-West as US-East, for example. Since Fernet can be regenerated based on the rest of the backend data, it vastly reduces wire traffic.

Let's talk about speed. How does Fernet improve performance and response time, and by how much?

I'd encouraging everyone to read Dolph's blog post on benchmarking these tokens. This study has done benchmarking of the different token formats for keystone, including token regeneration and revocation.

In my session we'll explain these numbers and run it against some of the OpenSource cloud projects I've worked on with an eye toward token creation and validation performance.

You have spent a number of years working on large enterprise cloud projects. For power users, will you share some tips and tricks when you scale Keystones out in a clustered or distributed environment across multiple data centers?

Absolutely. Keystone not only provides a platform for authentication, but also authorization. Enterprise clouds have their own use cases, including number of users, users isolation based on BU, and policies and roles for each BU. We'll also talk through policies and roles per BU, and common issues for public and reseller clouds with restrictions on provisioning users.

We'll dig into the issue of compatibility. OpenStack projects are released approximately every 6 months, Juno last October, Kilo last April. Generally you want to deploy the associated Kilo components with Kilo, and not mix and match. Deployment of each of the pieces of a particular release for live deployment becomes very critical.

One tip we'll talk about with Fernet tokens is that in many cases they're backward compatible going back to the past 4 releases beginning with Havana. For instance, if you've just updated Keystone to Kilo, but all other services are still Juno, for Fernet that's okay. This works nicely when you have a PKI token and want to change the token format to Fernet.

How did you get started with open source?


In 2013, I was frustrated with the team I was with and was looking for a change. I started interviewing at different companies, showing my resume. While describing a configuration and package management tool I'd built to help me on a project, one interviewer expressed surprise and wondered if I'd ever heard of Puppet, Chef, etc. Why was I reinventing the wheel?

That's when I discovered open source and dove in. I quickly realized that while you might gain great problem solving and coding skills working on closed source, you may be missing out on the breadth of solutions and world of collaboration opportunities open source offers.

After joining as the fourth employee on the OpenStack team at Symantec, I started reading and exploring OpenStack documentation and maintaining my own solutions as I found issues. I discovered that by sharing my solutions, I could solve problems for others and get over hurdles more effectively. And, of course, you can get a bit of recognition by blogging and contributing to the community.

Open source has lots of innovative engineers around the world. It blows away the traditional belief in proprietary solutions. You don't have to hide eight hours a day in your cube or office. Step out and have some fresh air. You are not alone. Others in the world may be facing the same problems. Talk to people. Get some ideas on how can we better solve our problems together.

Your success in such a male dominated field is an inspiration. According to OpenStack Foundation marketing coordinator Allison Price, only about 10 percent of this year's Vancouver Summit attendees were female. What advice would you give to other women aspiring to similar technical roles?

Don't be afraid and be fearless. When you hear your colleague (male or female) with a loud or strong voice at the head of the table, he might not be right. If you think you are right and you know the solution, do not cower back in your chair. Speak up and talk loud. Last but not least, be confident and believe in yourself.

Although it's nice to have shorter lines for the women's restroom during OpenStack Summit, I'm looking forward to a change.

At the OpenStack Summit Tokyo, the Women of OpenStack will host several networking events for female attendees and male allies, including a "Command Presence" workshop and a session on overcoming subtle bias.

Check out videos from Priti's past presentations:

OpenStack Summit
Speaker Interview

This article is part of the Speaker Interview Series for OpenStack Summit Tokyo, a four-day conference for developers, users, and administrators of OpenStack cloud software.

Michelle Buckner
Michelle is an Information Security Professional in the San Francisco Bay Area. Her expertise is in Web Application and Network Security Risk Management. A CISSP and CISM with a passion for compliance, she has worked at Cisco, Symantec, and a number of startups. Her interest in Open Source began with her work at Sendmail and on early Linux integration projects.

1 Comment

Thanks everyone for wonderful support, here is the link to the recording from OpenStack Tokyo Summit:

Also, here is the link to the slide deck:

Enjoy !!!

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.