June 11
Vittorio Bertocci | Principal Architect, Auth0

Completing the Foundation: A JWT Profile for OAuth2 Access Tokens

The world of open standards never stays still.

The evolution of the IT industry, together with the changing habits of users, developers and IT professionals, continuously creates new scenarios and puts a new twist on old ones. Open standards must keep abreast of those developments, supplying architects and developers with the tools required to tackle the new challenges in a secure, interoperable and sustainable manner.

Some of the most recent examples of the phenomenon include all the work being done around achieving sender constraint capability in OAuth2 and OpenID Connect (see token binding, MTLS, d-pop) and the encouraging progress the industry is making toward the goal of reducing password usage (FIDO2, webauthn). This is important work, spearheading the creation of the new capabilities we need to add to our solutions to be able to move to the next generation.

However, if you take a pragmatic view, you soon discover that there are plenty of improvement opportunities already in the current set of core specifications and practices—low-hanging fruit that’s easier to tackle and has the potential to make the life of developers substantially better.

For example, there’s outdated guidance still reflecting industry conditions that are no longer true, as it was the case for the use of the implicit grant in OAuth2. There are areas left unspecified by the core specification, which forced vendors and products to come up with a Babel tower of solutions that get their job done, but can’t do so in interoperable fashion. In the following, I’d like to briefly share how the identity community is attacking one of the problems in the later category.

JWT Access Token Use in the Wild

Providing interoperable guidelines for the use of the JWT format in OAuth2 access tokens is one of the most effective bang-for-the-buck initiatives we identified among the low-hanging fruit I mentioned above. Here’s a bit of background explaining the challenge and the opportunity.

The OAuth2 specification introduces access tokens (ATs) as artifacts that applications can use to gain delegated access to resources on behalf of users. The specification describes in detail the function ATs need to perform, but explicitly stays out of giving any indication about AT encoding format.

That stance makes perfect sense in a large number of scenarios. In particular, in any situation where the authorization server and the resource server are co-located, the AT can simply be a reference to some shared memory location. For example, the user grants consent for a given operation, the authorization server records it in some database and returns as an AT an opaque reference to the corresponding row. When the client application presents the AT to the API at resource access time, the resource server resolves the reference, reads the recorded grant from the shared memory location, and applies the delegated authorization directives it finds there.

That approach isn’t as suitable when the authorization server and the resource server run from different locations, and/or are administered by distinct and separate business entities. That’s the case for every scenario in which resources offload identity management to external products—cloud services, on-premises products acting as identity providers, and the like. In those scenarios, your API might be hosted in AWS lambda or Azure functions, while you might have outsourced authorization server functionalities to a PingFederate server, an Auth0 tenant, a local AD FS, or any other authorization server product running in a completely different location and infrastructure. In that topology there’s no obvious way of harnessing shared memory: you now have the problem of teaching APIs how to evaluate incoming access tokens without the luxury of looking info up in the same database the authorization server uses

Although there are approaches to the problem that can preserve the use of opaque tokens (authorization servers can expose introspection endpoints that APIs can use to inquire on the validity and “content” of the tokens), one of the most common solutions IDaaS providers and products adopted en masse is to issue ATs in a known format, and to teach resource servers how to validate such tokens at access time. As a result, after an initial setup, APIs can now independently decide whether an incoming AT should be considered valid without relying on any further communication with the authorization server. Furthermore, the authorization server can package any information (claims) in the ATs that the API might need, so that it’s made available at every API call, again without the need for further network traffic. In other words, this approach transmits information by value, as opposed to the original by-reference method.

Providers overwhelmingly opted to encode their ATs in the JWT format, which has many good properties that make it suitable for the task. But the most practical reason it was picked is that it’s already the format of choice for a similar solution. OpenID Connect introduced a new token type, the ID token, and used the JWT format to prescribe the ID Token’s exact layout, content and validation criteria. Access tokens and ID tokens serve different purposes, but using the same underlying format made it possible for authorization servers to reuse infrastructure (token issuance pipeline, discovery documents, etc.) and for API developers to reuse low-level validation and parsing components for the JWT format.

The fact that so many providers chose to use JWT for their ATs created a high-level pattern developers could harness to reason about topologies and solutions. The practice is so widespread that many practitioners are actually convinced that the use of the JWT format for ATs is actually enshrined in OAuth2 itself.

Given that all ATs must perform the functions OAuth2 created them for, all implementations share the same high-level traits; however the lack of any guidance in this area forced every vendor to come up with their own solutions. Even when such solutions are functionally equivalent, any difference in syntax makes interoperability impossible. In turn, that translates to the impossibility of creating software development kits (SDKs) that work across vendors, the emergence of anti-patterns (such as potential privacy violations, routes to elevation of privilege and session forging, etc.) and overall confusion. Every time they engage with a new service or product, developers must learn about the choices and idiosyncrasies the vendor introduced when designing their JWT encoding for access tokens.

The JWT Profile for OAuth2 Access Tokens Specification Draft

Having observed developers struggling to deal with the variations in the way different vendors use JWT to encode access tokens, last February I decided to find out just how profound those differences were. And I wanted to know if it would be possible to find enough similarities to devise an interoperable solution that could satisfy every scenario.

I reached out to several identity experts representing the most widely adopted services and products using JWT as the encoding for their access tokens, asking for real-world instances of the tokens they issue, hoping to better understand the semantics of the various components. (Almost) everyone responded enthusiastically, generously sharing their knowledge and ideas. After collating the results in a single table, I confirmed my initial hypothesis: nearly every vendor implemented the same functional areas, with the differences mostly limited to syntax.

I presented my results at the OAuth2 Security Workshop 2019 in Stuttgart, in front of an audience of protocol experts who provided invaluable feedback. This helped shape an initial proposal for an interoperable profile using JWT for encoding access tokens. You can find the presentation deck here.

I was invited to submit a draft specification to the Internet Engineering Task Force (IETF), and remotely present the proposal at the IETF meeting in Prague just the week after. This made for an interesting flight back from Stuttgart to Seattle (thank goodness for inflight WiFi).

The presentation was well received, and you can find the deck here. As a result, the working group chairs proposed its adoption as Internet draft on the OAuth WG mailing list, where it received widespread support. After the voting period, the spec was officially adopted as Internet draft and can be found here.

At this point, the discussion continues to refine the details. There are numerous threads on the mailing list discussing specific design choices, and whenever I participate at some identirati congregation (such as the latest IIW, the upcoming Identiverse and the IETF meeting in Montreal), I take the opportunity to tap into the remarkable brains of the identity experts in the room.

The details do need to be ironed out, but at this point it’s clear that there’s widespread desire to solve this problem together and get to an interoperable solution we can all benefit from—IDaaS/product vendors and app developers alike.

What’s Next?

The daily life of developers in the identity space is punctuated by challenges, big and small. Some of them require tectonic shifts at the industry level, paradigm changes of the magnitude of sender constraints or FIDO2/webauthn technology. Some others can be solved by much less dramatic advancements, such as vendors agreeing on the basic format of fundamental artifacts already in widespread use—making it possible to adopt vendor-neutral SDKs that improve productivity while enshrining the security and privacy best practices every solution should enjoy, without forcing each individual developer to get a PhD in identity management and reinvent the wheel at every new project.

Agreeing on a common format for access tokens is one of such low-hanging fruit that has real potential to make developers’ lives easier. The discussion of the draft is ongoing on the IETF OAuth working group mailing list: we hope you’ll join us and contribute!

If you want to hear more about the importance and evolution of standards, don’t forget to attend the keynote panel Standards: the Bedrock of Identity on Thursday morning at Identiverse.

View More Posts

Identiverse is Fantastico!

— Vittorio Bertocci