#45 March 21, 2019

SPIFFE, with Andrew Jessup

Hosts: Craig Box, Adam Glick

SPIFFE is the Secure Production Identity Framework for Everyone. Craig hates the name. Andrew Jessup, co-founder and VP of Product at Scytale (with a C) tells him and Adam why they should look past that and how Jason Bourne fits into the world of Cloud Native.

Do you have something cool to share? Some questions? Let us know:

Chatter of the week

News of the week

ADAM GLICK: Hi and welcome to the "Kubernetes Podcast" from Google. I'm Adam Glick.

CRAIG BOX: And I'm Craig Box.


By the time you hear this, I'll be in Christchurch, New Zealand. It's a place that's very special to me. I lived there for a few years as a child. My family are from nearby Timaru. My aunt and uncle, who I'm staying with, live in Christchurch.

And I'd really just like to say thank you for the messages of support and well-wishing for the Christchurch community, for New Zealanders, for Muslims around the world. It's been a tough week here. There have been a number of vigils around the country. I don't really have anything to say above what's already been said by many, many other people, but just find someone you love and hug them a little bit tighter tonight.


What's up with you, Adam?

ADAM GLICK: I am discovering that there's certain inalienable laws in the universe. And one of the things that I've discovered is that anytime that you take apart something that you've gotten from IKEA and you go to put it back together, there will be less pieces than when you started. It's kind of the opposite of entropy, but it no doubt proves out to be true as I've discovered, as we're now on the hunt for various little hardware pieces to put things back together as we're in the middle of our move.

CRAIG BOX: I read a book once, I think it was by the author Robert Rankin. And he has a theory-- the small screw theory. And basically, it says that when you take something apart, you'll always end up with one screw left over, which you can't find a home for. And you should never take the thing apart to try and find a home for that screw because you'll end up with two small screws left over.


CRAIG BOX: And his theory is basically, small screws breed in toasters and other electronic appliances, and that's what's happening.

ADAM GLICK: Sounds true enough. Shall we get to the news?

CRAIG BOX: Let's get to the news.


CRAIG BOX: Istio 1.1 is out. It's hot off the press, so we can't bring you the full release notes, but we can bring you the headline features, which include order of magnitude improvements in reliability and scalability, multi-cluster support, and the ability to bring your own certificate authority to the mesh. Find out more at istio.io.

ADAM GLICK: F5 Networks has announced the acquisition of NGINX for a reported $670 million, continuing the streak of open source companies being purchased. F5 has stated that they expect the acquisition to increase both their revenue mix and their revenue growth. F5 has also stated that they intend to continue the NGINX name and brand after the acquisition.

CRAIG BOX: Stealth service mesh startup Tetrate have launched this week with $12.5 million in funding to build the Istio and Envoy-based service mesh for the enterprise. Tetrate-- the Heptio of Istio-- now employs a number of early Istio engineers and PMs, including its CEO Varun Talwar, who worked on gRPC and Istio, in his 10 years at Google. Along with the funding announcement came GetEnvoy, with no C or K-- a service which offers certified, compliant, and conformant builds of Envoy.

ADAM GLICK: Meanwhile, Buoyant-- creators of Linkerd-- have announced a $10 million investment from Alphabet's VC arm, GV. Linkerd2 has pivoted to a "it's like Istio, but simpler" business model and Docker Swarm would like to wish them well with that.

CRAIG BOX: The schedule for KubeCon in Barcelona is now live. With hundreds of sessions, there is plenty to learn for beginners and experts alike. The show takes place starting on Tuesday May 21 with two pre-days [before]. And we're looking forward to seeing all the great new advancements that are announced at the show.

If you didn't get your talk into KubeCon, don't despair. The delightfully-named Cloud Native Rejekts -- with a K -- is a community conference being organized the weekend before. The deadline for CFP closes the day this episode is released, so you may have missed it, but don't miss buying a ticket if you can be in Barcelona a few days early.

ADAM GLICK: Rancher has released Submariner-- a multi-cluster network connectivity for Kubernetes. Submariner is an open source project that is designed to help users connect across clusters, premises, and geographies. This can be useful for doing actions like database replication and in connecting services that do not share the same network addressing space. Submariner also helps secure the communications between clusters by using an IPSec tunnel and is CNI compatible, so it can work with other Kubernetes projects like Calico and Flannel.

CRAIG BOX: Google Cloud hosted a Container Security Summit recently and wrote a blog post summarizing the key learnings. Don't run your build processes as root. Make sure your builds are repeatable, that you can create the same containers from the same source code, and that they are hermetic, which is an overly fancy way of saying it should be an isolated process and no one build should impact any other.

Additionally, the summit identified how the community is handling the discovery and disclosure of bugs, and it reinforced that CIS benchmarks are still the standard for security, even if there are times where it doesn't make sense.

ADAM GLICK: The CNCF has announced it now has 375 members, adding 59 last week at the Open Source Leadership Summit. New silver members range from giants like Ericsson and NVIDIA to cloud-native specialists like ControlPlane and Tetrate.

CRAIG BOX: The CNCF Technical Operating Committee has taken its shape for 2019. New members include Liz Rice from Aqua Security, selected by the governing board to take over for the departing Kelsey Hightower who has resigned from the committee. She was first runner up in their previous election. The committee also get to vote two of their own members. and their pick for the final spot was Michelle Noorali from Microsoft.

ADAM GLICK: In a recent cloud Kubernetes analysis, Andrei Dascalu has looked at the Kubernetes services from Google Cloud, Azure, AWS, and DigitalOcean. He covers his experiences setting up and managing each of these and lists out his perspectives on the good and the bad for each. If you're thinking about setting up a cloud-based Kubernetes cluster, it might be worth your time to give his experiences and conclusions a read through to know what to expect.

CRAIG BOX: A new version of Microsoft's Kubernetes plugin for VS Code has been released with live debugging of Node.js apps, an interactive log viewer, and right-clicking on a YAML file to get options to create, apply, or delete. Plus it can also download the right version of kubectl for each cluster you connect to.

ADAM GLICK: The CNCF released a new case study on Kubernetes this week highlighting NetEase-- the world's fifth largest gaming company-- and their use of cloud-native technologies. In the case study, the CNCF talks about how NetEase has increased their research and development efficiency by 100% since starting their cloud-native journey. You can find additional details on NetEase's use of Kubernetes and their 30,000-node cluster on the CNCF website.

CRAIG BOX: Finally, while we're all here arguing about how to pronounce Ancient Greek words, a team of researchers from the University of Oxford have brought some music from the era back to life. A short documentary on Aeon explains the process of deciphering fragments of papyrus and takes us back to 700 BC, settling-- hopefully for the last time-- how the word kubectl should be sung. Find it in the show notes.

ADAM GLICK: And that's the news.


ADAM GLICK: Andrew Jessup is a co-founder of and head of product for Scytale, the company behind SPIFFE. Welcome to the show, Andrew.

ANDREW JESSUP: Hi, Adam and Craig. Great to be here.

CRAIG BOX: Thanks for joining us.

ADAM GLICK: So you used to be a Googler, right? You were a PM?

ANDREW JESSUP: Yeah, I was. I was a PM on Google Cloud. Craig and I crossed paths a few times actually.

CRAIG BOX: We did. And we just would hang out in London for a few days. That's where I met him.

ANDREW JESSUP: Yeah, yeah.

CRAIG BOX: You and your British passport.

ANDREW JESSUP: Exactly. I started on App Engine and over a time, moved over to Compute Engine. And one of the things I worked on actually with Compute Engine was the autoscaling feature, which we didn't have at the time. And Amazon had had it for a while and we had a lot of customers asking us, why? I thought the whole point of moving to cloud was to be able to autoscale. Why can't we do that?

So I worked on that initially, actually, with another product manager at Google, a gentleman called Sunil James-- who I don't know if either of you ran into him at the time, but actually he'd come from Amazon where he'd been running networking and security on the product side on AWS, joined Google to do much the same. And he and I sort of went back and forth on the autoscaling project.

And I enjoyed working with him so much that when several years later, he called me up and said, hey, listen. I'm thinking about starting a company. Are you interested? I said, yes, without even really thinking about it too much. So Sunil is now my boss and the CEO of Scytale.

ADAM GLICK: Interesting.

CRAIG BOX: So you obviously pronounce it "sigh-tale".


CRAIG BOX: It's a hard C. It's a hard K. It's one of those it could-go-either-way names.

ANDREW JESSUP: It is. It's a tough one. And very few people-- I hesitate to almost say that's the correct pronunciation because most other people pronounce it either "sky-tale" or increasingly recently, I've been getting "skid-ah-lay," which if we ever become an Italian pasta sauce company, that's going to be helpful. But we say "sigh-tale."

The second question we tend to get is, what is a scytale exactly? And what I do, as it turns out, there's a couple of things. One is it's a "Dune" reference for anyone who's a fan of the "Dune" series, as is our CTO Emiliano.

The other thing it is it turns out in the, I think it was the ancient Persians, a scytale was an encryption device that they used. So what it was was a hexagonal stick of a very particular size in diameter. And they would wrap a piece of leather around this wooden stick, and then they'd write crosswise on it to write the message. And so the idea was that you would need the exact same size stick in order to decode the message properly, otherwise things wouldn't quite line up.

And we liked that because it spoke both to our Sci-Fi roots, as well as our encryption and security roots.

CRAIG BOX: And your need to name things in Ancient Greek.


ANDREW JESSUP: Exactly. There's certain rules of consistency there in infrastructure software that you just can't ignore.

CRAIG BOX: So SPIFFE started out as a talk that Joe Beda, who's one of the co-creators of Kubernetes, gave at KubeCon 2016. How did it become a real thing?

ANDREW JESSUP: That's a great question. I mean, you know, how Joe came around to SPIFFE, I think, was you know Kubernetes of course came from Borg at Google.

And you know, after Joe left Google, he spent a little bit of time on what I think he calls his vision quest, where he was looking for systems or projects that could work out in the same way that Kubernetes had, in that take a system, a design pattern that had become more or less ubiquitous at Google, and had become largely proven, and indeed had been replicated inside some other Bay Area Silicon Valley companies, and create a relatively-easily consumable, externalizable version that could be applied to any organization, not just these organizations with the sort of physics of Google.

And so when he was looking around, he was looking around for other software systems that could serve the same treatment. And one of the ones he landed on was a similar system at Google called LOAS-- the Low-Overhead Authentication Service-- which I think now is being called, or parts of it are now being called, ALTS.

And so the LOAS system, what it does at its heart, it's a system for issuing identities to software systems. So the way identity works with LOAS, and with SPIFFE in fact, it's the I tend to call it the Jason Bourne model of identity. Normally when we need to identify ourselves to certain systems, we have some idea of our own identity to begin with. When I log into something like Salesforce, for example, I know who I am fundamentally, and I probably know my username, and I maybe even know my password. So I carry around a certain sense of identity with me at all times.

With software though, it turns out that's not always true. In fact, it's possible, of course, for software to have its own concept of identity which it carries around with it, but it turns out that's not a particularly great way of managing identity with software systems, partly because it means that this identity needs to be securely encoded with the application in some way. So maybe that's a configuration file, or hardcoded into the binary, or something like that. That tends to be very much a security anti-pattern.

So the alternative-- the way identity works at Google is this Jason Bourne model. And if you haven't seen the first movie, you know, I'll ruin the first 30 minutes of it for you. You know what happens is this action hero Jason Bourne, he wakes up floating in the middle of the ocean. And he's rescued by a kindly fisherman who helps him learn who he is. And once he learns who he is, then the action happens and the fun begins.

And so LOAS, and later SPIFFE, in this world, play the role of the kindly fisherman. So what happens is a workload, a software system at runtime, when it starts up and even repeatedly after it starts up, they call into this LOAS system or into the SPIFFE system and ask simply, who am I? Tell me who I am.

And it's up to this system to do two things. Firstly to figure out who it is. And we can talk a little bit more about the details of that later. And then once it's done that, give it an ID in the form of a label, a string-- with SPIFFE, it's this thing called a SPIFFE ID-- and then more usefully, to be able to give it documents that can prove that identity. So you know, sort of the equivalent here of a driver's license, if you will. And in the case of SPIFFE, that could be done in a couple of ways, but it's typically done with an X.509 key and certificate.

And so the end state of this is you have a whole set of different software systems, all of which are leaning on SPIFFE to retrieve their identities, and then also to be able to validate the identities of other software systems of other workloads that have been given identities as well. And in doing this, what you're able to do is to establish trust and authenticate these different software systems together without them ever needing to actually have their own identities embedded inside them. Instead, identity effectively becomes a kind of function of the platform.

CRAIG BOX: Is there a YAML Bourne?


ANDREW JESSUP: There is actually no YAML in SPIFFE today, you'll be pleased to hear. There's certainly talk about adding some and adding configuration options, but we're shying away from that just for now. We're still trying to figure out exactly what to call our command-line tool, so the c-t-l versus control debate is still raging in the SPIFFE community.

CRAIG BOX: We've been talking a lot about SPIFFE to this point. SPIFFE is an acronym?


CRAIG BOX: What's it [an acronym] for?

ANDREW JESSUP: SPIFFE stands for the Secure Production Identity Framework For Everyone. And the next question we tend to get after that is how on Earth did you come up with that acronym? Well, how did Joe come up with that acronym? And SPIFFE, it turns out, is a play on GIFEE-- Google Infrastructure For Everyone-- which I think CoreOS might have coined.

CRAIG BOX: It was actually Google Infrastructure for Everyone Else. It has two Es, which always confuses me when I go to type SPIFFE.

ANDREW JESSUP: Ah, gotcha. I didn't know that either. Well, there we go. Yeah, so there's one E in SPIFFE and we lost the else, but we're paying homage to that same idea. And it is very much that same idea-- taking ideas that worked well inside Google, and other companies too for that matter like Netflix, and making them available for everyone and making them communal.

CRAIG BOX: So as the new stewards of SPIFFE, why haven't Scytale considered changing the admittedly very stupid name?

ANDREW JESSUP: You know something? Stupid names tend to stick. And to a certain extent, it's kind of calcified now. So there's a lot of things that a branding expert, I'm sure, would recommend changing a lot of things-- the name to begin with, possibly the logo, possibly the logo's colors. But you know, to be honest, we've kind of grown to love it. And for better or worse, the community now knows what it is, so I think now we're stuck with it.

ADAM GLICK: Why is this identity need important now? There's lots of identity providers and identity protocols and specs that are out there already. What is SPIFFE doing that's new and necessary for the cloud-native world?

ANDREW JESSUP: It really has been around for a long time, as has authentication, but I think there's a few trends we're seeing in software development in general that are priming us to rethink this. The first is just the sheer heterogeneity of the software systems that we're working with now.

You know, it used to be the case that I'd run everything on something like WebSphere. That WebSphere cluster would be running inside a fixed set of physical machines running inside a data center. And for the most part, you know, it would be a web application talking-- the interactions within an application would be something like a web front-end talks to a database or something like that. And occasionally, you'd have a little bit of middleware in there, but the solution space was pretty straightforward.

And if you fast forward to today, it's a very different world. We have cloud providers, for one thing. So some of my infrastructure might still be running on-premises, but a lot of it may now be running in the cloud as well. And as often as not, it's not just one cloud provider, but several. And a single customer experience might require interactions between software systems that are actually running across multiple clouds, as well as on-premises.

And then within that, we also now have this diversity of middleware options. And so we have some people running things like Kubernetes, some people running things like Cloud Foundry, some people running past systems they've built themselves. Some people are still running things like JBoss and WebSphere. And again, all of these software systems need to interact in order to actually deliver some meaningful experience.

And I guess, that sort of brings us to the other trend that's really driving this, which is this idea of microservices. And you know, whereas previously, we used to have software components communicate by passing data structures around in memory, now we need to serialize messages and send them over the wire. And we need to make sure that those messages are strongly authenticated, that the message hasn't been tampered with while it's in transit, and that the message is actually protected and is confidential in transit.

So all of these forces, in some ways, have put us in a software development environment that's quite different from what it was 15 years ago or even less. And so that's the core piece that's forced us to rethink a little bit about how we think about identity. And as you say, there's a lot of existing mechanisms for doing workload identity, but they tend to fall down or prove somewhat inadequate on a couple of fronts.

You know, the simplest way of doing identity is just with things like usernames and passwords, but when you do that, you then get into a secrets management problem of how do you manage these passwords and usernames. And then you end up centralizing them using a secret store usually of some kind. And then you have a new problem, which is how do you authenticate to the secret store?

Another model a lot of people use is to lean on the-- you know, if you're running in, say, a platform of some kind, whether that's a platform like Kubernetes or an infrastructure provider like Amazon Web Services, then I can often lean on that platform to establish the identity of my software system. So I can call into Kubernetes, for example, and get a service account or I can call into Amazon and get an instance identity document with an IAM role associated with it-- things like that. But those systems only work when I'm running on a particular platform.

And so then the other thing people do is they tend to use other centralized identity systems. So things like OAuth for example, or Kerberos in some older places, tend to come up quite a bit. And those worked reasonably well when everything was connected over a flat network and I had-- they typically worked with HTTP requests and I had relatively small number of services that I was connecting amongst them.

But all of these things rely on the continual availability and uptime of a central identity provider. And if that identity provider is unavailable, even transiently, then it becomes extremely difficult to actually broker authentication or to verify the authentication token between different software systems. And doing that in a distributed way is actually quite difficult.

So all of these forces really added up to us seeing a need for-- and Joe, in fact, seeing a need for-- a new model of workload identity. And it needed to have a few important properties to it.

The first property was it needed to have this Jason Bourne idea of actually the workload itself not needing to know who it was. It couldn't have any ID key, or embedded shared secret, or anything like that embedded into the workload itself when it was configured or deployed. The environment needed to know who the workload was. And it had to be able to verify who the workload was.

The second property that was ideal for something like this is for it to be-- rather than how software systems like Kerberos or OAuth work, where you pass a bearer token between-- a ticket, if you will-- between one workload and another to prove who you are and to authenticate, it was important that we had a system that supported PKI so that if one workload wants to talk to another workload, it can do that without having to share any kind of secrets. You could have a public key and a private key, and keep them separate on the two systems.

ADAM GLICK: Is SPIFFE a spec, a protocol, a product, all of these? What is it?

ANDREW JESSUP: That's a great question. So when Joe started out, SPIFFE was both of these things. And over time as we built out an implementation of what Joe was talking about, we realized pretty quickly that we certainly were building a reference implementation, and later probably "the" reference implementation. But we saw other vendors over time being able to implement the same specification. So in other words, if I'm writing SPIFFE-compliant software, then I should be able to run that software against any vendor that exposes or complies with the SPIFFE specification, not necessarily one that is running a particular piece of software.

And so we ended up splitting it out. There are now two projects. We tend to talk about the two together, but they are separate. SPIFFE is the specification, effectively. It sort of defines things like the format of identities, the format of identity documents, the way that workloads can actually retrieve their identity. How do they actually, in practice, make that "who am I" call and how do they get their documents back? How do they verify the identities of other workloads?

And then there's some newer pieces to the spec that the community is still working on for things like federation, for example. How can different SPIFFE providers talk to each other and mutually establish trust? But fundamentally, SPIFFE is a specification.

And then we have this sister project, if you will, SPIRE-- which stands for the SPIFFE Runtime Environment-- which provides a working implementation of all of these specifications. So that's the software you actually download, and run, and configure.

ADAM GLICK: You've mentioned that a lot of this came from some of the early thinking and work with Joe Beda. Is he still involved in the project?

ANDREW JESSUP: He is, yes. He's been behind the scenes, but enormously helpful with the SPIFFE project. So he formally is a member of our Technical Steering Committee-- our TSC-- which is the governing body that effectively sits between the Cloud Native Computing Foundation, who are now the owners of the project, and the community of contributors. And should there ever be a need for a technical tiebreaker or simply to be able to kind of course correct the direction of the project, the TSC is responsible for ensuring that. And so Joe has one of those seats.

CRAIG BOX: Joe's involvement also lives on in the logo, which you mentioned earlier.

ANDREW JESSUP: It does. Yes, he came up with the logo. And I'll give you a little bit of trivia for your listeners. The logo has two colors-- a sort of green and a sort of cyany-greenish blue. And it turns out the colors-- the way they were chosen is if you look at the hex codes for those-- the HTML hex codes-- one of them is 00BEDA and the other is BEDA00.

ADAM GLICK: Fascinating. That is an interesting little Easter egg there.

ANDREW JESSUP: It is, yeah. If you look at a few of the project-- I won't give away all the Easter eggs, but if you look at some of the projects Joe has been involved with and companies he's started, there's little Easter eggs embedded into most of their logos.

ADAM GLICK: I love it.

CRAIG BOX: So at an Alice and Bob level, if I am workloads running in this cluster, how am I proving who I am in order to get these identities out of SPIFFE and its runtimes?

ANDREW JESSUP: So Alice wants to talk to Bob. Alice calls into the local SPIFFE API and asks, who am I? And the local spiffy API, which is actually node-local-- so it's a UNIX domain socket. It's not talking over IP address or anything like that-- that API returns back and it says, hey, guess what? You're Alice. Here's your SPIFFE ID. Here's your string.

And then also, here's a private key that you hold onto and never leaves you. It never leaves, in fact, the machine that you're running on. And here's also a certificate that corresponds to that key. That certificate has been signed by a central signing authority and the certificate is valid for anything from five minutes to five hours-- whatever makes sense in your environment, but it could be very short.

CRAIG BOX: How does the friendly fisherman on the node know that that is actually Alice?

ANDREW JESSUP: Ah, yeah, so that's a great question. So this is outside the scope of the spec, but it is part of whatever software system implements that spec. So in the case of SPIRE, how that works is we actually have a-- we call it an attestation framework.

So the first thing that happens-- so there's a central server called the SPIRE server that maintains a registry of all of these SPIFFE IDs in a system, as well as all of these attestation policies. That is the list of conditions that need to be proven or that a workload needs to prove that it has in order for a software system to be granted that identity.

And these things tend to have multiple selectors, if you will, multiple different attributes that you want to verify for a workload before issuing it an identity. They may be around the actual infrastructure it's running on. So you know, is it running in a node or a machine that is part of this Kubernetes cluster or is part of this Amazon security group or elastic scaling group? You also want to verify properties of the workload on the machine itself, so maybe you care about the user ID it's running on.

If it's in Kubernetes, maybe you care about the fact that it's running in a pod was in a particular namespace, or has a particular deployment entity associated with it, or something like that. And you might also care about attributes of the binary itself. So you might care, for example, that it's been signed by a CI/CD system or that its content SHA matches something that you know.

CRAIG BOX: So does it depend on Alice claiming, hi, I'm Alice, and then SPIRE verifies this? Or is it more that SPIRE actually has a list of all of the workloads and says, yes, I know that you're Alice.

ANDREW JESSUP: So it's the latter. So the SPIRE infrastructure has a list of all of the workloads and all of the conditions. Alice wakes up in the water not knowing who she is and asks SPIFFE to tell her. And so SPIFFE goes through its list of policies, verifies who Alice is, checks all of these conditions that are necessary. And if it finds a match, then it will say, hey, listen, you're Alice. Here's your ID, and then here's also your private key and certificate.

CRAIG BOX: For many people, their first introduction to SPIFFE will be through Istio, which uses SPIFFE as the identity format and gives out an identity to each workload, which you can then use to do rules or control, for example -- it's the identity used for the mutual TLS. How does SPIFFE relate to Istio and to Envoy?

ANDREW JESSUP: You know, we were fortunate, I think, that SPIFFE and Istio came out of the gates and started evolving at the same time, so that the Istio project was able to benefit from some of the SPIFFE work and then vice versa. When I think about Istio-- this is just one person's view-- is, it's really a consolidation of three ideas. Within Istio, there is this identity infrastructure, which I think in Istio is called Citadel, that is able to do exactly what you said, which is issue identities-- these digital driver's licenses-- to each software system. And then there is a data plan which within Istio is the Envoy project that's able to take those driver's licenses and then use them to actually broker communication, usually via MTLS, between those different software systems.

And then there's a higher level, I guess, above that, which is to simplify might be just called the control plane, which is the thing that maintains all of the configuration of all of these moving parts. And I was also able to do things like consolidate telemetry and pieces like that. So apologies to anyone who's worked with or works on Istio. I probably butchered an explanation there. But at a crude level, you can almost think of it as identity, a data plane, and then control plane sitting on top of that.

So SPIFFE the spec is really solving for the interface between the identity plane and the data plane, as well as for other things. And it's also the project SPIRE you can think of as being very much an analog of the Citadel project in many ways. They're fundamentally trying to solve the same problem. And I think the world we want to get to is one where, whether you're using SPIRE-- to a certain extent SPIRE is interchangeable with other SPIFFE identity providers. And you can choose whichever one makes the most sense for you, which I suspect for most of your users will certainly be Citadel.

But more importantly, that those same SPIFFE identities that are being issued by Citadel, or SPIRE or whatever it might happen to be-- those same identities then can be consumed not just by, say, Envoy, but also by other software systems or other applications where identity might be important. So that might be signing something and sticking it on a queue, for example. Or it might be doing secure introduction to a secret store, or talking to an API gateway, or something like that. So at a crude level, that's really how we look at the differences between them.

CRAIG BOX: If SPIRE is the reference implementation and Citadel is the Istio implementation of that, are you seeing uptake in other products building their own implementations of SPIFFE?

ANDREW JESSUP: Yeah, so there's a few projects now that are implementing SPIFFE either as identity providers themselves or as consumers. So some of them aren't live yet, but for ones that are talking about what they're doing, there's a hash of core products, for example, are becoming SPIFFE identity providers too. I believe Consul does it. Vault can now also be configured to accept SPIFFE identities-- so to be a SPIFFE consumer. Envoy works really well with SPIFFE identities, whether with Istio or without. That's a great fit.

And there are a few other secret stores as well. So for example, Pinterest has a secret stored called Nox that accepts SPIFFE IDs. Square, who are very active SPIFFE users, have a proxy project called Ghosttunnel, which is really interesting and has been modified to work really well as SPIFFE. And there's a few other projects as well. If you go to spiffe.io, you'll see an ever-growing list of who are working with us.

ADAM GLICK: Was Scytale founded to solve these identity challenges?

ANDREW JESSUP: Yes. Absolutely. When we founded Scytale, the big idea that we had was, we saw this emerging world of service mesh. We also saw a whole set of other challenges that people were hitting when they tried to stand up new software systems in the cloud. Another big problem people have actually, is just being able to connect back and authenticate back to your existing software systems running maybe in a different cloud or on-prem. And particularly, if you live in a world where you're using something like Kerberos, for example. This is actually really difficult to do today with something like Kubernetes. And then there's other challenges as well.

And we saw all of these could be solved once you have a really strong idea of who your workload is-- once you have this identity infrastructure in place. A lot of other things suddenly become a lot simpler. It's one of the reasons that the precedent for SPIFFE, LOAS at Google, and things like Metatron at Netflix have been so resilient and so popular as pieces of infrastructure. It's because they turned out to be such pivotal building blocks on top of other things.

So our goal as a company, really was to provide the best building block that you could build other solutions on top of. So whether you're building a service mesh, or whether you need to do something like identity translation back to existing infrastructure, we can help you get there and help you solve for those problems and do so in a way that's very much future-proof.

CRAIG BOX: As founders, did you see SPIFFE and say, we want to build a company around that? Or did you see the problem and then go out and say, how can we solve this problem?

ANDREW JESSUP: We did both in some ways. We'd spent quite a bit of time talking to companies about the challenges they'd been having as a product manager. In Google, I'd been doing that for a while. My co-founder, Sunil, as a product manager and later as a venture partner had spent a lot of time talking to organizations about some of the challenges they were finding, principally around moving to cloud and then being successful on the cloud.

And what we aligned on was that moving to cloud-- the reason people do it is because they want to move faster. And we found people could move faster when they were building new software systems entirely on the cloud for a while. But then they would hit a certain tipping point of scale where that was becoming difficult. And then even before that, they were hitting a problem where they still needed to be able to talk back to existing infrastructure and existing software systems. And they were having a lot of trouble doing that too.

And so there's an enormous ecosystem of companies and technologies that are solving parts of those problems. But the one thing we didn't see anyone really solving for was this problem of identity and distributed authentication. So that's really where we saw the need and the opportunity.

CRAIG BOX: You've just launched an enterprise product as well. How does this differ from what we've talked about so far?

ANDREW JESSUP: Certainly, yeah. So we saw the SPIFFE projects and SPIRE projects really as sort of seed crystals. They are really useful primitives upon which you can build really interesting systems and solve really interesting problems. As a service mesh, for example, they're just one of those systems you can build. But there are certainly others.

So what we wanted to do with Scytale enterprise was really two things. First, at the end of the day, what we wanted people to be able to do is to have SPIFFE identities everywhere they needed them-- whether that was in Kubernetes, whether that was on-premise, whether that was running in the cloud in various different forms. We wanted to make sure you could have SPIFFE identities everywhere so that if you wanted to build interesting systems on top of them, then you can, and that you have the right tooling to do that. SPIRE certainly gets you a long way there, but what Scytale enterprise gives you on top of that are frankly a lot of the things that enterprise companies and security teams care about-- so ordered controls, a nice UI, obviously support, and also the ability to be able to scale more efficiently over even more heterogeneous environments.

The second thing that Scytale enterprise does is this thing we call identity translation. So what this is, is a case where typically, I have existing identity providers running inside my infrastructure. And they're usually sitting in a data center somewhere. So these are things like Active Directory, or things like CyberArk, or things like Venafi, for example.

These software systems, in order for something running in the cloud to talk to something running on-prem, they need to be able to talk firstly to these existing identity providers in order to get the credentials they need. And that turns out to be a really difficult thing to do in practice. And it turns out SPIFFE and SPIRE are really good building blocks for solving for that problem. Because once you've been able to identify something running, say, in cloud and/or in Kubernetes, then you can use a SPIFFE ID to start to do identity exchange back to the existing on-premise system.

So the other big capability, I guess, I get out of Scytale enterprise is that ability to be able to take identities running in things like Active Directory, and in things like CyberArk and being able to get those identities into the cloud, to my Kubernetes-hosted workloads, to my cloud-hosted workloads when I need to, without needing to worry too much about how it's done. And doing it in a way that a security team, when they looked at the solution, wouldn't have problems with it. That doesn't require you to do things like deploy hard-coded credentials along with your workload. Or it doesn't require you to open up ports from your Kubernetes pod run by some development team directly to Active Directory running on-premise, which is often quite difficult to do so.

ADAM GLICK: Why is now the right time to commercialize?

ANDREW JESSUP: Now I think is the right time because your organization is really hitting a tipping point in terms of cloud. We've been seeing more and more organizations-- they've gone past the first steps in moving things to cloud. They've gone past trying it out and seeing that it can work for them. They've crossed proof-of-value, if you will. And now they're starting to do serious migrations and build serious systems on cloud.

And this is really where the identity challenge is coming up headstrong, particularly with Kubernetes, actually. How do I bridge all the existing worlds together with the new ones? And so the tipping point feels like it's there. It's still very much an unsolved problem. And it's becoming increasingly a blocker for folks as they run more and more stuff on the cloud and need to build more and more interesting experiences.

For SPIFFE, it's also the right time, I think, because what SPIFFE is really trying to do at the end of the day is to provide a clear dividing line between the infrastructure that gives you identity, which might be SPIRE, might be Scytale, might even be one day, cloud platform providers themselves. And the things that consume identity, whether that's a service mesh, authentication libraries, or middleware, or other pieces. So we think now is a good time, as this ecosystem is starting to grow and mature, to start drawing some lines in the sand there as to how these different systems should work.

CRAIG BOX: Is SPIFFE quote, unquote, "1.0?"

ANDREW JESSUP: It's getting there. It's getting there. We haven't called it 1.0 yet. And there's a few significant functional pieces we'd like to land. Probably, the biggest one is this idea of federation. So we talked about the idea of having several different SPIFFE identity providers working together. So maybe I have Citadel issuing SPIFFE IDs somewhere. And I have SPIRE issuing SPIFFE IDs somewhere. And again, maybe the cloud platform providers or Consul are issuing SPIFFE IDs somewhere.

And it would be wonderful if I had software systems that were running on each of these things-- that they could use their SPIFFE IDs to establish trust and to authenticate to a software system that's being managed by a different SPIFFE identity provider. And so this is what federations are really designed to solve for. It allows in the background-- the workload doesn't have to do anything. The software system doesn't have to do anything. But it allows different providers of SPIFFE IDs to be able to peer with each other and to mutually establish trust between each other. And then there's a few semantic nuances that we're working through with SPIFFE as well. But it's getting pretty close.

ADAM GLICK: All right. Andrew, thank you so much for your time today.

ANDREW JESSUP: Not a problem. Thanks so much for having me on.

CRAIG BOX: You can find Andrew on Twitter @whenfalse. Can you find him on Twitter when he's true?


ANDREW JESSUP: There's no such date.

CRAIG BOX: Or you can find the SPIFFE project @spiffe.io or at the website, spiffe.io. And you can find Scytale at scytale.io, with a C.


ADAM GLICK: Thank you for listening. As always, if you enjoyed today's show, please help us spread the word and tell a friend. If you have any feedback for us, you can find us on Twitter @kubernetespod, or reach us by email at kubernetespodcast at google.com.

CRAIG BOX: You can also check out our website at kubernetespodcast.com, finding all the links and our show notes and transcripts of all our episodes. Until next time, take care.

ADAM GLICK: Catch you next week.