πCreds: Privately Inferred Credentials

So I'll give a lightning talk on some research that, my group has been working on. Released this paper on archive last month.

So the space we're in is this privacy-preserving credential space, and it's a somewhat crowded space, you could say. There's been quite a bit of work, academic work, that's come out of, IC3, among other people, that looks at this. But the problem we have here is that these credentials that we're working with in these systems are very limited to simple predicates. So if Alice has some structured data that encodes her birthday, maybe she can generate a privacy-preserving credential that proves that she's over eighteen. But for better or for worse, the data that exists about us online is a lot more than just this kind of structured data like our birthdays. We have, like, shopping data. Like, what orders have I made on Amazon?

What medical documents are there online through hospital portals? And in this scenario, like, maybe from Alice's Amazon order history, I could tell that she's a coffee expert if she's made lots of purchases of espresso machines, for instance. But so the missing gap is this semantic reasoning over unstructured data, which is actually something we can address. We know that LLMs are pretty good at this. So we propose this new architecture, which we're calling PiCreds, in which we essentially run an oracle and an LLM inside a TEE to issue credentials of this type. And so a user is able to provide their credentials to the TEE. Within the TEE, an oracle goes out, fetches the user's private data, and inputs it to an LLM along with some prompt, and that issues a credential.

And so what this looks like is a pretty typical credential. You have the fields indicating who issued it, who the credential is referring to. But instead of just, like, these basic attributes, you have a prompt fed to the LLM that's parametrized by these sources. So you know where the data is coming from. You have this provenance guarantee, and you know what the LLM is trying to tell you about that user. And then in terms of trust, you verify the, that the enclave has signed it and that the attestation checks out. So within this threat model, there are kind of two interesting things I want to call attention to, which arise from the use of LLMs in this way, and I think they're pretty interesting things that have to do with the security of LLM applications broadly.

The first is, like, what can an adversarial user do in this setting where they're trying to, generate a credential for themselves? Well, we have to acknowledge the possibility that a user that gets some credential, for instance, based on their Amazon order history, is able to then make these adversarial purchases to boost their expertise in a way that is inaccurate. And so this is similar to the problem of adversarial examples or prompt injections, but it looks a bit different because here in this setting we're working with this data where we have provenance guarantees, and so an adversary can't just post whatever they want as a transaction, for instance, but have has to work in this kind of source-constrained setting.

So looks like kind of similar to adversarial ML research, but different. The other thing I would want to highlight is this adversarial verifier case where if someone can fine-tune the model that's being used for this pipeline, potentially they can leak information about users. So for instance, we want this credential to indicate the expertise of users, but here the verifier fine-tunes the expertise score to be even or odd based on the demographics of the user and leaks unintentional information. And so I think these are both kind of threat models that we'll have to reckon with more as we integrate LLMs into applications of this type. One especially cool application I want to highlight of this system is what we're calling, trustless code audits.

So essentially, you can issue credentials over code. And so how you do that is by generating a credential that binds the hash of some code to some property over it. And then if a company wants to run that code in a proprietary way on some TEE, a user can interact with that and get some assurance about its properties without ever inspecting the source through this flow. And so we're pretty excited about this for interacting with, like, proprietary agents, for instance. I'll leave you with, this, which I came across last week, which was, some agents on Molpuk came across the paper and are discussing it there. So I'm not sure we have any citations yet, but it's, being discussed where it matters, I guess.

I was just fanboying. My, statement was more that, we have I have an active use case that I'm that I was gonna, like, ugly reverse engineer Attested Audits onto, so maybe this would be great to use. And the, setting was, block building related. So there's some algorithm I want to trust. The algorithm itself is, like, valuable information, and so just attesting to it in an open source TEE setting would kind of, like, be suboptimal. Ideally, I just, you know, I trust the properties of the algorithm, but I don't know how it's been optimized. And that's exactly what, this would work for. I think the real question is, like, how good, how good can we make this auditing?

And in our case, the challenge is also that the security properties, are hard to state, like, formally. And I don't know if you guys have made progress on those.

No, that's a great point. This idea came up kind of last minute in the late stages of the project, so we haven't looked too much into, like, these formal properties. But there's some really cool work out there that I want to look into that actually does try to use LLMs in this way to get more formal guarantees of auditing. And in particular, like, tries to make LLMs audit code more along the lines of how humans do. So, working through, like, control flow in the program and things like that, rather than just sending the whole, image to the LLM. So I think it's really cool because, like, as the models get better, right, you would expect this use case to be more and more valuable.

And, who knows if that ends up being from, like, formal guarantees that people figure out or just from, like, the models getting ridiculously good.