From af1a869de9e0573da1b61c8b46621e79b88f039f Mon Sep 17 00:00:00 2001 From: Stefan Bethke Date: Tue, 21 Jul 2026 21:24:30 +0200 Subject: [PATCH] Add exporting individual attributes Based off #32 Also: * rework README.md * improve configuration error messages --- README.md | 117 +++++------------- .../attribute_endpoints/AdminUiPage.java | 47 +++---- .../AttributeEndpointsResourceProvider.java | 49 ++++++-- ...ibuteEndpointsResourceProviderFactory.java | 35 +++--- ...ttributeEndpointsResourceProviderTest.java | 8 -- testing/import/testing.json | 4 +- 6 files changed, 118 insertions(+), 142 deletions(-) diff --git a/README.md b/README.md index 9cd3766..c1f0bd3 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,38 @@ -# Attribute Endpoints Provider +# Attribute Endpoints Provider - Export User Attributes -This is a Keycloak Provider that exports an anonymized list of user profile attribute values. -For this it will provide API endpoints for every configured attribute-group. -The configuration of the provider is possible via an admin page. +This is a Keycloak Provider that exports user profile attribute values. +The selection of attributes and authorization is manage through an endpoint configuration. -Every endpoint responds with a list of all attribute values, that: -- are in the attribute group matching `attribute-group` -- match an optional RegEx Pattern `attribute-regex` -- belong to a user with a role matching `match-role` -- are non-empty +## Usage + +The provider adds an object type "Attribute Endpoints" to the Keycloak admin website. +You configure the provider by creating one or more endpoint configurations. + +Each endpoint configuration requires: +- a name that is the reference in the endpoint URL (slug) +- the name of a role that the calling user must have to be allowed to query this endpoint +- the name of an attribute or an attribute group that should be exported +- the name of a role that users must have to be included in the list +- an optional regular expression that must match to have a value included + +> **Note** The attribute (group) and the roles need to exist before you can create the endpoint configuration. + +There are two endpoints returning JSON: +- `export/slug`: collate all attribute values into a single list +- `export/slug/map`: produce a map of lists, with the key being the attribute name + +No other data is included in the response, in particular, there is no information on which attribute value is associated with which user. + +User Attribute values can be single or multi-value; the resulting list will include them as a flattened list. + +> **⚠️ Note** The authorization and the selection of the user attributes are not tied together in any way. +> Every user that has the matching role will have all the attributes included that are specified in the endpoint configuration. +> In other words, if you are exporting an attribute group, all attributes will be included. + +> **⚠️ Note** Keycloak has no concept of authorization for individual user attributes (for example, based on assigned roles). +> Users will be able to see and edit any attribute that has been made available to users, irrespective of whether a user would +> be included in an endpoint configuration export or not. -Multivalue attributes are flattened in the response. ## Building @@ -20,79 +42,8 @@ Once all dependencies are met, simply call `make` to build the provider, which s There's also `make clean` available for removing the output directory. +To add the provider to your Keycloak install, copy `attribute-endpoints-provider/target/attribute-endpoints-provider-1.0-SNAPSHOT.jar` to the Keycloak Provider directory (`/opt/keycloak/providers/`). + ## Testing Setup See [testing/README.md](testing/README.md) for details on the Docker Compose based setup that includes a realm and the ability to attach a debugger to the running Keycloak. - -## Example Setup - -We assume an unconfigured, fresh Keycloak installation running under `http://localhost:8080`. -(This can be achieved by running the provided `compose.yaml` after building the provider as outlined in [Building](#building).) - - 1. Add a new realm - e.g. "TestRealm" - 2. Under `Realm Settings > User profile > Attributes Group`, add a new attribute Group - Example: - - `Name` = `"my-attributes-group"` - - `Display name` = `"Endpoint Attributes"` - - `Display description` = `"Attributes exported by the provider."` - 3. Under `Realm Settings > User profile > Attributes`, add a new attribute - Example: - - `Attribute [Name]` = `"ssh-keys"` - - `Display name ` = `"SSH Keys"` - - `Multivalued` = `On` - - `Attribute group` = `"my-attributes-group"` - - `Who can edit?` = `user, admin` - - `Validators` - You can add validators, which will limit what values the user can enter. These validators are ignored by the provider. - 4. Under `Realm roles`, add two new roles - Example: - 1. `Role name` = `"myattribute-match"` - 2. `Role name` = `"myattribute-export"` - 5. Under `Users`, add a new user - Example: - - `Username` = `"user"` - - `Email` = `"user@example.com"` - - `First name` = `"User"` - - `Last name` = `"User"` - - `SSH Keys` = `"example-value-1", "example-value-2"` - 6. In the Settings of the newly created user, go to `Role mapping > Assing role > Realm roles` and check the role `myattribute-match` - 7. create a second user to use the provider - - `Username` = `"bot-user"` - - `Email` = `"bot@example.com"` - - `First name` = `"Bot"` - - `Last name` = `"Bot"` - - After creating: - - give it the role `myattribute-export` - - set a password in the users settings `Creadentials > Set password`. For Example `"password"` -8. Under `Attribute Endpoints > Create item`, add a new endpoint to the provider - Example: - - `Slug` = `"ssh_keys"` - - `Attribute Group` = `"my-attributes-group"` - - `Match Role` = `"myattribute-match"` - - `Auth Role` = `"myattribute-export"` - - `Attribute RegEx` = `".*"` -9. Aquire an OIDC Access Token: - ```shell - curl --request POST \ - --url http://localhost:8080/realms/TestRealm/protocol/openid-connect/token \ - --header 'content-type: application/x-www-form-urlencoded' \ - --data scope=openid \ - --data username=bot-user \ - --data password=password \ - --data grant_type=password \ - --data client_id=admin-cli - ``` -10. copy the value of the response key `access_token` and use it in a second request: - ```shell - curl --request GET \ - --url http://localhost:8080/realms/TestRealm/attribute-endpoints-provider/export/ssh_keys \ - --header 'authorization: Bearer ey...' \ - --header 'content-type: application/json' - ``` -11. You should get a response like this: - ```json - ["example-value-1","example-value-2"] - ``` - -Although this example uses a simple bot account to authenticate to Keycloak, we recommend using a client with service account, when using this provider programmatically. diff --git a/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AdminUiPage.java b/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AdminUiPage.java index cfd26b6..7c88e49 100644 --- a/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AdminUiPage.java +++ b/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AdminUiPage.java @@ -1,15 +1,12 @@ package de.ccc.hamburg.keycloak.attribute_endpoints; -import java.util.List; -import java.util.regex.Pattern; - +import com.google.auto.service.AutoService; import org.keycloak.Config; import org.keycloak.component.ComponentModel; import org.keycloak.component.ComponentValidationException; import org.keycloak.models.KeycloakSession; import org.keycloak.models.KeycloakSessionFactory; import org.keycloak.models.RealmModel; -import org.keycloak.models.RoleModel; import org.keycloak.provider.ProviderConfigProperty; import org.keycloak.provider.ProviderConfigurationBuilder; import org.keycloak.representations.userprofile.config.UPConfig; @@ -17,7 +14,8 @@ import org.keycloak.services.ui.extend.UiPageProvider; import org.keycloak.services.ui.extend.UiPageProviderFactory; import org.keycloak.userprofile.UserProfileProvider; -import com.google.auto.service.AutoService; +import java.util.List; +import java.util.regex.Pattern; /** * Implements UiPageProvider to show a config page in the admin @@ -43,6 +41,7 @@ public class AdminUiPage implements UiPageProvider, UiPageProviderFactory g.getName().equals(configAttributeGroup))) { + } else if (upconfig.getAttributes().stream().noneMatch(a -> + a.getName().equals(configAttributeGroup) + || (a.getGroup() != null && a.getGroup().equals(configAttributeGroup)))) { hasError = true; - errorString += " • [Attribute Group] does not exist\n"; + errorString += " • [Attribute Group] no matching Attribute or Attribute Group\n"; } String configAttributeRegex = model.getConfig().getFirst("attribute-regex"); - Boolean regexIsBlank = configAttributeRegex == null; + boolean regexIsBlank = configAttributeRegex == null; if (!regexIsBlank) { try { @@ -113,37 +114,37 @@ public class AdminUiPage implements UiPageProvider, UiPageProviderFactory userList = ctx.users.toList(); + + Map> attributeMap = ctx.attributeNames.stream() + .collect(Collectors.toMap( + attributeName -> attributeName, + attributeName -> userList.stream() + .flatMap(user -> user.getAttributeStream(attributeName)) + .filter(attribute -> !attribute.isEmpty()) + .filter(ctx.filter::matches) + .toList())); + + return Response.ok(attributeMap).build(); + } + + /** * Resolves and validates the configuration and request state needed to export attribute * values for a given slug, exposing the results as member variables. @@ -87,37 +117,38 @@ public class AttributeEndpointsResourceProvider implements RealmResourceProvider .toList(); if (componentList.isEmpty()) { - throw new NotFoundException("Endpoint not found."); + throw new NotFoundException("Attribute Endpoint " + slug + " not found"); } if (componentList.size() > 1) { throw new NotFoundException( - "Endpoint Configuration Error - Multiple configurations exist for this endpoint."); + "Attribute Endpoint Configuration Error - Multiple configurations exist for " + slug); } ComponentModel component = componentList.get(0); RoleModel authRole = realm.getRole(component.getConfig().getFirst("auth-role")); if (authRole == null) { - throw new ServerErrorException("Endpoint Configuration Error - auth-role does not exist.", 500); + throw new ServerErrorException("Attribute Endpoint " + slug + " Configuration Error - auth-role does not exist.", 500); } RoleModel matchRole = realm.getRole(component.getConfig().getFirst("match-role")); if (matchRole == null) { - throw new ServerErrorException("Endpoint Configuration Error - match-role does not exist.", 500); + throw new ServerErrorException("Attribute Endpoint " + slug + " Configuration Error - match-role does not exist.", 500); } upconfig = session.getProvider(UserProfileProvider.class).getConfiguration(); configAttributeGroup = component.getConfig().getFirst("attribute-group"); - if (upconfig.getGroups().stream().noneMatch(g -> g.getName().equals(configAttributeGroup))) { - throw new ServerErrorException("Endpoint Configuration Error - attribute-group does not exist.", 500); + if (upconfig.getGroups().stream().noneMatch(g -> g.getName().equals(configAttributeGroup)) && + upconfig.getAttributes().stream().noneMatch(a -> a.getName().equals(configAttributeGroup))) { + throw new ServerErrorException("Attribute Endpoint " + slug + " Configuration Error - no attribute or attribute group named " + configAttributeGroup + " found", 500); } try { filter = new RegExFilter(component.getConfig().getFirst("attribute-regex")); } catch (Exception e) { throw new ServerErrorException( - "Endpoint Configuration Error - attribute-regex is not a valid regex pattern.", 500); + "Attribute Endpoint " + slug + " Configuration Error - attribute-regex is not a valid regex pattern.", 500); } authUser = AttributeEndpointsResourceProvider.getAuth(session).getUser(); @@ -129,7 +160,9 @@ public class AttributeEndpointsResourceProvider implements RealmResourceProvider // select all attributes that match configAttributeGroup, or that are in a group that matches configAttributeGroup attributeNames = upconfig.getAttributes() .stream() - .filter(a -> a.getGroup() != null && a.getGroup().equals(configAttributeGroup)) + .filter(a -> + a.getName().equals(configAttributeGroup) + || (a.getGroup() != null && a.getGroup().equals(configAttributeGroup))) .map(UPAttribute::getName) .toList(); diff --git a/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderFactory.java b/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderFactory.java index e4dc476..0c84a4b 100644 --- a/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderFactory.java +++ b/attribute-endpoints-provider/src/main/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderFactory.java @@ -1,36 +1,35 @@ package de.ccc.hamburg.keycloak.attribute_endpoints; +import com.google.auto.service.AutoService; import org.keycloak.Config; import org.keycloak.models.KeycloakSession; import org.keycloak.models.KeycloakSessionFactory; import org.keycloak.services.resource.RealmResourceProvider; import org.keycloak.services.resource.RealmResourceProviderFactory; -import com.google.auto.service.AutoService; - @AutoService(RealmResourceProviderFactory.class) public class AttributeEndpointsResourceProviderFactory implements RealmResourceProviderFactory { static final String PROVIDER_ID = "attribute-endpoints-provider"; @Override - public RealmResourceProvider create(KeycloakSession keycloakSession) { - return new AttributeEndpointsResourceProvider(keycloakSession); - } + public RealmResourceProvider create(KeycloakSession keycloakSession) { + return new AttributeEndpointsResourceProvider(keycloakSession); + } - @Override - public void init(Config.Scope config) { - } + @Override + public void init(Config.Scope config) { + } - @Override - public void postInit(KeycloakSessionFactory keycloakSessionFactory) { - } + @Override + public void postInit(KeycloakSessionFactory keycloakSessionFactory) { + } - @Override - public void close() { - } + @Override + public void close() { + } - @Override - public String getId() { - return PROVIDER_ID; - } + @Override + public String getId() { + return PROVIDER_ID; + } } diff --git a/attribute-endpoints-provider/src/test/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderTest.java b/attribute-endpoints-provider/src/test/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderTest.java index 7458a1f..11dd3a2 100644 --- a/attribute-endpoints-provider/src/test/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderTest.java +++ b/attribute-endpoints-provider/src/test/java/de/ccc/hamburg/keycloak/attribute_endpoints/AttributeEndpointsResourceProviderTest.java @@ -112,7 +112,6 @@ public class AttributeEndpointsResourceProviderTest { } } - /* NOTYET @Test public void exportAttributeValuesMap_doorisSlug_returnsSshKeysPerAttribute() { try (Response response = callAuthenticatedAs("export-dooris-ssh-keys", @@ -125,9 +124,7 @@ public class AttributeEndpointsResourceProviderTest { response.getEntity()); } } - */ - /* NOTYET @Test public void exportAttributeValues_mailingListChaosSlug_returnsAddressesOfAllMatchingUsers() { try (Response response = callAuthenticatedAs("export-mailing-list-addresses", @@ -138,9 +135,7 @@ public class AttributeEndpointsResourceProviderTest { assertEquals(List.of("hacker+chaos@example.net", "tester+chaos@example.com"), response.getEntity()); } } - */ - /* NOTYET @Test public void exportAttributeValues_mailingListInternSlug_returnsAddressOfSingleMatchingUser() { try (Response response = callAuthenticatedAs("export-mailing-list-addresses", @@ -151,9 +146,7 @@ public class AttributeEndpointsResourceProviderTest { assertEquals(List.of("hacker+intern@example.net"), response.getEntity()); } } - */ - /* NOTYET @Test public void exportAttributeValuesMap_mailingListChaosSlug_returnsAddressesPerAttribute() { try (Response response = callAuthenticatedAs("export-mailing-list-addresses", @@ -166,7 +159,6 @@ public class AttributeEndpointsResourceProviderTest { response.getEntity()); } } - */ @Test public void exportAttributeValues_unknownSlug_throwsNotFound() { diff --git a/testing/import/testing.json b/testing/import/testing.json index dfa0fa7..363df47 100644 --- a/testing/import/testing.json +++ b/testing/import/testing.json @@ -1605,7 +1605,7 @@ "subComponents" : { }, "config" : { "match-role" : [ "mailing-list-intern-member" ], - "attribute-group" : [ "mailing-list-addresses" ], + "attribute-group" : [ "mailing-list-address-intern" ], "auth-role" : [ "export-mailing-list-addresses" ], "slug" : [ "mailing-list-addresses-intern" ] } @@ -1626,7 +1626,7 @@ "config" : { "match-role" : [ "mailing-list-chaos-member" ], "auth-role" : [ "export-mailing-list-addresses" ], - "attribute-group" : [ "mailing-list-addresses" ], + "attribute-group" : [ "mailing-list-address-chaos" ], "slug" : [ "mailing-list-addresses-chaos" ] } } ]