Keystone integration with IDM
Keystone integration with IDM
Overview
This page describes a possible Keystone and Red Hat IdM integration scenario. There are many possible ways to integrate the two technologies. This scenario meets the following requirements:
- Keystone Accounts, Tenants, and Roles are stored in the Red Hat IdM database.
- User accounts are managed by Red Hat IdM.
- Tenants and Roles are managed by Keystone.
- The Keystone service catalog is stored in MySQL.
Roles and privileges are created so that a Keystone administrator (or service) can not inadvertently write to objects in the standard Red Hat IdM tree.
Example Architecture
In this example, two systems will be used. The first (ipa01
) is an IdM master server. Information on initial installation and configuration of the IdM software is here:
The second system (keystone
) is a RDO Keystone server. Information on the initial installation and configuration of Keystone is here:
This guide assumes a manual installation of RDO and its services (i.e. not using packstack).
IdM Preparation
Either install Red Hat IdM from scratch or start with an existing Red Hat IdM directory. In this example, we have an existing directory rooted at dc=example,dc=com
User accounts are stored in cn=users,cn=accounts,dc=example,dc=com
as per the default.
Install the Keystone schema
Note: the following assumes that you want to manage assignment data in LDAP. In the Havana and later releases, the Identity backend only contains user and group information. Projects, roles, role assignments, and domains are all stored in the Assignment backend. The preferred approach is to store assignments in SQL, not in LDAP. Thus, the following custom schema is not necessary.
Keystone will pull user account information from cn=users,cn=accounts
, but we’ll configure it to store Tenants and Roles in cn=openstack
. This allows us to grant permission to that tree while restricting permissions to the standard tree. The following schema needs to be imported to support Keystone:
dn: cn=openstack,dc=example,dc=com
objectClass: top
objectClass: nsContainer
cn: openstack
dn: cn=enabled_users,cn=openstack,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
cn: enabled_users
dn: ou=tenants,cn=openstack,dc=example,dc=com
objectClass: top
objectClass: organizationalUnit
ou: tenants
dn: cn=enabled_tenants,cn=openstack,dc=example,dc=com
objectClass: groupOfNames
objectClass: top
cn: enabled_tenants
dn: ou=roles,cn=openstack,dc=example,dc=com
objectClass: top
objectClass: organizationalUnit
ou: roles
The enabled_users
and enabled_tenants
groups are used to emulate the “enabled” attribute, since it is unsupported in Red Hat IdM’s default schema. It is possible to use any groupOfNames for the “enabled emulation” feature. Implementations may wish to use an existing group in IdM instead. For example, using the cn=ipausers,cn=groups,cn=accounts,dc=example,dc=com
group will automatically “enable” all IdM users in Keystone.
To add the schema to the Red Hat IdM database, first save the LDIF above into a text file in root’s home directory on ipa01
. Edit the file, replacing dc=example,dc=com
with the correct domain suffix. Then load the schema with ldapadd
:
[admin@ipa01 ~]$ ldapadd -x -D"cn=Directory Manager" -W < openstack.ldif
You will be prompted for the Directory Manager password.
Next, create an “OpenStack Administrator” role with permission to edit that subtree. Before running these or any other ipa
commands, authenticate to IdM as a user which has administrative privileges.
[admin@ipa01 ~]$ ipa role-add --desc="OpenStack Administrator" "OpenStack Administrator"
[admin@ipa01 ~]$ ipa permission-add "Manage OpenStack Tenants and Roles" \
--subtree="ldap:///cn=openstack,dc=example,dc=com" \
--permissions=write,add,delete \
--attrs=member,roleOccupant
[admin@ipa01 ~]$ ipa privilege-add "Manage OpenStack" --desc "Manage OpenStack Tenants and Roles"
[admin@ipa01 ~]$ ipa privilege-add-permission "Manage OpenStack" \
--permissions="Manage OpenStack Tenants and Roles"
[admin@ipa01 ~]$ ipa role-add-privilege --privileges "Manage OpenStack" "OpenStack Administrator"
Last, create a group named osadmins
with the role that was created above.
[admin@ipa01 ~]$ ipa group-add --desc="OpenStack Admins" osadmins
[admin@ipa01 ~]$ ipa role-add-member --groups=osadmins "OpenStack Administrator"
Creating the OpenStack administrator account
User accounts in OpenStack will be drawn from the IdM user list. The following steps will create an initial administration account to use in creating the initial Keystone Tenants, Roles, and Service Catalog. An existing administrator or human account could be used as well - add the account to the osadmins
group to inherit the “OpenStack Administrator” role which grants write access to the subtree.
[admin@ipa01 ~]$ ipa user-add
First name: RDO
Last name: Administrator
User login [radministrator]: rdoadmin
---------------------
Added user "rdoadmin"
---------------------
[admin@ipa01 ~]$ ipa user-mod --email=rdoadmin@example.com rdoadmin
[admin@ipa01 ~]$ ipa passwd rdoadm
[admin@ipa01 ~]$ ipa group-add-member --users=rdoadmin osadmins
Next, add the user to the enabled_users
group in the cn=openstack
tree. Use the account which was added to the osadmins
group to verify that the privilege was inherited correctly:
[admin@ipa01 ~]$ ldapmodify -x -D"uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com" -W <<EOF
> dn: cn=enabled_users,cn=openstack,dc=example,dc=com
> changetype: modify
> add: member
> member: uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com
>
> EOF
Enter LDAP Password:
modifying entry "cn=enabled_users,cn=openstack,dc=example,dc=com"
If the ldapmodify
command does not succeed, troubleshoot the role, permission, and privilege created in the first step.
If using an existing IdM group to determine whether or not an account is “enabled” for OpenStack, the administrative user will need to be added to that group instead of cn=enabled_users,cn=openstack
.
Keystone Configuration
Creating a keystone.conf
Install the Keystone software, create the MySQL database, create the service token, and populate the MySQL database using the instructions provided here:
http://docs.openstack.org/grizzly/openstack-compute/install/yum/content/install-keystone.html
After running keystone-manage db_sync
, edit /etc/keystone/keystone.conf
. In the identity
section, comment out the SQL Identity driver and uncomment the LDAP driver:
[identity]
#driver = keystone.identity.backends.sql.Identity
driver = keystone.identity.backends.ldap.Identity
If you wish to store assignments in the LDAP store, configure the ldap
section to use the Red Hat IdM LDAP database.
[ldap]
url = ldap://ipa01.example.com
user = uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com
password = <password>
suffix = cn=openstack,dc=example,dc=com
use_dumb_member = False
allow_subtree_delete = False
# dumb_member = cn=dumb,dc=example,dc=com
# Maximum results per page; a value of zero ('0') disables paging (default)
# page_size = 0
# The LDAP dereferencing option for queries. This can be either 'never',
# 'searching', 'always', 'finding' or 'default'. The 'default' option falls
# back to using default dereferencing configured by your ldap.conf.
# alias_dereferencing = default
# The LDAP scope for queries, this can be either 'one'
# (onelevel/singleLevel) or 'sub' (subtree/wholeSubtree)
# query_scope = one
user_tree_dn = cn=users,cn=accounts,dc=example,dc=com
# user_filter =
user_objectclass = person
#user_domain_id_attribute = businessCategory
user_id_attribute = uid
user_name_attribute = uid
user_mail_attribute = mail
# user_pass_attribute = userPassword
# user_enabled_attribute = enabled
# user_enabled_mask = 0
# user_enabled_default = True
user_attribute_ignore = tenant_id,tenants
user_allow_create = False
user_allow_update = False
user_allow_delete = False
user_enabled_emulation = True
user_enabled_emulation_dn = cn=enabled_users,cn=openstack,dc=example,dc=com
tenant_tree_dn = ou=tenants,cn=openstack,dc=example,dc=com
# tenant_filter =
# tenant_objectclass = groupOfNames
tenant_domain_id_attribute = businessCategory
tenant_id_attribute = cn
tenant_member_attribute = member
# tenant_name_attribute = ou
tenant_desc_attribute = description
# tenant_enabled_attribute = enabled
tenant_attribute_ignore =
# tenant_allow_create = True
# tenant_allow_update = True
# tenant_allow_delete = True
tenant_enabled_emulation = True
tenant_enabled_emulation_dn = cn=enabled_tenants,cn=openstack,dc=example,dc=com
role_tree_dn = ou=roles,cn=openstack,dc=example,dc=com
# role_filter =
# role_objectclass = organizationalRole
# role_id_attribute = cn
# role_name_attribute = ou
# role_member_attribute = roleOccupant
role_attribute_ignore =
# role_allow_create = True
# role_allow_update = True
# role_allow_delete = True
Change the above suffixes to reflect the IdM installation (i.e. dc=example,dc=com). Note that this configuration uses the rdoadmin
account created above to authenticate Keystone to IPA. If a different account was created for that purpose above, change the user
and password
value in the configuration file.
If you wish to only use Users and Groups from IPA in a read only manner, and manage assignments in SQL the configuration file can be significantly simplified. Since LDAP is read only, you don’t have to provide a user or password for the admin: it will do an anonymous bind instead. Since a user added from Keystone would not be a valid IPA user, a user should only be added via the IPA API.
[assignment]
driver = keystone.assignment.backends.sql.Assignment
[identity]
driver = keystone.identity.backends.ldap.Identity
[ldap]
url = ldap://ipa01.example.com
user_tree_dn=cn=users,cn=accounts,dc=example,dc=com
user_id_attribute=uid
group_tree_dn=cn=groups,cn=accounts,dc=example,dc=com
Here are some possible variations on the configuration values above:
- The
user_id_attribute
anduser_name_attribute
could be mapped to other LDAP attributes in IdM. For example, to get a UUID for a user_id,user_id_attribute
could be set to theipaUniqueId
attribute. TheuidNumber
attribute would be another good candidate for theuser_id_attribute
mapping. To get human readible user names, theuser_name_attribute
could be mapped to thecn
attribute. Note that most authentication and role assignments are performed against the user name and not the user id. In most installations, bothuser_id_attribute
anduser_id_attribute
should be set touid
. - As discussed above, the
user_enabled_emulation_dn
setting could be assigned to an existing IdM group. This simplifies new OpenStack account creation somewhat.
Testing the configuration
Once the keystone.conf
file has the correct mappings to the IdM database, restart the openstack-keystone
service. Set the SERVICE_TOKEN
and SERVICE_ENDPOINT
environment variables to match the settings in keystone.conf
:
[root@keystone ~]# export SERVICE_TOKEN=012345SECRET99TOKEN012345
[root@keystone ~]# export SERVICE_ENDPOINT=http://127.0.0.1:35357/v2.0/
If the LDAP mappings are correct in keystone.conf
, the user-list
command should show the list of users in the IdM database, including the one we enabled above.
[root@keystone ~]# keystone user-list
+------------+------------+---------+-------------------------------+
| id | name | enabled | email |
+------------+------------+---------+-------------------------------+
| admin | admin | False | |
....
| rdoadmin | rdoadmin | True | rdoadmin@example.com |
...
+------------+------------+---------+-------------------------------+
If keystone returns a 401 error or a 404 error, confirm the settings in /etc/keystone/keystone.conf
. Also consider setting debug = True
in the [DEFAULT]
section. This will print the actual LDAP queries being run against the IdM server and their return codes in /var/log/keystone/keystone.log
.
Creating Tenants and Roles
Next, create the initial set of Tenants for OpenStack users and services to use. The following examples use the Tenant names from the OpenStack installation guide:
[root@keystone ~]# keystone tenant-create --name demo --description "Default Tenant"
[root@keystone ~]# keystone tenant-create --name service --description "Service Tenant"
Verify that the tenants were created:
[root@keystone ~]# keystone tenant-list
+----------------------------------+---------+---------+
| id | name | enabled |
+----------------------------------+---------+---------+
| 573429b5b7cc4312b981117890c1e9d8 | demo | True |
| a459741dfa8f4a8cb74306f001c564e3 | service | True |
+----------------------------------+---------+---------+
Once the initial tenants have been created, add the rdoadmin
user to the demo
tenant on the IdM server:
[admin@ipa01 ~]$ ldapmodify -x -D"uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com" -W <<EOF
dn: cn=573429b5b7cc4312b981117890c1e9d8,ou=tenants,cn=openstack,dc=example,dc=com
changetype: modify
add: member
member: uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com
EOF
Enter LDAP Password:
modifying entry "cn=573429b5b7cc4312b981117890c1e9d8,ou=tenants,cn=openstack,dc=example,dc=com"
Next, create the initial set of Roles for OpenStack users and services to use back on the keystone
server. The following examples create an admin
and user
role.
[root@keystone ~]# keystone role-create --name admin
[root@keystone ~]# keystone role-create --name user
[root@keystone ~]# keystone role-list
+----------------------------------+-------+
| id | name |
+----------------------------------+-------+
| d797691eb43640adb401c9b698fb4cef | admin |
| 69dd03c5b0fb43c38da33c0af7b52cfc | user |
+----------------------------------+-------+
Lastly, grant the admin
role to the OpenStack Administrator account in the demo Tenant:
[root@keystone]# keystone user-role-add --user-id rdoadmin --tenant-id 573429b5b7cc4312b981117890c1e9d8 --role-id d797691eb43640adb401c9b698fb4cef
To test, create a keystonerc_rdoadmin
file with the following contents:
export OS_USERNAME=rdoadmin
export OS_TENANT_NAME=demo
export OS_PASSWORD=<password>
export OS_AUTH_URL=http://10.17.12.12:35357/v2.0/
export PS1='[\u@\h \W(keystone_admin)]$ '
Initiate a new shell session on keystone
without the SERVICE_TOKEN
environment variable set, source the keystonerc_rdoadmin
script and then test retrieving a token:
[root@keystone ~(keystone_admin)]# keystone token-get
+-----------+----------------------------------+
| Property | Value |
+-----------+----------------------------------+
| expires | 2013-06-15T02:15:17Z |
| id | a6a8fd0ac3c443dd9425489a4990be3b |
| tenant_id | 573429b5b7cc4312b981117890c1e9d8 |
| user_id | rdoadmin |
+-----------+----------------------------------+
The rdoadmin
user should also be able to list all users as it was granted the admin
role:
[root@keystone ~(keystone_admin)]# keystone user-list
+------------+------------+---------+-------------------------------+
| id | name | enabled | email |
+------------+------------+---------+-------------------------------+
| admin | admin | False | |
....
| rdoadmin | rdoadmin | True | rdoadmin@example.com |
...
+------------+------------+---------+-------------------------------+
Creating Service Users
The glance, nova, ec2, cinder, and swift services all have service accounts in IdM to authenticate to Keystone. To create these accounts, use the following procedure:
- Create the user account in IdM
[admin@ipa01 ~]$ ipa user-add glance --cn=glance --first=glance --last=service
-------------------
Added user "glance"
-------------------
User login: glance
First name: glance
Last name: service
Full name: glance
Display name: glance service
Initials: gs
Home directory: /home/glance
GECOS field: glance service
Login shell: /bin/sh
Kerberos principal: glance@EXAMPLE.COM
UID: 1840200016
GID: 1840200016
Password: False
Kerberos keys available: False
- Set the user’s password
[admin@ipa01 ~]$ ipa passwd glance
New Password:
Enter New Password again to verify:
--------------------------------------------------
Changed password for "glance@EXAMPLE.COM"
--------------------------------------------------
[admin@ipa01 ~]$ kinit glance@EXAMPLE.COM
Password for glance@EXAMPLE.COM:
Password expired. You must change it now.
Enter new password:
Enter it again:
- Add the user to the
enabled_users
group in thecn=openstack
subtree
[admin@ipa01 ~]$ ldapmodify -x -D"uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com" -W <<EOF
> dn: cn=enabled_users,cn=openstack,dc=example,dc=com
> changetype: modify
> add: member
> member: uid=glance,cn=users,cn=accounts,dc=example,dc=com
> EOF
Enter LDAP Password:
modifying entry "cn=enabled_users,cn=openstack,dc=example,dc=com"
- Add the user to the
service
tenant in thecn=openstack
subtree
[admin@ipa01 ~]$ ldapmodify -x -D"uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com" -W <<EOF
> dn: cn=a459741dfa8f4a8cb74306f001c564e3,ou=tenants,cn=openstack,dc=example,dc=com
> changetype: modify
> add: member
> member: uid=glance,cn=users,cn=accounts,dc=example,dc=com
> EOF
Enter LDAP Password:
modifying entry "cn=a459741dfa8f4a8cb74306f001c564e3,ou=tenants,cn=openstack,dc=example,dc=com"
- Add the
admin
role to the user account in theservice
tenant.
[root@keystone ~]# keystone user-role-add --user-id glance --tenant-id a459741dfa8f4a8cb74306f001c564e3 --role-id d797691eb43640adb401c9b698fb4cef
Repeat this process with each of the other services which will be enabled in this OpenStack instance. For each of those services, edit the appropriate configuration file and set the keystone_authtoken
attributes to match the IdM user account. For example:
[keystone_authtoken]
auth_host = <your keystone IP>
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = glance
admin_password = <password>
Populating the Service Catalog
The last step in Keystone configuration is to populate the Service Catalog. In this example, services are stored in the MySQL database. The following page describes the population of the Service Catalog:
Managing User Accounts
Granting OpenStack privileges to an existing Red Hat IdM user
To grant privileges to an existing IdM user, use the following process:
- Add the account to the OpenStack enabled_users group:
[admin@ipa01 ~]$ ldapmodify -x -D"uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com" -W <<EOF
> dn: cn=enabled_users,cn=openstack,dc=example,dc=com
> changetype: modify
> add: member
> member: uid=msolberg,cn=users,cn=accounts,dc=example,dc=com
> EOF
Enter LDAP Password:
modifying entry "cn=enabled_users,cn=openstack,dc=example,dc=com"
- Add the account to the appropriate tenant
[admin@ipa01 ~]$ ldapmodify -x -D"uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com" -W <<EOF
dn: cn=573429b5b7cc4312b981117890c1e9d8,ou=tenants,cn=openstack,dc=example,dc=com
changetype: modify
add: member
member: uid=msolberg,cn=users,cn=accounts,dc=example,dc=com
EOF
Enter LDAP Password:
modifying entry "cn=573429b5b7cc4312b981117890c1e9d8,ou=tenants,cn=openstack,dc=example,dc=com"
- Add the the appropriate role to the tenant in Keystone
[root@keystone ~]# keystone user-role-add --user-id msolberg --tenant-id 573429b5b7cc4312b981117890c1e9d8 --role-id 69dd03c5b0fb43c38da33c0af7b52cfc
Securing OpenStack Services
By default, communication between the services which comprise RDO is not encrypted or authenticated. The following steps walk through creating certificates for these services using the Red Hat IdM certificate infrastructure. It is assumed that each service is already configured as per the OpenStack Installation Guide:
http://docs.openstack.org/grizzly/openstack-compute/install/yum/content/
Securing Horizon
OpenStack end users communicate with the Horizon dashbard over HTTP using a web browser. The following steps will encrypt that communication:
- Join the host to IdM (if you haven’t already)
[root@dashboard]# yum -y install ipa-client
[root@dashboard]# ipa-client-install
- Test IdM
[root@dashboard]# getent group ipausers
- On the IdM server, create a service principal for the host, substituting the hostname of the service which runs the dashboard:
[admin@ipa01]$ ipa service-add HTTP/dashboard.example.com
- On the dashboard host, install mod_ssl on the system:
[root@dashboard ~]# yum -y install mod_ssl
- Request a web certificate
[root@dashboard ~]# ipa-getcert request -r -f /etc/pki/tls/certs/`hostname -s`-http.crt -k /etc/pki/tls/private/`hostname -s`-http.key -N CN=`hostname --fqdn` -D `hostname` -U id-kp-serverAuth -K HTTP/`hostname --fqdn`
- Edit ssl.conf and point httpd to the new cert and key:
[root@dashboard ~]# vi /etc/httpd/conf.d/ssl.conf
- Configure it like:
SSLCertificateFile /etc/pki/tls/certs/<host>-http.crt
SSLCertificateKeyFile /etc/pki/tls/private/<host>-http.key
SSLCertificateChainFile /etc/ipa/ca.crt
- Optionally create a VirtualHost entry which redirects clients to the SSL. Substitute the hostname of the server running the dashboard in this example.
[root@dashboard ~]# cat >> /etc/httpd/conf.d/redirect.conf <<EOF
> NameVirtualHost *:80
> <VirtualHost *:80>
> ServerName dashboard.example.com
> Redirect permanent / https://dashboard.example.com/
> </VirtualHost>
EOF
- Restart apache
[root@dashboard ~]# service httpd restart
Requests to the web service on dashboard.example.com now be redirected to the SSL virtual host on that server. Ensure that the certificate presented to the web browser is valid and signed by the IdM master server.
Securing MySQL (WIP)
- On the IdM server, create a service principal for the host, substituting the hostname of the server running MySQL:
[admin@ipa01]$ ipa service-add mysql/mysql.example.com
- Request a certificate
[root@mysql ~]# ipa-getcert request -r -f /etc/pki/tls/certs/`hostname -s`-mysql.crt -k /etc/pki/tls/private/`hostname -s`-mysql.key -N CN=`hostname --fqdn` -D `hostname` -U id-kp-serverAuth -K mysql/`hostname --fqdn`
- Edit /etc/my.conf and point mysqld to the new cert and key:
[client]
ssl_ca=/etc/ipa/ca.crt
...
[mysqld]
ssl_ca=/etc/ipa/ca.crt
ssl_cert=/etc/pki/tls/certs/<host>-mysql.crt
ssl-key=/etc/pki/tls/private/<host>-mysql.key
- Restart mysqld:
[root@mysql ~]# service mysqld restart
- Test that SSL is working:
[root@mysql ~]# mysql -e "\s"
--------------
mysql Ver 14.14 Distrib 5.5.31, for Linux (x86_64) using readline 5.1
Connection id: 487
Current database:
Current user: root@localhost
SSL: Cipher in use is DHE-RSA-AES256-SHA
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.5.31 MySQL Community Server (GPL)
Protocol version: 10
Connection: Localhost via UNIX socket
Server characterset: latin1
Db characterset: latin1
Client characterset: utf8
Conn. characterset: utf8
UNIX socket: /var/lib/mysql/mysql.sock
Uptime: 2 sec
Threads: 10 Questions: 791702 Slow queries: 0 Opens: 91 Flush tables: 1 Open tables: 84 Queries per second avg: 4.800
- Configure the OpenStack services to use SSL. Set the SQL connection (either connection or sql_connection, depending on the service) . Note: that this also changes from addressing the host by IP address to FQDN:
connection = mysql://keystone_admin:a3a9e1219b81473e@mysql.example.com/keystone?ssl_ca=/etc/ipa/ca.crt
- To these files:
- /etc/glance/glance-api.conf
- /etc/glance/glance-registry.conf
- /etc/cinder/cinder.conf
- /etc/nova/nova.conf
- /etc/keystone/keystone.conf
- Restart the services:
[root@mysql ~]# service openstack-glance-api restart
[root@mysql ~]# service openstack-glance-registry restart
[root@mysql ~]# service openstack-cinder restart
[root@mysql ~]# service openstack-nova-api restart
[root@mysql ~]# service openstack-nova-cert restart
[root@mysql ~]# service openstack-nova-compute restart
[root@mysql ~]# service openstack-nova-conductor restart
[root@mysql ~]# service openstack-nova-consoleauth restart
[root@mysql ~]# service openstack-nova-network restart
[root@mysql ~]# service openstack-nova-novncproxy restart
[root@mysql ~]# service openstack-nova-scheduler restart
[root@mysql ~]# service openstack-keystone restart
Securing qpidd with SSL (WIP)
Configuring qpidd does not currently work without making some manual changes, depending on the version of OpenStack you are running. nova and glance do not properly allow SSL configuration due to launchpad bug https://bugs.launchpad.net/oslo-incubator/+bug/1158807 . It is a one-line change to make in the nova and cinder python files.
- Install the qpid-cpp-server-ssl package
# yum install qpid-cpp-server-ssl
- Create the IPA service
# ipa service-add qpid/qpid.example.com
- Create a directory to store the NSS certificate database for qpidd
# mkdir /etc/pki/qpid
# certutil -N -d /etc/pki/qpid (note the password you use)
- Create a password file containing the password you set on the NSS database
# echo password > /etc/pki/qpid/password.conf
- Fix permissions and ownership of files
# chown -R qpidd /etc/pki/qpid
# chmod 600 /etc/pki/qpid/password.conf
# restorecon /etc/pki/qpid/*
- Request an SSL certificate
# ipa-getcert request -d /etc/pki/qpid -n broker -p /etc/pki/qpid/password.conf -N CN=`hostname --fqdn` -D `hostname` -U id-kp-serverAuth -K qpidl/`hostname --fqdn`
- Add configuration options to
/etc/qpidd.conf
require-encryption=yes
ssl-require-client-authentication=no
ssl-cert-db=/etc/pki/qpid
ssl-cert-password-file=/etc/pki/qpid/password.conf
ssl-cert-name=broker
ssl-port=5671
- Configure nova to use ssl in
/etc/nova/nova.conf
qpid_protocol=ssl
qpid_port=5671
- Configure cinder to use ssl in
/etc/cinder/cinder.conf
qpid_protocol=ssl
qpid_port=5671
- Restart the nova and cinder services
# service openstack-cinder-api restart
# service openstack-cinder-scheduler restart
# service openstack-cinder-volume restart
# service openstack-glance-api restart
# service openstack-glance-registry restart
# service openstack-keystone restart
# service openstack-nova-api restart
# service openstack-nova-cert restart
# service openstack-nova-compute restart
# service openstack-nova-conductor restart
# service openstack-nova-consoleauth restart
# service openstack-nova-network restart
# service openstack-nova-novncproxy restart
# service openstack-nova-scheduler restart
Securing qpidd with Kerberos (WIP)
You can secure qpidd with either Kerberos or SSL, but not both. Securing with Kerberos provides an encrypted channel as well as authentication.
- Install the python-saslwrapper package
# yum install python-saslwrapper
- Add an IPA service for qpidd
[admin@ipa01]$ ipa service-add qpidd/openstack.example.com
- Retrieve a keytab for this new service
[root@openstack]# ipa-getkeytab -s ipa01.example.com -k /etc/qpidd.keytab -p qpidd/openstack.example.com
- Fix permissions and ownership of the keytab
[root@openstack]# chown qpidd /etc/qpidd.keytab
[root@openstack]# chmod 600 /etc/qpidd.keytab
- Configure qpidd by setting auth and realm in
/etc/qpidd.conf
auth=yes
realm=EXAMPLE.COM
- Configure GSSAPI as the only allowed mechanism for qpidd in
/etc/sasl2/qpidd.conf
mech_list: GSSAPI
- Create the file
/etc/sysconfig/qpidd
to tell qpidd the location of its Kerberos keytab:
KRB5_KTNAME=/etc/qpidd.keytab
- In Fedora we need to do another couple of steps so systemd will load the sysconfig file. Create a copy of the qpidd systemd service:
# cp /usr/lib/systemd/system/qpidd.service /etc/systemd/system
- Edit the version in
/etc/systemd/system
and add this to the [Service] section
EnvironmentFile=-/etc/sysconfig/qpidd
- Restart qpidd
[root@openstack]# service qpidd restart
/var/log/messages
should be complaining now about cinder and nova
ERROR [cinder.openstack.common.rpc.impl_qpid] Unable to connect to AMQP server: Error in sasl_client_start (-1) SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Cannot determine realm for numeric host address).
- Configure nova to use a DNS name instead of an IP address in
/etc/nova/nova.conf
qpid_hostname=openstack.example.com
- Configure cinder to use a DNS name instead of an IP address in
/etc/nova/cinder.conf
qpid_hostname=openstack.example.com
- Restart nova and cinder
[root@openstack]# service openstack-cinder-api restart
[root@openstack]# service openstack-cinder-scheduler restart
[root@openstack]# service openstack-cinder-volume restart
[root@openstack]# service openstack-nova-api restart
[root@openstack]# service openstack-nova-cert restart
[root@openstack]# service openstack-nova-compute restart
[root@openstack]# service openstack-nova-conductor restart
[root@openstack]# service openstack-nova-consoleauth restart
[root@openstack]# service openstack-nova-network restart
[root@openstack]# service openstack-nova-novncproxy restart
- Now if you look in
/var/log/messages
there should be errors about a missing credentials cache:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credentials cache file '/tmp/krb5cc_162' not found)
- Create an IPA service for nova and cinder
[admin@ipa01]$ ipa service-add nova/openstack.example.com
[admin@ipa01]$ ipa service-add cinder/openstack.example.com
- Get the uid for nova and cinder
[root@openstack]# id nova
uid=162(nova) gid=162(nova) groups=162(nova),99(nobody),107(qemu)
[root@openstack]# id cinder
uid=165(cinder) gid=165(cinder) groups=165(cinder),99(nobody)
- Create a keytab for nova
[root@openstack]# ipa-getkeytab -s ipa01.example.com -k /etc/nova.keytab -p nova/openstack.example.com
- Create a keytab for cinder
[root@openstack]# ipa-getkeytab -s ipa01.example.com -k /etc/cinder.keytab -p cinder/openstack.example.com
- Create a credentials cache for nova using the id for the nova user
[root@openstack]# kinit -k /etc/nova.keytab -c /tmp/krb5cc_162 nova/openstack.example.com
[root@openstack]# chown nova /tmp/krb5cc_162
- Create a credentials cache for cinder using the id for the cinder user
[root@openstack]# kinit -k /etc/cinder.keytab -c /tmp/krb5cc_165 cinder/openstack.example.com
[root@openstack]# chown cinder /tmp/krb5cc_165
- Restart qpidd again to force a reconnection attempt
[root@openstack]# service qpidd restart
The GSSAPI errors should be gone, nova and cinder are now using Kerberos for authentication and encryption.
The problem with this is that the key we just obtained is only good for a specified period of time: 24 hours by default. Once 24 hours passes the Kerberos ticket will no longer be valid and nova and cinder will no longer be able to communicate with qpidd.
The fix for now is to create a cron job which will renew these credentials.
[root@openstack]# crontab -e
1 0,6,12,18 * * * su - nova -c "kinit nova/openstack.example.com -kt /etc/nova.keytab -c /tmp/krb5cc_162"
1 0,6,12,18 * * * su - cinder -c "kinit cinder/openstack.example.com -kt /etc/cinder.keytab -c /tmp/krb5cc_165"
Securing Keystone
When a server joins an IdM domain, the OpenLDAP client libraries are configured to communicate with the IdM server over LDAPS. To verify that the system has been configured correctly, check /etc/openldap/ldap.conf
. It should contain a reference to a certificate file:
[root@keystone]# cat /etc/openldap/ldap.conf
#File modified by ipa-client-install
URI ldaps://ipa01.example.com
BASE dc=example,dc=com
TLS_CACERT /etc/ipa/ca.crt
If this is configured correctly, the connection between Keystone and the IdM server can be encrypted by changing the protocol from ldap://
to ldaps://
in /etc/keystone/keystone.conf
. For example:
[ldap]
url = ldaps://ipa01.example.com
user = uid=rdoadmin,cn=users,cn=accounts,dc=example,dc=com
...
Support for LDAP with StartTLS in Keystone should be available in the Havana release. That functionality can be tracked here: https://bugs.launchpad.net/keystone/+bug/1040115
Other Considerations
- The default password policy for IdM users requires them to change their password every 90 days. This will wreak havok upon most OpenStack environments. To mitigate this, create a “Service Account” group in IdM and assign a new password policy to that group. Add cinder, nova, etc. to that group.
- In the same vein, the account that Keystone uses to authenticate to IdM (set in
/etc/keystone/keystone.conf
is also a service account. It probably makes more sense to name the useruid=keystone
and add the user to the “Service Account” IdM group as well. Make sure that this service account also has the “OpenStack Administrator” privilege created above. - This example architecture is not highly available. It probably makes sense to use a load balancer instead of a direct LDAP connection to the IdM master.
- It is tempting to use the groups defined in IdM as tenants in OpenStack. This cuts down on the amount of LDIF typing required. Note that OpenStack roles are created underneath tenants, which is probably not what we want in the IdM tree:
# d797691eb43640adb401c9b698fb4cef, a459741dfa8f4a8cb74306f001c564e3, tenants
, openstack, example.com
dn: cn=d797691eb43640adb401c9b698fb4cef,cn=a459741dfa8f4a8cb74306f001c564e3,ou
=tenants,cn=openstack,dc=example,dc=com
objectClass: organizationalRole
objectClass: top
roleOccupant: uid=glance,cn=users,cn=accounts,dc=example,dc=com
roleOccupant: uid=nova,cn=users,cn=accounts,dc=example,dc=com
roleOccupant: uid=cinder,cn=users,cn=accounts,dc=example,dc=com
roleOccupant: uid=ec2,cn=users,cn=accounts,dc=example,dc=com
roleOccupant: uid=swift,cn=users,cn=accounts,dc=example,dc=com
cn: d797691eb43640adb401c9b698fb4cef