From cc34e25fd54e60caf744130d90c14f8d839c7823 Mon Sep 17 00:00:00 2001 From: Nick Walker Date: Mon, 7 Dec 2015 07:55:32 -0800 Subject: Final steps to make Code Manager work - Moved ssh key generation and git deploy key out of the puppetmaster profile and into zack_r10k and code_manager - Swapped code manager into the all_in_one role - Made a 2015.2 all_in_one role if users prefer to use it - Conditionally move all existing code out of environmentpath to allow file sync to sync files - Update the README to compliment the new puppet code --- README.md | 73 ++++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 42 insertions(+), 31 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 837e5f0..8d14212 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Before Starting: +# Before Starting: This control repo and the steps below are intended to be used during a new installation of PE. @@ -17,15 +17,15 @@ extension_requests: 1.3.6.1.4.1.34380.1.1.13: 'all_in_one_pe' ``` -### If You Have Not Installed PE +### If You Have Not Installed PE -Good then you can proceed forward and the trusted fact will be used when you get to the install step. +Good then you can proceed forward and the trusted fact will be used when you get to the install step. ### If You Have Already Installed PE -Trusted facts are created at the time a CSR is generated. So, we need to regenerate the certificate on the master for the above trusted fact to be created. +Trusted facts are created at the time a CSR is generated. So, we need to regenerate the certificate on the master for the above trusted fact to be created. -Follow this document to regenerate the certificate on your master. +Follow this document to regenerate the certificate on your master. http://docs.puppetlabs.com/pe/latest/regenerate_certs_master.html @@ -53,20 +53,20 @@ http://docs.puppetlabs.com/pe/latest/regenerate_certs_master.html - In the left hand pane, select memembers - Add the `r10k_api_user` with `master` permissions -7. Add your user to the `puppet` group as well +7. Add your user to the `puppet` group as well 8. Create a project called `control-repo` and set the Namespace to be the `puppet` group 9. Logout of root and login as the `r10k_api_user` - Go to profile settings -> account ( https:///profile/account ) - Copy the api token - + 10. Clone this control repository to your laptop/workstation - `git clone ` - `cd control-repo` 11. `git mv hieradata/nodes/example-puppet-master.yaml hieradata/nodes/.yaml` - - Open `hieradata/nodes/.yaml` + - Open `hieradata/nodes/.yaml` - edit `gms_api_token` to be your api token - edit `git_management_system` to be 'gitlab' - edit the `gms_server_url` @@ -99,7 +99,7 @@ Coming soon! ###Install PE 1. Download the latest version of the PE installer for your platform and copy it to your master - - https://puppetlabs.com/download-puppet-enterprise + - https://puppetlabs.com/download-puppet-enterprise 2. Expand the tarball and `cd` into the directory 3. Run `puppet-enterprise-installer` to install @@ -109,34 +109,50 @@ http://docs.puppetlabs.com/pe/latest/install_basic.html ###Get the Control-Repo Deployed On Your Master -At this point you have my control-repo code deployed into your git server. However, we have one final challenge getting that code onto your puppet master. In the end state the master will pull code from the git server via r10k, however, at this moment your puppet master doesn't have credentials to get code from the git server. +At this point you have our control-repo code deployed into your git server. However, we have one final challenge: getting that code onto your puppet master. In the end state the master will pull code from the git server via r10k, however, at this moment your puppet master doesn't have credentials to get code from the git server. -So, we'll set up a deploy key in the git server that will allow a ssh-key we make to deploy the code and configure everything else. +So, we'll set up a deploy key in the git server that will allow a ssh-key we make to deploy the code and configure everything else. 1. On your puppet master, make an ssh key for r10k to connect to gitlab - - `/usr/bin/ssh-keygen -t rsa -b 2048 -C 'r10k' -f /root/.ssh/r10k_rsa -q -N ''` + - `/usr/bin/ssh-keygen -t rsa -b 2048 -C 'code_manager' -f /etc/puppetlabs/puppetserver/code_manager.key -q -N ''` - http://doc.gitlab.com/ce/ssh/README.html - https://help.github.com/articles/generating-ssh-keys/ 2. Create a deploy key on the `control-repo` project in Gitlab - Paste in the public key from above - - `cat /root/.ssh/r10k_rsa.pub` -3. Follow https://docs.puppetlabs.com/pe/latest/r10k_config_console.html - - The remote is on the front page of the project in the gitlab UI - - git_settings should be: - - `{"provider": "rugged", - "private_key": "/root/.ssh/r10k_rsa"}` -3. Run `puppet agent -t` + - `cat /etc/puppetlabs/puppetserver/code_manager.key.pub` +3. Login to the PE console +4. Select Access Control in the left hand panel +5. On the User Roles page, add a new role called `Deploy Environments` + - NOTE: Make sure to name it exactly as I have because the puppet code expects that exact name +6. After creating the role click through and select the permissions tab + - Add Puppet Environment type, Deploy Code permission, and All object + - Add Tokens type, override default expiry permission +7. Still in the PE Console, navigate to the Classification page + - Click on the PE Master group + - Click the Classes tab + - Add the `puppet_enterprise::profile::master` + - Set the `r10k_remote` to the ssh url from the front page of your gitlab repo + - Set the `r10k_private_key` parameter to `/etc/puppetlabs/puppetserver/code_manager.key` + - Commit your changes +8. Run `puppet agent -t` - Expect to see changes to `r10k.yaml` -3. Run `r10k deploy environment -pv` -4. Run `puppet agent -t` +9. Run `r10k deploy environment -pv` +10. Run `puppet agent -t` + - Expect to see code manager enabled +10. `echo 'code_manager_mv_old_code=true' > /opt/puppetlabs/facter/facts.d/code_manager_mv_old_code.txt` +11. Run `puppet agent -t` - Now you should see many more changes + - Your code has been deployed with code manager now + +## Test Code Manager + ## Test The Zack/r10k Webhook -One of the components setup by this control-repo is that when you "push" code to your git server, the git server will inform the puppet master to run `r10k deploy environment -p`. +One of the components setup by this control-repo is that when you "push" code to your git server, the git server will inform the puppet master to run `r10k deploy environment -p`. -1. Edit README.md +1. Edit README.md - Just add something to it 2. `git add README.md` 3. `git commit -m "edit README"` @@ -144,16 +160,14 @@ One of the components setup by this control-repo is that when you "push" code to 5. Allow the push to complete and then give it few seconds to complete - Open `/etc/puppetlabs/code/environments/production/README.md` and confirm your change is present - - ---- #Miscellaneous ## If You Want to Install Pointing To This Repo on Github -### Setting Up Gitlab +### Setting Up Gitlab -1. Install Gitlab on a server by specifying the following trusted fact on the soon-to-be Gitlab server and then [install the PE agent](http://docs.puppetlabs.com/pe/latest/install_agents.html#using-the-puppet-agent-package-installation-script). +1. Install Gitlab on a server by specifying the following trusted fact on the soon-to-be Gitlab server and then [install the PE agent](http://docs.puppetlabs.com/pe/latest/install_agents.html#using-the-puppet-agent-package-installation-script). ``` --- @@ -164,7 +178,7 @@ One of the components setup by this control-repo is that when you "push" code to ### Setting up Github -Not yet completed. +Not yet completed. ### Setting up Stash @@ -173,6 +187,3 @@ Not yet completed. #TODO Flush out generating an answer file and then appending extra answers onto the end of it. - - - -- cgit v1.2.3 From 5dcdf019634070e8f52b53f32074b3d14f4555f3 Mon Sep 17 00:00:00 2001 From: Nick Walker Date: Thu, 17 Dec 2015 10:33:25 -0800 Subject: Add instructions for testing code manager Also, add a table of contents to the README. I generated the table of contents using https://github.com/ekalinin/github-markdown-toc.go --- README.md | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) (limited to 'README.md') diff --git a/README.md b/README.md index 8d14212..ed90d4f 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,26 @@ +Table of Contents +================= + + * [Before Starting:](#before-starting) + * [Setup a Trusted Fact On Your PE Master](#setup-a-trusted-fact-on-your-pe-master) + * [If You Have Not Installed PE](#if-you-have-not-installed-pe) + * [If You Have Already Installed PE](#if-you-have-already-installed-pe) + * [Copy This Repo Into Your Own Git Server](#copy-this-repo-into-your-own-git-server) + * [Gitlab](#gitlab) + * [Stash](#stash) + * [Github](#github) + * [Configure PE to Use the Control\-Repo](#configure-pe-to-use-the-control-repo) + * [Install PE](#install-pe) + * [Get the Control\-Repo Deployed On Your Master](#get-the-control-repo-deployed-on-your-master) + * [Test Code Manager](#test-code-manager) + * [Test The Zack/r10k Webhook](#test-the-zackr10k-webhook) + * [Miscellaneous](#miscellaneous) + * [If You Want to Install Pointing To This Repo on Github](#if-you-want-to-install-pointing-to-this-repo-on-github) + * [Setting Up Gitlab](#setting-up-gitlab) + * [Setting up Github](#setting-up-github) + * [Setting up Stash](#setting-up-stash) + * [TODO](#todo) + # Before Starting: This control repo and the steps below are intended to be used during a new installation of PE. @@ -146,7 +169,18 @@ So, we'll set up a deploy key in the git server that will allow a ssh-key we mak ## Test Code Manager +One of the components setup by this control-repo is that when you "push" code to your git server, the git server will inform the puppet master to deploy branch you just pushed. +1. In one terminal window, `tail -f /var/log/puppetlabs/puppetserver/puppetserver.log` +2. In a second terminal window + - Add a new file, `touch test_file` + - `git add test_file` + - `git commit -m "adding a test_file"` + - `git push origin production` +3. Allow the push to complete and then wait a few seconds for everything to sync over + - `ls -l /etc/puppetlabs/code/environments/production` + - Confirm test_file is present +4. In your first terminal window review the `puppetserver.log` to see the type of logging each sync will create ## Test The Zack/r10k Webhook @@ -184,6 +218,5 @@ Not yet completed. Not yet completed. - #TODO Flush out generating an answer file and then appending extra answers onto the end of it. -- cgit v1.2.3 From ad00dd7a9a5e06d1aadbd68043979d4e060b6c04 Mon Sep 17 00:00:00 2001 From: Nick Walker Date: Mon, 21 Dec 2015 17:47:05 -0800 Subject: Add an exec to create the Deploy Environments RBAC Role Prior to this commit there was a requirement for the user of this repo to create a RBAC role in order for code manager to work. After this commit an exec statement will curl the RBAC API to create the role one time and hopefully it works otherwise the exec will not run again. --- README.md | 10 ++------- site/profile/manifests/git_webhook/code_manager.pp | 24 +++++++++++++++++++++- 2 files changed, 25 insertions(+), 9 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index ed90d4f..9818554 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ Table of Contents This control repo and the steps below are intended to be used during a new installation of PE. -This control repo has only been tested against PE2015.2.z, it's likely close to working on PE3.8.z but has not been tested. +This control repo has only been tested against PE2015.2.z and PE2015.3.z. It is likely close to working on PE3.8.z but has not been tested. If you intend to use it on an existing installation then be warned that if you've already written or downloaded modules when you start using r10k it will remove all of the existing modules and replace them with what you define in your Puppetfile. Please copy or move your existing modules to another directory to ensure you do not lose any work you've already started. @@ -144,13 +144,7 @@ So, we'll set up a deploy key in the git server that will allow a ssh-key we mak - Paste in the public key from above - `cat /etc/puppetlabs/puppetserver/code_manager.key.pub` 3. Login to the PE console -4. Select Access Control in the left hand panel -5. On the User Roles page, add a new role called `Deploy Environments` - - NOTE: Make sure to name it exactly as I have because the puppet code expects that exact name -6. After creating the role click through and select the permissions tab - - Add Puppet Environment type, Deploy Code permission, and All object - - Add Tokens type, override default expiry permission -7. Still in the PE Console, navigate to the Classification page +7. Navigate to the Classification page - Click on the PE Master group - Click the Classes tab - Add the `puppet_enterprise::profile::master` diff --git a/site/profile/manifests/git_webhook/code_manager.pp b/site/profile/manifests/git_webhook/code_manager.pp index 60cabf4..7470e1c 100644 --- a/site/profile/manifests/git_webhook/code_manager.pp +++ b/site/profile/manifests/git_webhook/code_manager.pp @@ -35,13 +35,35 @@ class profile::git_webhook::code_manager { unless => "/usr/bin/test \$(stat -c %U ${::settings::codedir}/environments/production) = 'pe-puppet'", } + $code_manager_role_name = 'Deploy Environments' + $create_role_creates_file = '/etc/puppetlabs/puppetserver/.puppetlabs/deploy_environments_created' + $create_role_curl = @(EOT) + /opt/puppetlabs/puppet/bin/curl -k -X POST -H 'Content-Type: application/json' \ + https://<%= $::trusted['certname'] %>:4433/rbac-api/v1/roles \ + -d '{"permissions": [{"object_type": "environment", "action": "deploy_code", "instance": "*"}, + {"object_type": "tokens", "action": "override_lifetime", "instance": "*"}],"user_ids": [], "group_ids": [], "display_name": "<%= $code_manager_role_name %>", "description": ""}' \ + --cert <%= $::settings::certdir %>/<%= $::trusted['certname'] %>.pem \ + --key <%= $::settings::privatekeydir %>/<%= $::trusted['certname'] %>.pem \ + --cacert <%= $::settings::certdir %>/ca.pem; + touch <%= $create_role_creates_file %> + | EOT + + exec { 'create deploy environments role' : + command => inline_epp( $create_role_curl ), + creates => $create_role_creates_file, + logoutput => true, + path => $::path, + require => File[$token_directory], + } + rbac_user { $code_manager_service_user : ensure => 'present', name => $code_manager_service_user, email => "${code_manager_service_user}@example.com", display_name => 'Code Manager Service Account', password => $code_manager_service_user_password, - roles => [ 'Deploy Environments' ], + roles => [ $code_manager_role_name ], + require => Exec['create deploy environments role'], } file { $token_directory : -- cgit v1.2.3 From 752f2ef87bb56895f5a63c89fd01fe0ed0c623be Mon Sep 17 00:00:00 2001 From: Nick Walker Date: Mon, 4 Jan 2016 20:22:06 -0800 Subject: Update README with info on how to upgrade from 2015.2 Mostly reorganized the README to be 2015.3 first and then have some extra info at the bottom about upgrading from 2015.2. Added a blurb about what the control-repo provides. --- README.md | 102 ++++++++++++++++++++++++++++++++------------------------------ 1 file changed, 52 insertions(+), 50 deletions(-) (limited to 'README.md') diff --git a/README.md b/README.md index 9818554..24eb30f 100644 --- a/README.md +++ b/README.md @@ -1,33 +1,44 @@ Table of Contents ================= - * [Before Starting:](#before-starting) - * [Setup a Trusted Fact On Your PE Master](#setup-a-trusted-fact-on-your-pe-master) - * [If You Have Not Installed PE](#if-you-have-not-installed-pe) - * [If You Have Already Installed PE](#if-you-have-already-installed-pe) - * [Copy This Repo Into Your Own Git Server](#copy-this-repo-into-your-own-git-server) - * [Gitlab](#gitlab) - * [Stash](#stash) - * [Github](#github) - * [Configure PE to Use the Control\-Repo](#configure-pe-to-use-the-control-repo) - * [Install PE](#install-pe) - * [Get the Control\-Repo Deployed On Your Master](#get-the-control-repo-deployed-on-your-master) - * [Test Code Manager](#test-code-manager) - * [Test The Zack/r10k Webhook](#test-the-zackr10k-webhook) - * [Miscellaneous](#miscellaneous) - * [If You Want to Install Pointing To This Repo on Github](#if-you-want-to-install-pointing-to-this-repo-on-github) - * [Setting Up Gitlab](#setting-up-gitlab) - * [Setting up Github](#setting-up-github) - * [Setting up Stash](#setting-up-stash) - * [TODO](#todo) - -# Before Starting: +* [Before Starting](#before-starting) +* [What You Get From This control\-repo](#what-you-get-from-this-control-repo) +* [How To Set It All Up](#how-to-set-it-all-up) + * [Setup a Trusted Fact On Your PE Master](#setup-a-trusted-fact-on-your-pe-master) + * [If You Have Not Installed PE](#if-you-have-not-installed-pe) + * [If You Have Already Installed PE](#if-you-have-already-installed-pe) + * [Copy This Repo Into Your Own Git Server](#copy-this-repo-into-your-own-git-server) + * [Gitlab](#gitlab) + * [Stash](#stash) + * [Github](#github) + * [Configure PE to Use the Control\-Repo](#configure-pe-to-use-the-control-repo) + * [Install PE](#install-pe) + * [Get the Control\-Repo Deployed On Your Master](#get-the-control-repo-deployed-on-your-master) + * [Test Code Manager](#test-code-manager) +* [Updating From a Previous Version of PE](#updating-from-a-previous-version-of-pe) + * [Upgrading to PE2015\.3\.z from PE 2015\.2\.z](#upgrading-to-pe20153z-from-pe-20152z) +* [Appendix](#appendix) + * [Test The Zack/r10k Webhook](#test-the-zackr10k-webhook) + +# Before Starting This control repo and the steps below are intended to be used during a new installation of PE. -This control repo has only been tested against PE2015.2.z and PE2015.3.z. It is likely close to working on PE3.8.z but has not been tested. +The instructions are geared towards a new installation of PE2015.3.z. However, the control-repo should work just fine on [PE2015.2.z](#upgrading-to-pe20153z-from-pe-20152z) -If you intend to use it on an existing installation then be warned that if you've already written or downloaded modules when you start using r10k it will remove all of the existing modules and replace them with what you define in your Puppetfile. Please copy or move your existing modules to another directory to ensure you do not lose any work you've already started. +If you intend to use this control-repo on an existing installation then be warned that if you've already written or downloaded modules when you start using r10k it will remove all of the existing modules and replace them with what you define in your Puppetfile. Please copy or move your existing modules to another directory to ensure you do not lose any work you've already started. + +# What You Get From This control-repo + +As a result of following the instructions below you will receive at least the beginning of a best-practices installation of PE including... + + - A git server + - The ability to push code to your git server and have it automatically deployed to your PE Master + - A config_version script to output the commit of code that your agent just applied + - Optimal tuning of PE settings for this configuration + - Working and example roles/profiles code + +# How To Set It All Up ## Setup a Trusted Fact On Your PE Master @@ -176,8 +187,26 @@ One of the components setup by this control-repo is that when you "push" code to - Confirm test_file is present 4. In your first terminal window review the `puppetserver.log` to see the type of logging each sync will create +---- +# Updating From a Previous Version of PE + +## Upgrading to PE2015.3.z from PE 2015.2.z + +Remove `pe_r10k` from the PE master group in the console and instead add the following two parameters to the `puppet_enterprise::profile::master` class under the PE master group. + +- `r10k_remote` = the ssh url for your internal repo +- `r10k_private_key` = `/etc/puppetlabs/puppetserver/code_manager.key` + +When upgrading the `puppet_enterprise::profile::master` class has the `file_sync_enabled` parameter set to `false`. This parameter should be removed so that code manager can configure file sync. + +Finally, you’ll need to `echo 'code_manager_mv_old_code=true' > /opt/puppetlabs/facter/facts.d/code_manager_mv_old_code.txt` so that my puppet code will redeploy all of your code with code manager. + +# Appendix + ## Test The Zack/r10k Webhook +If you are using PE2015.2.z or if you've forced the use of the zack/r10k webhook then you'll want to test that it works. + One of the components setup by this control-repo is that when you "push" code to your git server, the git server will inform the puppet master to run `r10k deploy environment -p`. 1. Edit README.md @@ -187,30 +216,3 @@ One of the components setup by this control-repo is that when you "push" code to 4. `git push origin production` 5. Allow the push to complete and then give it few seconds to complete - Open `/etc/puppetlabs/code/environments/production/README.md` and confirm your change is present - ----- -#Miscellaneous - -## If You Want to Install Pointing To This Repo on Github - -### Setting Up Gitlab - -1. Install Gitlab on a server by specifying the following trusted fact on the soon-to-be Gitlab server and then [install the PE agent](http://docs.puppetlabs.com/pe/latest/install_agents.html#using-the-puppet-agent-package-installation-script). - - ``` - --- - extension_requests: - #pp_role - 1.3.6.1.4.1.34380.1.1.13: 'gitlab' - ``` - -### Setting up Github - -Not yet completed. - -### Setting up Stash - -Not yet completed. - -#TODO -Flush out generating an answer file and then appending extra answers onto the end of it. -- cgit v1.2.3