
This week on the podcast, we share a database migration Oh No! story. Then Dan and Kyle talk about updates to the Vagabond and psadmin-plus projects.
Show Notes
- Oh No! Story @ 1:45
- Vagabond – Windows Support @ 6:00
- psadmin-plus for Windows @ 20:30
This week on the podcast, we share a database migration Oh No! story. Then Dan and Kyle talk about updates to the Vagabond and psadmin-plus projects.
This week on the podcast, Charlie Sinks joins us to talk about the Upper Midwest Regional User Group. We talk about using Agile with PeopleSoft, our experiences with Elasticearch, the Idea Space for PeopleTools, and using Git with PeopleSoft.
pspuppet.sh
is used to prepare the environment. Need to invoke if the bootstrap or puppet failed and you need to re-runThis week on the podcast, Dan and Kyle discuss strategies to organize and manage Event Mapping code, writing code that other people will use, trying to work with Push Notifications and custom ACM modules. Kyle ends the podcast with a funny story about searching for images.
Since the Deployment Packages were released with PeopleTools 8.55, one of my criticisms has been that the DPK is a bit of a sledgehammer. If you define multiple PeopleSoft environments on a server and you want to configure one web server, ALL the domains that the DPK knows about are shut down.
Puppet has an Environments feature that lets you segregate your code and data. While the DPK does not support Puppet Environments out of the box, we can use them to make the DPK less of a sledgehammer when managing our domains. (There is still some sledgehammering going on, so go vote for this idea).
While environments let you separate the modules
, manifests
and data
folder, in this post we’ll separate just the data
folder. This will let us share a common set of code (the manifests
and modules
folders) but the configuration of each domain will be different.
If you want to extend this to the modules
and manifests
folder, copy those into the environment folders with the environment-specific changes. This is useful for testing new code changes or if you want an environment to use a different DPK Role in the site.pp
file.
dev
and tst
folders under c:\programdata\puppetlabs\puppet\etc\environments
You can have multiple environments under this folder – as many as you want. A strategy that I’m testing is using the database name as the environment name. For this post, I’ll stick with dev
and tst
puppet\etc\data
to puppet\etc\environments\dev\data
and puppet\etc\ environments\tst\data
.Under the puppet\etc
folder, add (or modify) the puppet.conf
file to look like this:
[main]
environment=production
parser=future
environmentpath=c:\programdata\puppetlabs\puppet\etc\environments
hiera_config=c:\programdata\puppetlabs\hiera\etc\hiera.yaml
basemodulepath=c:\programdata\puppetlabs\puppet\etc\modules
This file tells Puppet where to look for your environments, your Hiera configuration, your default module location, and the default Puppet Environment.
Last, we’ll modify the hiera.yaml
file in c:\programdata\puppetlabs\hiera\etc
to include environments:
---
:backends:
- yaml
:hierarchy:
- "environments/%{::environment}/data/psft_customizations"
- "environments/%{::environment}/data/psft_configuration"
- "environments/%{::environment}/data/psft_deployment"
- "environments/%{::environment}/data/psft_unix_system"
- "environments/%{::environment}/data/defaults"
:yaml:
:datadir: c:\programdata\puppetlabs\puppet\etc
If you want to share some of the files, like the defaults.yaml
or the psft_unix_system.yaml
file, you could keep those under the main puppet\etc\data
folder. Your hiera.yaml
file would look like this:
---
:backends:
- yaml
:hierarchy:
- "environments/%{::environment}/data/psft_customizations"
- "environments/%{::environment}/data/psft_configuration"
- "environments/%{::environment}/data/psft_deployment"
- data/psft_unix_system
- data/defaults
:yaml:
:datadir: c:\programdata\puppetlabs\puppet\etc
Once our Puppet changes are complete we can test some builds. When we run puppet apply
, we’ll add an additional paratemer: the environment. To build my dev
environment domains, I’ll use this procedure:
cd c:\programdata\puppetlabs\puppet\etc\manifests
puppet apply .\site.pp --environment=dev --debug
Once the dev
domains are built and running, you can kick off the tst
build with:
puppet apply .\site.pp --environment=tst --debug
As the tst
environment is building, your dev
domains should stay up and not be affected by the Puppet run. If they are affected, you may have some YAML changes that need to be made. Make sure your configuration’s between the environment don’t overlap (e.g, same PS_CFG_HOME
and domain names).
The way PeopleSoft delivers Puppet and the Hiera backend, is that everything you define in psft_customizations.yaml
overrides configuration defined elsewhere. This is a useful setup when getting started with the DPK and Puppet. But when using YAML files to manage your configuration across multiple servers, you’ll quickly find that you are re-entering the same configuration in many files.
Hiera, the tool Puppet uses to read YAML files, has multiple ways to look up data. First, let’s cover what a YAML hash is. A hash is a key-value structure used in the DPK to store configuration. For example, this is the hash for PS_HOME information:
ps_home:
db_type: "%{hiera('db_platform')}"
unicode_db: "%{hiera('unicode_db')}"
location: "%{hiera('ps_home_location')}"
remove: true
The main hash key is ps_home
, and its value is all the configuration below it. The next level down has 4 keys with 4 corresponding values. The appserver_domain_list
hash is a large one that contains all the configuration for one or more app server domains.
Under the delivered setup, if you want to change a value for a domain you need to copy the entire appserver_domain_list
hash into your psft_customizations.yaml
file and make the change. With Hiera hashing, you could define your domains in a file named appservers.yaml
and any specific server changes can be defined in hostname.yaml
. For example, the hostname.yaml
file could contain this hash to override a configuration:
appserver_domain_list:
DEV:
feature_settings:
SERVER_EVENTS: "Yes"
DOMAIN_GW: "Yes"
This provides far more flexibility when working with YAML files, but it does introduce some complexity. If you want to give this a try, here is how you can convert the current DPK to use Hiera hasing.
I used VisualStudio Code to do the find/replace. Open up the etc\modules
directory and do these against the modules\pt_profile
folder:
hiera('tns_admin_list
hiera_hash('tns_admin_list
I repeated this step for the following lookups.
tns_admin_list
appserver_domain_list
prcs_domain_list
pia_domain_list
You don’t want to replace all the lookups – that will cause errors. But, you can replace additional lookups if you want. Anything that is a hash in YAML files can use the hiera_hash()
lookup function. If you wanted to make the ps_home:
key support hash merging, you could replace hiera('ps_home
with hiera_hash('ps_home
.
By default, Hiera will look at the top-level keys of a hash and not merge the underlying settings. Hiera hashing will merge all the values inside the hash. This means you can you define a hash with default settings in a common file (e.g, default app server settings). Then you can specify server or application specific settings in a YAML file for that domain or server.
To enable the hash merging, open the hiera.yaml
file under c:\programdata\puppetlabs\hiera\etc
.
Add this line to the file:
:merge_behavior: deeper
With Hiera hash merging, we can utilize more than the psft_customizations.yaml
file to manage our configuration. We can use multiple YAML files to control our configuration. For example, we could have:
[hostname].yaml
dev.yaml
hr.yaml
common.yaml
So, this setup would let us define common configuration that is shared across all applications in the common.yaml
. Next, we could define anything related to servers that run HR applications in the hr.yaml
. For any settings that are specific to the Development region, we can add them into dev.yaml
. Last, for anything that is specific to the server, we can add into the [hostname].yaml
file. This setup would let you re-use the common
, hr
, and dev
YAML files across multiple servers, and anything specific to the server would be defined in [hostname].yaml
.
In the hiera.yaml
file, we can define this setup like this:
:hierarchy:
- "%{::hostname}"
- dev
- hr
- common
On the command line, you can use the hiera
utility to test lookups with Hiera. To do a normal Hiera lookup, use
hiera appserver_domain_list
To test a hiera hash lookup, use
hiera --hash appserver_domain_list
If you have multiple YAML files with the appserver_domain_list
hash, the first option will only show you the results from the top of the list. The second test should show you a merged appserver_domain_list
hash.
This week on the podcast, Dan dives into some advanced Puppet configuration to use with the DPK. Dan and Kyle discuss Hiera hash merging, using Puppet environments, and using the ACM with the DPK. Kyle talks about a bug with the Interaction Hub and My Favorites and some odd finds in PeopleBooks.
psft_customizations.yaml
@ 66:45The PeopleSoft Cloud Architecture is built on two technologies: Deployment Packages (DPK) and Automated Configuration Management (ACM). On this site, we’ve talked about Deployment Packages quite a bit, but we haven’t discussed Automated Configuration Management. This post will introduce you to what the ACM is and how the DPK uses it.
In the past, I have created custom DPK roles without the DPK profiles that run the ACM. This was mostly to prevent ACM runs when we build a domain. It turns out that you can control the ACM (globally) with your psft_customizations.yaml
file.
In the pt_tools_preboot_config
and pt_tools_postboot_config
profile, there is a Hiera lookup to see if ACM is enabled
$run_preboot_config_setup = hiera('run_preboot_config_setup', true)
or
$run_postboot_config_setup = hiera('run_postboot_config_setup', true)
If you haven’t defined these values, the default will be true
. You can disable the ACM by adding this your psft_customizations.yaml
:
run_preboot_config_setup: false
run_postboot_config_setup: false
There is a bug in the current DPK (8.55 and 8.56) if you set both values. In the modules/pt_profile/manifests/pt_tools_postboot_config.pp
file, line 98 should be:
notify {"POST-Boot setup run is false":}
(Switch the message “PRE-Boot” to “POST-Boot”).
When the DPK runs Automated Configuration Management, it will take the plugins and properties you define in the psft_customizations.yaml
file and build a template file on the fly. To define ACM steps to run, you’ll add hashes to one of two sections:
component_preboot_setup_list
component_postboot_setup_list
The labels are self-explanatory, but here is any easy way to know which plugs go in the post boot setup: If the setup requires IB to be running, it goes in post boot. That would include IB Configuration itself and Search Framework configuration.
Here is an example of setting up the Elasticsearch Search Instance via ACM. Unde the component_postboot_setup_list
hash, we create a searching
hash. (This name can be any string, but we’ll use that string later in the file). Then, we set the Run Control ID to use and the OS user who will run the App Engine.
component_postboot_setup_list:
searching:
run_control_id: searching
os_user: "%{hiera('domain_user')}"`
Next, we define the database connection settings for the web_profile
hash.
db_settings:
db_name: "%{hiera('db_name')}"
db_type: "%{hiera('db_platform')}"
db_opr_id: "%{hiera('db_user')}"
db_opr_pwd: "%{hiera('db_user_pwd')}"
db_connect_id: "%{hiera('db_connect_id')}"
db_connect_pwd: "%{hiera('db_connect_pwd')}"`
Then we define the ACM Plugins we want to run. The acm_plugin_list
is a hash of ACM Plugin names, and their repective configuration. To get a list of all the possible configuration, you can go into the PIA (PeopleTools > Automated Config Manager > ACM Templates > Template Definitions) and see the delivered ACM plugins.
Below is an example of the PTSFConfigureSrchInstance
plugin.
acm_plugin_list:
PTSFConfigureSrchInstance:
env.ptsf_search_instance: PTSF_DEFAULT
env.search_provider: ES
env.search_nodes: 1
env.node1_search_port: 9200
env.node1_search_host: psvagabond.psadmin.io
env.node1_search_admin_user: esadmin
env.node1_search_admin_password: esadmin
env.node1_search_read_user: people
env.node1_search_read_password: peop1e
env.search_call_back_user: PS
env.search_call_back_password: PS
env.gateway_host: psvagabond.psadmin.io
env.gateway_port: 8000
env.default_local_node: PSFT_LM
acm_plugin_order:
- PTSFConfigureSrchInstance
Once you have filled out the plugin configuration vaules, and if you have multiple ACM hashes, you need to tell the DPK what order you want to execute the ACM plugins. You use the component_preboot_setup_order
to specify that order. Below, you’ll see we have 3 ACM sections we want to execute. These names are the hashes we created under the component_preboot_setup_order
section.
component_preboot_setup_order:
- searching
- push_notifications
PeopleBooks has a good list of delivered plugins and what configuration options are available.
To test the ACM with the DPK, we can tell Puppet to run only the ACM preboot or postboot step. Use the command below to execute a single DPK Profile:
puppet apply -e "include ::pt_profile::pt_tools_postboot_config" --trace --debug
This will run the postboot configuration you have defined in psft_customizations.yaml
without running anything else defined in Puppet.
If you use the --trace --debug
options when running Puppet, you can see the output from the PTEM_CONFIG app engine.
###########################################################
######## AUTOMATED CONFIGURATION MANAGER ########
###########################################################
PTEM_CONFIG:PTSFConfigureSrchInstance: Configure Peoplesoft system to talk to Search Server and assign roles
Configuring plug in : PTEM_CONFIG:PTSFConfigureSrchInstance
STATUS: SUCCESS
DESCRIPTION: NONE
SEVERITY: NONE
Configuration completed : PTEM_CONFIG:PTSFConfigureSrchInstance
****Completed environment configuration****
You can also get information about missing or invalid configuration from your ACM definition with the output:
###########################################################
######## AUTOMATED CONFIGURATION MANAGER ########
###########################################################
Validation of variables failed : PTEM_CONFIG:PTSFConfigureSrchInstance
Required property env.ptsf_search_instance is missing
Here is a gist with the Elasticsearch ACM Plugin configured. This Hiera data will configure an Elasticsearch instance in your environment.
This week on the podcast, Frank Dolezal joins us to talk about how he measure user testing and how it builds confidence when deploying updates. We also discuss self-service and fast database refreshes, and what skills to look for when hiring a PS Admin.
PeopleTools 8.56 has been released. This week on the podcast we talk about new features in 8.56 that we are excited about and dive into some of the changes to Deployment Packages.
This week on the podcast Dan and Kyle discuss changes they would make if they started working on the DPK for the first again. They also discuss setting up Elasticsearch clusters, monitoring Tuxedo queues with Elasticsearch and an annoying Bash on Windows bug.
App Capacity Visualization