#83 – DPK – What We’d Do Different Next Time

This week on the podcast Dan and Kyle discuss changes they would make if they started working on the DPK for the first again. They also discuss setting up Elasticsearch clusters, monitoring Tuxedo queues with Elasticsearch and an annoying Bash on Windows bug.

Show Notes

  • Bash on Windows VPN Bug @ 3:15
  • 9.2 Upgrade Process @ 10:00
  • Monitoring Tuxedo Queueing in Elasticsearch @ 13:30
  • Tuxbeat Project @ 17:00
  • Elasticsearch DPK @ 21:45
  • Elasticsearch Clustering @ 27:00
  • Puppet on Linux @ 32:00
  • DPK Extract Only @ 34:30
  • What we’d do differently with the DPK @ 37:30
    • Co-locating Regions
    • Admin-only Test Servers
    • Config Homes
    • Use Git for managing code and YAML changes
    • Integrate all Puppet code into Classes and Modules
    • DPK Course
    • Use ACM via DPK
    • Go Vanilla

App Capacity Visualization

#79 – UMRUG 2017 Spring Recap

This week Charlie Sinks joins Kyle and Dan to recap the 2017 Spring UMRUG meeting. We discussion transitioning to Fluid, selective adoption strategies, PTF improvements, Cloud Manager, Interaction Hub and Elasticsearch. The spring meeting was a great User Group meeting and generated excellent conversations.

Show Notes

  • Robin Velayedam (PeopleSoft HCM Strategy) Keynote @ 1:45
  • Workcenters or Nav Collections? @ 9:00
  • Going to 8.55 with or without Fluid Nav @ 12:00
  • Related Actions and Fluid Navigation @ 16:00
  • July HCM Image on 8.56 @ 20:15
  • Selective Adoption Strategies @ 22:00
  • Technical Roundtable @ 31:30
    • Elasticsearch Clustering
    • How easy to set up Elasticsearch?
  • PTF Improvements in 8.56 @ 35:30
  • Oracle 12c patch and tuning recommendations @ 38:15
  • How long does it take to apply images? @ 39:30
  • Security Identity Validation Framework and Secure Transactions @ 40:45
  • Interaction Hub Panel @ 43:15
    • Unified Navigation
    • Federated Search and Notifications
  • Cloud Manager in Interaction Hub @ 53:00
  • Applying Maintenance to Cloud Manager @ 56:00
  • Elasticsearch and Alternative Uses @ 59:00
  • Taking Control of Logs with Elasticsearch @ 62:00
  • Archiving IB Data @ 67:15

#75 – Selling Yourself

This week, Dan and Kyle talk about testing different web server configurations, using the ACM for Elasticsearch, and how mobile browsers work with websites. Then, they discuss different ways to promote yourself and your position to a boss or organization.

Show Notes

#65 – CPU Patching

This week on the podcast, Dan and Kyle talk about using web traffic data to analyze user activity, new information on Jolt Failover, and how we generate and distribute compare reports. Then, they discuss Critical Patch Updates and how they affect PeopleSoft Administrators.

Show Notes

#57 – REST Services

This week on the podcast, Dan and Kyle talk about Elasticsearch, using Powershell with SSH libraries, why the DPK doesn’t merge Hiera data, and how they format SQL files. Then Dan explains a new REST-based web service he built with new features in PeopleTools 8.55.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

#54 – Elasticsearch w/ Wayne Fuller

This week on the podcast we welcome back Wayne Fuller to talk about Elasticsearch. Wayne gives an excellent introduction into using Elasticsearch with PeopleSoft, some lesson’s learned from his testing, and highlights the benefits of switching to the new search tool.

One quick note – after recording, Wayne got clarification that you cannot cut over to Elasticsearch by index, only by user. We’ll talk with Wayne in an upcoming episode again about that change so everybody has accurate information. -dan

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

  • Testing Elasticsearch @ 1:00
  • Elasticsearch and IB Changes @ 5:00
  • Shards and Replicas @ 8:00
  • Elasticsearch Infrastructure @ 13:00
  • Elasticsearch Speed @ 17:15
  • Search Framework Architecture Changes @ 18:00
  • Integration Broker Impacts @ 25:00
  • Attachment Processing @ 28:00
  • Planning an Elasticsearch Deployment @ 31:30
  • Improving the Search Framework @ 38:30
  • Elasticsearch REST Interface @ 42:45
  • Health Center and Elasticsearch @ 46:00
  • Delete Processing @ 50:00
  • Beta Testing Experience @ 55:30

Deploy and Configure Elasticsearch

Elasticsearch support is here for PeopleTools 8.55. In PeopleTools 8.55.11, Elasticsearch and SES are supported search engines for PeopleSoft applications. For the next 18 months, PeopleSoft will support both search engines in 8.55. After those 18 months and starting in 8.56, Elasticsearch will be the only search engine supported with PeopleSoft. Since the PeopleSoft team announced that Elasticsearch would replace SES in December 2015, the community has been eagerly waiting for Elasticsearch support go live.

In the video below, we’ll provide an in-depth walk though of this post.

 

In this post, we’ll cover the installation of Elasticsearch, how to use the REST API, and some tips when using Elasticsearch.

DPK Only

Elasticsearch is the first PeopleTools component to be distributed only by Deployment Packages. There is no virtual CD option to install Elasticsearch. This also means that Elasticsearch is a separate download from PeopleTools. To download the Elasticsearch DPK, visit the PeopleTools Patch Documentation Home and click on the “Additional DPKs” tab. Ther is also an Elasticsearch Documentation Home with information about installing Elasticsearch, migrating from SES, and more.

Downloading

Like the PeopleTools and PeopleSoft Image DPKs, you download the .zip files from Oracle Support and run a bootstrap script to start the installation. The Elasticsearch DPK is only one .zip file, and is substantially smaller than other DPKs. I like using the getMOSPatch utility for downloading patches from MOS. Here is the command to download the Windows version of the Elasticsearch DPK:

java -jar getMOSPatch.jar patch=24924150 platform=233P download=all

This is the LInux version of the command:

java -jar getMOSPatch.jar patch=24924136 platform=226P download=all

Next, unzip the ELASTICSEARCH-DPK-WIN-2.3.2_00.zip file. Unlike other DPKs, there is only one .zip file. The .zip file contains setup files, the Elasticsearch binaries, and documentation.

Installation

After you unzip the file, you run the bootstrap script under scripts to start the installation:

cd .\scripts
psft-dpk-setup.ps1 -env_type es

When you run the Elasticsearch DPK, make sure to pass the -env_type es parameter. Without the parameter, the bootstrap script will fail looking for a file that doesn’t exist in the Elasticsearch DPK.

The bootstrap script will ask you a series of questions:

  • Do you want to install Puppet: Yes
  • Enter the ES Base folder: e:\psft
  • Elasticsearch Admin Password: Passw0rd1
  • Proxy User Password: Passw0rd1
  • Elasticsearch Cluster Name: srch-d1
  • Elasticsearch Port: 9200
  • Elasticsearch Discovery Host: ["127.0.0.1"]
  • Enter Java Heap Size: 2

The Elasticsearch Discovery Host is used when you are building a cluster with more than 1 node. In our case, we will enter the local machine’s IP address since we’ll run our nodes on only this machine. If you were building a cluster with multiple nodes on different machines, you would list the IP addresses for each server running Elasticsearch. After you answer the questions, the bootstrap script will start building the Elasticsearch instance.

In my testing on the current Elasticsearch DPK, there is a bug in the bootstrap script. The script ends early and doesn’t complete the installation. If this happens to you, it is easy to resolve.

First, let’s make sure the psft_es.yaml file is updated with out settings. Under C:\ProgramData\Puppetlabs\puppet\etc\data\ open the psft_es.yaml file. Find the section

#es_data
es_http_port:        

Enter 9200 for the es_http_port: value and save the file.

es_http_port:        9200

If you changed the Discover Host value, update that line as well and save the file.

discovery_zen_ping_unicast_hosts:           '["10.0.1.173"]'

Next, navigate to C:\ProgramData\Puppetlabs\puppet\etc\manifests. We’ll start Puppet and have it finish the Elasticsearch deployment and configuration.

puppet apply .\site.pp

At the end of the run, let’s verify that Elasticsearch is up and listening on port 9200.

netstat -an | findstr 9200

You should see something like this:

  TCP    10.0.1.173:9200        0.0.0.0:0              LISTENING
  TCP    127.0.0.1:49200        127.0.0.1:49201        ESTABLISHED
  TCP    127.0.0.1:49201        127.0.0.1:49200        ESTABLISHED

Administration

Before we jump into configuring PeopleSoft to use our Elasticsearch instance, I want to talk some basic Elasticsearch administration. Unlike the SES, there is not web-based admin console. Elasticsearch uses a REST-based API for all administration. Let’s look at what this means. In your browser, go to your Elasticsearch URL http://servername:9200/ and login with esadmin and the Administrative password you entered in the bootstrap script. You’ll get a response similar to this:

{
  "name" : "elastic11.psadmin.io",
  "cluster_name" : "srch-d1",
  "version" : {
    "number" : "2.3.2",
    "build_hash" : "b9e4a6acad4008027e4038f6abed7f7dba346f94",
    "build_timestamp" : "2016-04-21T16:03:47Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

If you want to get status of your Elasticsearch cluster, you would use this URL: http://servername:9200/_cluster/health?pretty=true.

{
  "cluster_name" : "srch-d1",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Currently, my cluster srch-d1 has a status of “green”. But it also has no data…

PeopleSoft Configuration

Now that Elasticsearch is running, it is time to configuration our PeopleSoft application to use it. There are a few requirements in PeopleSoft and they are similar to the SES requirements:

  • You must be running 8.55.11
  • The Integration Broker is configured and running
  • The integrationGateway.properties file has an encrypted value for secureFileKeystorePasswd
  • IB Domains are active
  • REST Service URLs are configured (PeopleTools > IB > Configuration > Service Configuration > Setup Target Locations)
  • A callback user account with these roles:
    • Search Developer
    • Search Server
    • Search Query Administrator
    • Search Administrator
    • PeopleSoft User

Search Instance

Go to PeopleTools > Search Framework > Administration > Search Instance. Starting with 8.55.11, we can have 2 or more Search Instances defined. The first search instance is named PSFT_DEFAULT. We’ll leave that configured to use SES. Create a new Search Instance named ELASTIC.

The Search Instance page looks the same as before, but with the addition of a “Search Provider” drop-down menu.

  1. Select “Elasticsearch”
  2. Enter the server name where you installed Elasticsearch
  3. Enter the Elasticsearch port (default is 9200)
  4. Enter esadmin for the User Name
  5. Enter the administrative password you set in the Bootstrap script
  6. Enter people for the Proxy Name
  7. Enter the proxy password you set in the bootstrap script

In the Call Back Properties:

  1. Enter the URL for the REST Target Connector you defined under “Service Configuration > Setup Target Locations”
  2. Enter the Call Back User’s name and password.

Verify all the Ping, Login, and Validate tests return successfully.

Last, we can set the order of the Search Instances. Since we haven’t fully tested Elasticsearch yet, set it to a lower priortiy until we are ready to release it to all users. Under “PeopleTools > Search Framework > Administration > Search Instance Administration”, set the “ELASTIC” instance to Priority 10.

Deploy Indexes

Go to “PeopleTools > Search Framework > Administration > Deploy/Delete Objects”. On this page, you have to select the Search Instance you want to deploy indexes to.

  1. Select “ELASTIC” for the Search Instance.
  2. Select the checkboxes for “PTPORTALREGISTRY” and “PTSEARCHREPORTS”.
  3. Click Deploy.

There seems to be bug in the Report Sync Issues action. If you select deployed indexes and click Report Sync Issues, it will return some errors. Ignore those errors for now; Elasticsearch works despite the “errors”.

  1. Navigate to “PeopleTools > Search Framework > Administration > Schedule Search Index”
  2. Create a new run control called PTPORTALREGISTRY_FULL
  3. Select “ELASTIC” as the Search Instance.
  4. Select the search index “PTPORTALREGISTRY”.
  5. Save the run control and run the process.

Once the process starts, you can view the Asynchronous Services page to see the messages sent to Elasticsearch. Navigate to “PeopleTools > Integration Broker > Service Operation Monitor > Monitoring > Asynchronous Services”. Once the message are successfull (Operation Instance and Subscription Contracts), it’s time to test.

Test Elasticsearch

Currently, SES is still our primary search provider. We can set up per-user search provicers so individual users can begin testing Elasticsearch.

  1. Go to “PeopleTools > Search Framework > Administration > Search Instance/User”
  2. Enter your user name and “ELASTIC”.
  3. Save the page.
  4. Log out of the application and log back in.
  5. In the search bar, search for User.

You should see search results returned from Elasticsearch! Once you are comfortable with Elasticsearch, simply change the priority of the search instances to activate Elasticsearch for everyone.

#53 – UMRUG 2016 Fall Recap

This week on the podcast, Charlie Sinks joins us to talk about everything we learned at the Upper Midwest Regional User Group meeting. We talk about PeopleTools 8.56, changes in Lifecycle Management, Fluid adoption, Elasticsearch, Dan’s DPK session, and much more.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

  • Benefits of the Roundtable @ 3:30
  • Roundtable Topics:
    • Extending 9.1 Support @ 5:15
    • Current Versions of PeopleTools @ 6:30
    • Fluid, Fluid, and more Fluid @ 8:45
    • Classic Plus @ 10:45
    • Classic Pages and Support @ 14:00
    • Elasticsearch in 8.55.11 @ 17:45
    • Getting started with Decoupled Homes @ 25:30
    • Conditional Navigation @ 32:00
    • 9.2 Upgrade and Tools? @ 35:45
    • PeopleTools 8.56 Details @ 39:30
      • Selective Adoption
      • ACM
      • Event Mapping
      • BI Publisher
      • The “PeopleSoft” name
  • Dan’s DPK Talk @ 52:00
  • LifeCycle Management
    • PTF, Usage Monitor and PTF Upgrade Utility @ 57:00
    • PUM Environments for Testing @ 61:30
    • How to determine the PeopleSoft Image you are on @ 67:45
    • Keeping older PeopleSoft Images @ 72:15
    • Best Practices for Lifecycle Management Red Paper @ 78:00
    • PUM Analytics @ 79:00

#51 – Elastic{on}

This week on the podcast, Kyle and Dan record from the Elastic{on} Conference in Chicago. We attended the conference to learn more about Elasticsearch and the Elastic platform. The tl;dr, we are excited about the future of the Elasticsearch and the Elastic ecosystem. We also talk about purging log files, Change Assistant and passwords, and sharing PSPing results.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

  • Windows 2016 is Out @ 1:00
  • PSPing Results @ 2:45
  • Extract Data Mover DML Statements @ 4:30
  • Wiki to OneNote @ 6:45
    • Control-Enter will create a new row in a OneNote table
  • Change Assistant and Passwords @ 16:00
  • Vagrant Introduction @ 20:45
  • How do you do it? Purging Log Files @ 23:15
  • Elastic{on} Recap @ 28:15
    • Expanding on Elasticsearch with PeopleSoft @ 41:00