#97 – Dozens of Us

This week on the podcast, Dan and Kyle discuss strategies to organize and manage Event Mapping code, writing code that other people will use, trying to work with Push Notifications and custom ACM modules. Kyle ends the podcast with a funny story about searching for images.
 

Show Notes

 

#74 – Killing COBOL

This week on the podcast, Kyle and Dan talk about planning PeopleTools and Catch-up projects, BI Publisher security, and how to turn off excessive BI Publisher logging. We also talk about slowly killing COBOL with PeopleSoft (it’s not dead yet) and using multiple Change Assistant installations.

Show Notes

#65 – CPU Patching

This week on the podcast, Dan and Kyle talk about using web traffic data to analyze user activity, new information on Jolt Failover, and how we generate and distribute compare reports. Then, they discuss Critical Patch Updates and how they affect PeopleSoft Administrators.

Show Notes

#62 – PeopleTools Patch Testing

This week on the podcast, Dan and Kyle talk about load balancing all environments or some environments, Diagnostic Plugins and syntax coloring code. Then, they dive into the getting current and how to test PeopleTools Patches.

Show Notes

Automate PeopleTools Database Patches

With PeopleTools 8.55, applying database PeopleTools patches with Change Assistant is easy to automate. In 8.54, Change Assistant gained the ability to apply patches from the command line. But with 8.55 and the PeopleTools Client DPK, the process is even better.

There are three main steps to getting your PeopleTools 8.55 database patched:

  1. Install the new PeopleTools Client (and Change Assistant version)
  2. Configure your target environment
  3. Run the PeopleTools patch command

Let’s look at each step.

PeopleTools Client and Patch Files

The first step is to install the updated PeopleTools client using the Client DPK (PeopleTools Patch Client Install section). When you install the client, make sure you select “PeopleTools Patch” or “Full PeopleTools Upgrade”. Both of those options will create a PTP folder under the client home. The PTP folder will contain the PeopleTools Patch (PTP) .zip files and Change Assistant expects those files in the PTP folder. Make sure you install the new Change Assistant version too. With 8.55, you can install more than one version of Change Assistant; I used the convention C:\Program Files\PeopleSoft\Change Assistant-8.55.xx for the install path.

1-ptclient.gif

Change Assistant Configuration

In the updated version of Change Assistant, you need to create an environment definition for the target database. If the environment existed in an older version of Change Assistant, use the new Import feature under “Tools > Export/Import Change Assistant Settings”. If you don’t have an environment defined, to to “File > New Database” to create it. For your target database, change the PS_HOME directory to the new PeopleTools Client folder. Verify and Save the environment definition.

2-caenv

Automate

Open a command prompt and navigate to C:\Program Files\PeopleSoft\Change Assistant. We’ll kick off the PeopleTools Patch using the changeassistant.bat file. There are 4 parameters we specify:

  • -MODE: The Change Assistant mode to run (UM for Update Manager mode)
  • -ACTION: The change action to run (PTPAPPLY for a patch or PTUAPPLY for an upgrade)
  • -TGTENV: The target environment name
  • -UPD: PeopleTools Patch ID (PTP855xx)

3-ptpapply

Check the softwareupdatePTP855XX{-DBNAME}IP output folder (defined under Tools > Options > Output Directory in Change Assistant) for the logs. Change Assistant will create the usual logs, but you will also find a job.log file. job.log is the same output that you normally see in the left-pane inside Change Assistant.

I applied PeopleTools 8.55.08 (to an 8.55.05) database and the patch job took less than 10 minutes.

Managing Environment Variables When Using Decoupled Homes

As a reader of this blog or a listener of the podcast, you know I am a user of both Linux and decoupled homes. Traditionally with a Linux PeopleSoft installation you need to source the delivered psconfig.sh to set your environment variables. When an entire environment was contained under its own PS_HOME, you could tweak this psconfig.sh file if customizations were needed without fear of impacting other environments. Now with decoupled homes, the PS_HOME directory will likely be shared, so changing the psconfig.sh file located there is a bad idea.

When switching to decoupled homes, I was looking for a good way to manage sourcing the psconfig.sh file and the different environment variables. While attending Alliance 2015, I saw a presentation given by Eric Bolinger from the University of Colorado. He was talking about their approach to decoupled homes and he had some really good ideas. The approach I currently use is mostly based on these ideas, with a few tweaks. The main difference is that he has a different Linux user account specific to each environment. With this approach, he is able to store the environment specific configuration file in the users home directory and source it at login time. This is similar to the approach Oracle suggests and uses with their PIs(see user psadm2). My organization didn’t go down the road of multiple users to run PeopleSoft. Instead, we have a single user that owns all the environments and we source our environment specific configuration file before we start psadmin. We use a psadmin wrapper script to help with this sourcing(which I will discuss and share in a future post). The main thing to keep in mind is regardless of how these files are sourced, the same basic approach can still be used.

The idea here is to keep as much delivered and common configuration in psconfig.sh as possible and keep environment specific customization in their own separate files. I like to keep these config files in a centralized location, that each server has access to via a NFS mount. I usually refer to this directory as $PSCONFIGS_DIR. What I do is copy the delivered psconfig.sh file to $PSCONFIGS_DIR and rename it psconfig.common.sh. I then remove any configurations that I know I will always want to set in our custom environment specific file, mainly PS_HOME. I then add any needed configuration that I know will be common across all environments (Another approach would be to create a new psconfig.common.sh file from scratch, set a few variables and then just source the delivered file cd $PS_HOME && . psconfig.sh. Either way works, but I like the cloning approach). This common file will be called at the end of every environment specific file. Remember to take care when making any changes to this file, as it will impact any environment calling it. It is also a good idea to review this file when patching or upgrading your tools.

Next for the environment specific files, I create a new file called psconfig.[env].sh. The environment name is listed in its filename. An example would be psconfig.fdev.sh. You could really choose any name for this, but I found this approach to be handy. In this file you will set the environment specific variables as needed, then end with calling psconfig.common.sh. Here is an example file:

This approach allows you to be a little more nimble when patching or upgrading. You can install new homes or middleware, then update the psconfig.[env].sh file and build new domains. When you get to go-live for Production, you can have the domains all built ahead of time. When ready, just update the config file, upgrade the database, and you are good to go!

One final note, regarding directory naming conventions. My organization tends to always have our PS_CFG_HOME directory match the environment or database name exactly, ie. fdev. I’m considering changing this, however. During our last Tools patching project, I found it a little awkward to prebuild the domains and still end up with same directory name. It seems to make much more sense to include the PeopleTools version in the directory name. That way you can prebuild the domains in a new PS_CFG_HOME, and when you go-live just blow the old home away. Another great idea I took away from Eric’s presentation was how to dynamically generate a PS_CFG_HOME directory name:

export PS_CFG_HOME=/opt/pscfg/$ENV-`$PS_HOME/bin/psadmin -v | awk '{print $2}'`

If you use this technique, you will want this to be the last line in your config file – after sourcing the common file. What this does is concatenate your environment name with the PeopleTools version, using the psadmin version command – ie. fdev-8.55.03. This will give you more clarity on what tools version the domains under this PS_CFG_HOME were built with and it will make it easier to prebuild your domains.

#21 – Temp Tables w/ David Kurtz

We are excited to have David Kurtz join us! David wrote the book PeopleSoft for the Oracle DBA and is a popular blogger. We talked with David for almost 2 hours, so we are breaking his interview into 3 parts. In Part 1, we talk about Temp Tables, PSAESRV, and more. Dan and Kyle also talk about Microsoft and Linux, and Dan’s DPK patching experience with PeopleTools 8.55.03.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

Script WebLogic and Java Patches

In December, we talked quite a bit about patching Java and WebLogic on the blog and podcast. There was a WebLogic CVE, and then a patch, to apply. If you want a recap on the CVE and patching process, here are the posts:

While applying the patches, I wanted to script the process so patching would be consistent across all our servers. I pulled the scripts into a GitHub project for sharing and reuse. If you haven’t scripted a WebLogic patch, this would be a place to start. The scripts use PowerShell and built for WebLogic 10.3.6. So, they use SmartUpdate instead of OPatch. I also added in a Java patch to the process too. You could pull out the Java patch script to use by itself. One more note: all the patches, Java, and scripts were set to run from the folder e:\installers\weblogic1036-2015-CVE-Patches. If you use these for your environment, or just use them as a template, you’ll want to update those paths for your specific configuration.

There is nothing ground-breaking about these scripts 🙂 I can write scripts, but I’m not the best script developer out there. If you see places where the scripts need improvement, file an issue with the project or submit a pull request! The main goal with this project and post is to get others started with scripting. Scripting, even if the scripts are basic, can benefit administrators. I hope that this quick overview might help someone get started.

Scripts Overview

These scripts are writtin in PowerShell. If PowerShell scripts are not enabled on the server, run this command to allow PowerShell scripts to run:

set-executionpolicy unrestricted

  1. Install new SmartUpdate version (3.3.0)

    installSmartUpdate.ps1

    The silent.xml file is used for a silent install (no prompts). The installation directory is set to e:\oracle. If you want a different directory, change the value for “BEAHOME”. 1. Stop all web servers running on the server .stopPIAServices.ps1 The script looks for any Windows service that containts “*-PIA” in the name. If you have any WebLogic domains were not created by the

    installNTService script, you may need to shut them down by hand.

  2. Prepare and copy files from the weblogic1036-2015-CVE-Patches folder

    prepareFiles.ps1

    This script performs tasks to prepare different files for patching: On our servers, two files needed updates to run the Smart Update utility. registry.xml needed to remove a reference to Tuxedo; bsu.cmd needed an increase in memory to the Java Heap. The registry.xml file also contains a reference to the server where it was installed. The script will change that value based on the new server’s name. The original files are backed up first and a .bkp extension is added to the file name. The script also copies jdk-1.7.0_79 to our e:\java folder. If you want the new java version in a different location, you can change the path in the file.

  3. Apply both WebLogic patches The patches we are applying resolve the December 2015 CVE with WebLogic. If you are using these scripts for future patches, you’ll want to update the patch ID’s in the script.

    applyWebLogicPatches.ps1

    Both patches are applied to WebLogic using the bsu command. The script assumes your patches are in the folder e:\patches\cve-2015-4852. NOTE: On one of our servers, the second patch stalled during the “Checking for Conflicts” step. If the script stalls for more than a few minutes, hit Cntl-C.

  4. Update the JAVA_HOME values

    updateJavaVersion.ps1

    The JAVA_HOME value in the setEnv.cmd script will be updated to the new path. You must update this script for each server. The paths in the script are hard-coded. (The hard coding is an obvious candidate to fix next. Should be able to use the Get-ChildItem cmdlet to find all the setEnv.cmd files.)

  5. Update Registry value for JAVA_HOME

    updateRegistryJavaVersion.ps1

    The JAVA_HOME value in the Registry for each web service will be updated. You must update this script for each server. The paths in the script are hard-coded. (Again, another place for improvement. Need to find a search cmdlet for the Registry. Could look for -PIA in the service name.)

  6. Start all web servers running on the server.

    startPIAServices.ps1

    Again, this looks for all Windows services that have *-PIA in the name and starts them. That’s it.

The scripts are pretty simple, and you can write a wrapper script to run all the sub-scripts. That way you’d have one script to kick off. Or, you could add these into a tool like Rundeck to execute from a centralized place. Once you start down the path of scripting, many opportunities open up to speed up everyday tasks.

#13 – PeopleTools Patching

This week, Kyle shares his lessons learned from a recent PeopleTools patching project, we discuss the pro’s and con’s of Unified Navigation, new PeopleSoft Rundeck plug-ins, and changes to the 9.2 upgrade with PeopleTools 8.55.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

How to Apply WebLogic Patches

Oracle has released a patch for the latest CVE against WebLogic, so I wanted to walk though the steps to apply the patch to WebLogic and show how to use Smart Update. Smart Update is the utility used by WebLogic to apply patches to your installation.

UPDATE 12/8/2015 Thanks to Matt Tremblay for pointing out, WebLogic 12.1.2+ is now using OPatch for WebLogic patching. Look for an second WebLogic patching post soon about using OPatch with WebLogic.

Smart Update 3.3.0

Version 3.3.0 is the latest version and is included with WebLogic 10.3.5 and later. If you launch Smart Update and find that its an older version, go grab version 3.3.0 (Patch 12426828).

Launching Smart Update

On Windows, if you chose to create a Program Group, you can launch Smart Update from the Start Menu under the “Oracle WebLogic” folder. Or, you can launch it from the command line:

%BEA_HOME%utilsbsubsu.cmd

The first time you run Smart Update, it may ask you to provide a %BEA_HOME% path. Give the path to your BEA Home (e.g, e:oracle).

If you receive an error: “Unable to locate any supported product installations” or “The BEA Home directory selected does not contain any supported patch targets”, check out MOS Documents 946541.1 or 1063605.1 for the fix.

Applying Patches

In the Smart Update window, you will see the installed applications in the left pane. Make sure “WebLogic” is selected. On the right, the top pane shows you patches that have been applied. The lower pane displays patches in your download directory that are waiting to be applied.

By default, Smart Update will look for patches under %BEA_HOME%utilsbsucache_dir for patches. To start Smart update and have it look at a different path, use the -patch_download_dir=[path] flag or select File > Preferences to change the directory.

To apply a patch, click the green arrow in the “Downloaded Patches” pane for the patch. Smart Update will check for patch conflicts and the apply the patch.

Command Line

You can also run Smart Update from the command line. This is great when you have multiple servers to patch. Running bsu.cmd -help will give you all the options you need when scripting.

Let’s Fix CVE-2015-4852

Since we have new patches to fix CVE-2015-4852 (T3/Java Deserialization), let’s use those as our example. Go to this page to find the applicable patch (or patches if you are on 10.3.6) to apply.

Extract Patches

Download the patches you need and unzip them. Copy the .jar and .xml files from the patch folders to your web server. I put the patch files under e:patchescve-2015-4852 on the web server. We will tell Smart Update to use this directory. (Since we are running WebLogic 10.3.6, there are two patches to install in our example.).

Apply Patches

Before you apply any patches, make sure to stop any web servers running on the server. If don’t, Smart Update won’t be able to patch .jar files that are in use.

Let’s run Smart Update from the command line. Open a command prompt and navigate to %BEA_HOME%utilsbsu. We need to pass these values to the bsu program:

  • -install
  • -patch_download_dir=e:patchescve-2015-4852
  • -patchlist=EJUW (note, this is not the patch number, but the PSU Patch ID)
  • -prod_dir=e:oraclewlserver_10.3
  • -verbose

So, my command to apply the first CVE patch looks like this:

bsu -install -patch_download_dir=e:patchescve-2015-4852 -patchlist=EJUW -prod_dir=e:oraclewlserver_10.3 -verbose

I had to change the memory settings for Smart Update. In the bsu.cmd file, I modified the set MEM_ARGS line:

set MEM_ARGS=-Xms512m -Xmx1024m -XX:PermSize=64m -XX:MaxPermSize=128m -Xss512k

Smart Update will give you a “Success” message, or an error message. Next, let’s apply the second CVE patch:

bsu -install -patch_download_dir=e:patchescve-2015-4852 -patchlist=ZLNA -prod_dir=e:oraclewlserver_10.3 -verbose

Verify WebLogic Version

To verify WebLogic has the new patches, we can run two commands. The first command is to set the environment with:

%WL_HOME%serverbinsetWLSEnv.cmd

Then, run the command:

java weblogic.version

The output will look similar to this:

[code lang=”text”]
WebLogic Server Temporary Patch for BUG22248372 Tue Nov 24 00:35:04 MST 2015
WebLogic Server 10.3.6.0.12 PSU Patch for BUG20780171 THU JUN 18 15:54:42 IST2015
WebLogic Server 10.3.6.0 Tue Nov 15 08:52:36 PST 2011 1441050
[/code]

Another option to check the version of WebLogic is using the Smart Update utility:

bsu -prod_dir=e:oraclewlserver_10.3 -status=applied -verbose -view

The output shows that we have applied the 10.3.6.0.12 and CVE patch. Now, restart all your web servers and start testing!