8.55 – Build PeopleSoft Images with DPKs (Part 3)

As we have discussed in part 1 and part 2 of this discussion, Oracle is now shipping PeopleSoft Images with 8.55 and Deployment Kits(DPKs). Dan and I have talked quite a bit about our experiences using these DPKs with VirtualBox and NativeOS installments on Windows, so naturally Linux is up next. This is the OS that I spend most of my time in, so I have been excited to give it a try.

To get started, I took another read through the PeopleSoft Deployment Packages for Update Images Installation guide. Again, this can be found under Installation Documentation on the PUM Home Page. In this document it clearly states that Oracle Linux is supported for this installation. I normally don’t run Oracle Linux, so I was curious if it would work on other flavors. I gave it a try on both SuSE and Ubuntu without success. The bootstrap script basically fails right away, and I didn’t dig any further. So, I spun up a fresh lab install of Oracle Linux 7.2 and used that instead.

As with the other styles of DPK, the first step after you download the Linux .zip files is to extract the first file. Once extracted, you will find a setup directory which contains the bootstrap script psft-dpk-setup.sh. Before running this script, you will need to enable execution by running sudo chmod +x psft-dpk-setup.sh. After that, execute the bootstrap script and you are off and running.

I ran into a few issues with the installation, all related to dependencies(Update: More info here.). I ended up having to install all these packages to get past the issues:

sudo yum install libc.so.6 libgcc_s.so.1 libselinux.so.1 libxml2.so.2 libcrypto.so.10 libdb-5.3.so libffi.so.6 libgdbm.so.4 libncurses.so.5 libncursesw.so.5 libreadline.so.6 libssl.so.10 libtinfo.so.5 ruby rubygem oracle-rdbms-server-11gR2-preinstall

After getting these installed, it was time to give it another try. Running the bootstrap script again was not needed for this. Instead, I simply ran puppet apply site.pp which needed to be executed from the /etc/puppet/manifests directory. This time everything ran to success.

I chose the default initialization process, but you may want to make a few changes in your deployment. The changes you are most likely to make are related to security. By default the DPK will create 4 local user accounts: psadm1, psadm2, psadm3 and oracle2. This may not fit in with your security polices, so changing this could be crucial. In the installation guide, search for Task 6-1, which will walk you through the changes needed in your psft_customization.yaml file. If you do choose the defaults, then take a look at Task 7-1. This will cover all the default installation directories, as well as the default users and how they are used.

As always, if you ran into any other issues or have other observations related to the Linux NativeOS install, please let us know about it in the comments below!

Update: If you want to learn more about the DPK, check out our new PeopleSoft DPK QuickStart course. This free course will introduce you to the DPK, show you how to use the DPK with PeopleSoft Images, and show you how to customize the DPK for your servers.

 

#17 – PI’s and DPK’s (Part 2)

Kyle and Dan continue the discussion of DPK-based PeopleSoft Images. Dan shares a few frustrations with the new PI’s, we both learn a new way to generate trace files, and brainstorm how to manage the DPK’s YAML files.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

App Server Listening IP

The other day we had an app server that stopped responding to requests after it restarted. The domain was running, but our web server wasn’t able to talk to the app server. When we attempted to log in, we received the CHECK APPSERVER LOGS. THE SITE BOOTED WITH INTERNAL DEFAULT SETTINGS, BECAUSE OF: bea.jolt.ServiceException: Invalid Session error.

You can get this from a few errors:

  1. The Domain Connect Password doesn’t match.
  2. The Web Profile user or password is wrong (or locked).
  3. The app server domain is actually down.

But, none of these applied to us. We knew the domain was up, we hadn’t changed the Domain Connect Password, and the Web Profile user account was correct. I then ran a netstat -an command to make sure I could see the ports where listening. That’s when I found that our app server was listening on the wrong IP address.

In our environment, we have our regular IP addresses for the network, but our VM’s have a management network on a different IP range. The app server had booted up and was listening on the management network IP. I opened up the psappsrv.cfg file and the JSL and WSL listening IP’s were using the %PS_MACH% variable.

%PS_MACH%

The %PS_MACH% variable will grab the first network card’s IP address on the machine. With Windows, that’s not a consistent way to get the IP address if you have more than one netword card.

The fix is to assign an IP address to the JSL and WSL processes in the psappsrv.cfg file. You can assign a specific IP address using:

Address=//x.x.x.x

When assigning an IP address you have two options:

  • Listen to a specific IP address
  • Listen to all IP addresses

If you want to listen on all IP addresses, use 0.0.0.0 for the listening address value.

You may not want to do this if you have more than one network card on the machine. Often, the second (or more) network cards are for different networks that specific uses. In our case, the second network card was for VM management and I don’t want our app server accepting any connections from that network segment. Our network traffic should be limited to our internal PeopleSoft network segment. To do specify an IP, list the specific IP address you want your app server to listen on in the Address=// line.

Script WebLogic and Java Patches

In December, we talked quite a bit about patching Java and WebLogic on the blog and podcast. There was a WebLogic CVE, and then a patch, to apply. If you want a recap on the CVE and patching process, here are the posts:

While applying the patches, I wanted to script the process so patching would be consistent across all our servers. I pulled the scripts into a GitHub project for sharing and reuse. If you haven’t scripted a WebLogic patch, this would be a place to start. The scripts use PowerShell and built for WebLogic 10.3.6. So, they use SmartUpdate instead of OPatch. I also added in a Java patch to the process too. You could pull out the Java patch script to use by itself. One more note: all the patches, Java, and scripts were set to run from the folder e:\installers\weblogic1036-2015-CVE-Patches. If you use these for your environment, or just use them as a template, you’ll want to update those paths for your specific configuration.

There is nothing ground-breaking about these scripts 🙂 I can write scripts, but I’m not the best script developer out there. If you see places where the scripts need improvement, file an issue with the project or submit a pull request! The main goal with this project and post is to get others started with scripting. Scripting, even if the scripts are basic, can benefit administrators. I hope that this quick overview might help someone get started.

Scripts Overview

These scripts are writtin in PowerShell. If PowerShell scripts are not enabled on the server, run this command to allow PowerShell scripts to run:

set-executionpolicy unrestricted

  1. Install new SmartUpdate version (3.3.0)

    installSmartUpdate.ps1

    The silent.xml file is used for a silent install (no prompts). The installation directory is set to e:\oracle. If you want a different directory, change the value for “BEAHOME”. 1. Stop all web servers running on the server .stopPIAServices.ps1 The script looks for any Windows service that containts “*-PIA” in the name. If you have any WebLogic domains were not created by the

    installNTService script, you may need to shut them down by hand.

  2. Prepare and copy files from the weblogic1036-2015-CVE-Patches folder

    prepareFiles.ps1

    This script performs tasks to prepare different files for patching: On our servers, two files needed updates to run the Smart Update utility. registry.xml needed to remove a reference to Tuxedo; bsu.cmd needed an increase in memory to the Java Heap. The registry.xml file also contains a reference to the server where it was installed. The script will change that value based on the new server’s name. The original files are backed up first and a .bkp extension is added to the file name. The script also copies jdk-1.7.0_79 to our e:\java folder. If you want the new java version in a different location, you can change the path in the file.

  3. Apply both WebLogic patches The patches we are applying resolve the December 2015 CVE with WebLogic. If you are using these scripts for future patches, you’ll want to update the patch ID’s in the script.

    applyWebLogicPatches.ps1

    Both patches are applied to WebLogic using the bsu command. The script assumes your patches are in the folder e:\patches\cve-2015-4852. NOTE: On one of our servers, the second patch stalled during the “Checking for Conflicts” step. If the script stalls for more than a few minutes, hit Cntl-C.

  4. Update the JAVA_HOME values

    updateJavaVersion.ps1

    The JAVA_HOME value in the setEnv.cmd script will be updated to the new path. You must update this script for each server. The paths in the script are hard-coded. (The hard coding is an obvious candidate to fix next. Should be able to use the Get-ChildItem cmdlet to find all the setEnv.cmd files.)

  5. Update Registry value for JAVA_HOME

    updateRegistryJavaVersion.ps1

    The JAVA_HOME value in the Registry for each web service will be updated. You must update this script for each server. The paths in the script are hard-coded. (Again, another place for improvement. Need to find a search cmdlet for the Registry. Could look for -PIA in the service name.)

  6. Start all web servers running on the server.

    startPIAServices.ps1

    Again, this looks for all Windows services that have *-PIA in the name and starts them. That’s it.

The scripts are pretty simple, and you can write a wrapper script to run all the sub-scripts. That way you’d have one script to kick off. Or, you could add these into a tool like Rundeck to execute from a centralized place. Once you start down the path of scripting, many opportunities open up to speed up everyday tasks.

Fix NativeOS DPK Issues on Windows (Part 2)

In Part 1, Fix NativeOS DPK Issues on Windows, we covered the service changes and TNS Listener change to resolve issues with the NativeOS DPK. But, I realized today that I approached the TNS Listener issue the “old way”. We are working with DPK’s now and there is a different method to manage configuration changes: Puppet and Hiera.

In the new 8.55/DPK setup, configuration changes are handled in the psft_customizations.yaml Hiera file. psft_customizations.yaml overrides the default configuration provided by the DPKs. So, if we want to change the Oracle Listener port, let’s add it the YAML file.

psft_customizations.yaml

---

db_port:               1521

Don’t forget the --- at the top of the file. The --- is a file delimiter; you can include more than one YAML “file” inside a physical .YAML. file.

Copy the file to C:\ProgramData\PuppetLabs\Puppet\etc\data or /etc/puppet/data.

Fresh Install v. Existing Install

The theory behind Puppet (and the DPK’s) is that you describe your configuration and Puppet does the implementation. In our case above we have changed the configuration of db_port. If this was an existing installation of the DPK (e.g, you’ve already build HR 9.2 Image 16 using the DPK), we should be able to change that value. Unfortunately, the Puppet providers for PeopleSoft aren’t that robust. (I hope that is a feature that will be released). If you have already installed your PeopleSoft Image, you can make the change above, but when you run puppet apply site.pp the listener port is not updated.

But, if you are installing a NativeOS-based PeopleSoft you can include this change. When you run the bootstrap script, you are presented with the question “Do you want to continue the default initialization process?”. Answer No. (This [poorly worded] question means: Do you want to change any configuration?)

At this point, the bootstrap script will stop and you can copy your psft_customizations.yaml file into the data folder. Once your change is in place, we just start the Puppet process using puppet apply site.pp

You need to run the puppet apply site.pp command from C:\ProgramData\PuppetLabs\Puppet\etc\manifests or /etc/puppet/manifests.

#16 – PI’s and DPK’s

This week, Kyle and Dan talk about the new DPK-based PeopleSoft Images and the two options for building a PeopleSoft Image. Dan shares how to get Drop Down Navigation in 8.55 and why that’s a good thing, and Kyle finds a unreleased feature in Change Assistant.

Pasted image at 2016_02_12 01_34 PM

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

8.55 – 9.2 Upgrade Changes

If you are planning to upgrade to PeopleSoft 9.2 AND PeopleTools 8.55, there are changes in the upgrade process.

First, the PeopleTools upgrade is now a separate step. In the past, you could upgrade PeopleTools and the Application in one pass. Starting in 8.55, PeopleTools will be a separate pass. If you look at the FSCM 9.2 upgrade page, the 8.55 upgrade template has only the Application steps. To upgrade PeopleTools, you will generate the upgrade template from Change Assistant.

Second, the databases used in the upgrade process has changed. You no longer use a Copy of Current Demo to create the UPGCUST project. Instead, an app engine will create the UPGCUST project. The app engine will compare the LASTUPDOPRID for the objects and if the user is not PPLSOFT it will be added to the UPGCUST project. One impact, if you’ve deleted delivered objects from the system, you won’t see those in the compare report. (Not sure why you would do that, so this shouldn’t be an issue for you).

Third, you also need a PeopleSoft Image database for the Initial Pass. When you generate the Application Upgrade job in Change Assistant, you will also get a separate change package with all the Required for Upgrade fixes. These come from the PeopleSoft Image. To get a picture of how the new upgrade process works, the PeopleTools 8.55 PeopleBooks has a diagram on the process.    

Safely Using File Attachments

Allowing users to upload file attachments to multiple types of transactions within PeopleSoft is a very important PeopleTools feature. Adding files like resumes, expense receipts and other types of documents can be a huge win for organizations looking to move away from paper and fully leverage their ERP system. That being said, giving users the option to upload a file into your system has risks. There are plenty of people out there who would love to abuse this functionality.  You could be leaving your systems vulnerable to nasty viruses and malware.  Also, if not properly secured, these repositories of documents could be a data gold mine for people with malicious intent. Thankfully, PeopleTools does give us a few options to protect ourselves from these dangers. These are just a few considerations for safely setting up file attachments.

File Extension List

When we allow a user to upload a file related to a PeopleSoft transaction, we have a general idea what type of file to expect. Common file types would be images, Word documents or PDFs. Something like a .exe or .zip file is something that we would not expect, and should generally not allow. Well, good news! PeopleTools lets us specify what file types are allowed. By utilizing File Extension Lists, we can specify a list of extension types to accept or reject. These lists can then be associated with a URL by specifying the FILE_EXT_LIST property, using the URL Maintenance Page. When the file attachment architecture uses a URL for storage, it will look for this File Extension List and preform validation before a user is allowed to attach a file.

Virus Scanning

Limiting the file extensions of uploads is a good first step, but it really doesn’t get you very far. To really safe guard yourself from malicious file uploads, you need to be actively scanning all files for viruses. PeopleTools delivers a nice hook in the file attachment architecture to accomplish this. Buried in the PIA_HOME you will find the VirusScan.xml file. This can be used to configure virus scanning. Once this is done, all file attachment uploads will be scanned for viruses. If any are found, it will prevent the file from being saved on your system. Oracle doesn’t give us a lot of details on what exact virus scanning software is supported. However, it was built to work with ICAP supported scanners. So, if you own a virus scanner that works with ICAP then you are most likely in luck. There is also some level of configuration in the xml file to deal with different response codes.

Storage Locations

There are three delivered storage locations for file attachments in PeopleTools. All three options have their own pros and cons, so you will have to decide which to use depending on your needs. There will be a few things to consider with each approach to make sure each protocol is used safely.

Database

This is probably the most secure option. Generally the database is already very locked down, since the majority of other application data is stored there. No extra controls are really needed to make sure your files are safe. To be truly safe though, you really want to make sure this data is encrypted in flight and at rest. So if you do your due diligence with the security of your app as a whole, storing with this method takes care of itself.

FTP

This protocol has been around forever, but really has fallen out of favor in most organizations. You should under no circumstances be using plain old unencrypted FTP. PeopleTools does however support both FTPS and SFTP, a more secure option. To use these protocols you must use URL objects, not URL strings, in your PeopleCode. Take care to secure the attachments on the file system the FTP server is pointed to.

HTTP

The final storage option is HTTP. Similar to a Report Repository, this method leverages a PIA servlet(psfiletransfer) to handle the file storage. This setup is explained very well in

MOS Doc ID 1321581.1.  Similar to FTP, you should be using the secure option – HTTPS. To leverage this secure option you will need to make sure certificates are in place and your URL object is configured correctly. Review the MOS Doc ID 1408853.1 for further details. And again, make sure the attachments are secure in the file system.

Fix NativeOS DPK Issues on Windows

The NativeOS DPK’s for PeopleSoft Images are a welcome change. With the NativeOS DPK, you can deploy the PeopleSoft Images on Windows. This is great news for PeopleSoft customers who run Windows; the PeopleSoft Image can be on the same platform as the rest of your systems. The DPK process is great. There are so many advantages with DPK over the previous methods, but there are a few rough edges in the DPK process. Here are some changes I made to our HR 9.2 Image 16 virtual machine. We ran the NativeOS DPK on this Windows 2012 R2 server, with no changes to the configuration.

Services

The DPK installation will create a Windows service for the PIA, but no service is created for the Tuxedo domains. And, the services are set to “Manual”. So, when you reboot the VM, the web, app and batch server’s won’t start. This is an easy fix, but an annoying change to make when the DPK’s are supposed to automate everything. In psadmin,

  1. Select “Services Setup” option (6).
  2. Select “Configure Windows Service” (1).
  3. Enter “Y” to change the values.
  4. Enter “Y” to change the values (yes, you have to do this twice…)
  5. I changed the service delay to 10.
  6. Add APPDOM for the Application Server Domain.
  7. Add PRCSDOM for the Process Scheduler Domain.
  8. Leave the Search Server Domain blank
  9. Enter “N” – we are done with our chagnes.
  10. On the Services Administraton menu, select “Install Windows Service” (2).

Now we have a service for the Tuxedo domains, but the service is configured to start manually. Let’s go change that.

  1. From the Start menu, select Run and enter services.msc.
  2. Find the new service “PeopleSoft …” and double-click it.
  3. Change the Startup Type to “Automatic” and close the window.

We also want to change the PIA service to start automatically.

  1. Find the service “PIA Domain peoplesoft Service” and double-click it.
  2. Change the Startup Type to “Automatic” and close the window. The same is true for the Oracle services.

Let’s set those to start automatically.

  1. Find the service “OracleServiceCDBxxx” and double-click it.
  2. Change the Startup Type to “Automatic” and close the window.
  3. Find the service “OracleOraDB12cHomeTNSListener” and double-click it.
  4. Change the Startup Type to “Automatic” and close the window. The next time you reboot the server, your PeopleSoft and Oracle services will start automatically. But, you’ll probably run into this next error on the reboot.

Oracle Listener

After rebooting the NativeOS DPK installed server, the Oracle listener cannot find the database, which also prevents the app and batch server from starting. If you try to connect to the database you’ll get a

ORA-12514: TNS:listner does not currently know of service requested in connect descriptor error. This is a pretty easy fix (although I spent a few hours testing many different “fixes”). Change the port number for the Listener.

UPDATE: There is a different/better way to make this change if you haven’t run the DPK for your PeopleSoft Image yet

  1. In the tnsnames.ora file, change the port to 1521. The DPK default path is [base folder]dbtnsnames.ora
  2. In the listner.ora file, change the port to 1521. The DPK default path is [base folder]dboracle-server12.1.0.2networkadminlistener.ora. Restart the Oracle services and test your listener configuration. From the command prompt, try to connect to the database with SQL*Plus:

sqlplus sysadm/sysadm@hr92u016

Once you can connect to the database, you can boot the app and batch servers. Or, reboot the entire server and make sure all the services start automatically.

Final Thoughts

While there are some issues with the DPK process right now, it’s easy to get frustrated. The effort to build the DPK’s was incredible and a huge task. So, it’s not surprising that there are some rough edges with the DPK’s and the new PeopleSoft Image process. But, I’m trying to keep an eye on where these changes are taking us. The DPK’s are a very large undertaking from the PeopleTools group and will substantially improve the way we manage PeopleSoft installations.

#15 – Oracle Database w/ Tim Kroening-Smith (Part 2)

This week we finish our interview with Tim Kroening-Smith, an Oracle DBA and PeopleSoft Admin. Tim explains PeopleTools support for Partitioning, . We had such a fun time with Tim that we had to break the interview into two parts. Enjoy Part 2 this week!

If you want to get in contact with Tim, you can reach him at this email. Tim offers remote DBA and PeopleSoft Admin support.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes