Linux DPK: Dealing with Missing Required OS Packages

For those of you using the NativeOS Linux install for Update Images, you have probably come across this scenario. You start the DPK install and once you get to the Puppet installation section, the script comes to an abrupt end. What the heck! Looking in the log file, you quickly figure out your OS is missing some required packages. So now what?

In the PeopleSoft Deployment Packages for Update Images Installation document, task 2-3-3 walks you through how to get the required OS packages needed for Puppet. They make it clear that it is your job to obtain these packages and install them – you’re on your own. They then list a few steps on how to accomplish this. The steps pretty much come down to this:

  1. Install DPK
  2. DPK will fail on missing packages
  3. Find missing OS packages by reviewing the log
    • $DPK_INSTALL/setup/psft-dpk-setup.log
  4. Run DPK cleanup
  5. Install missing OS packages
  6. Install DPK again

Following the steps is pretty straight forward, but I don’t like having to manually dig through a log file and pick out the missing OS Packages. So, what I did is write a little shell script to extract them for me. This script will generate the list of missing OS packages and write it to a file. After reviewing the list, you can then use this file for installing the packages.

Here are the steps I follow to ensure I have all the needed OS packages when installing NativeOS Linux DPKs. These steps assume your current directory is $DPK_INSTALL/setup.

  1. Install DPK
  2. DPK will fail on missing packages
  3. Generate missing packages list
    • grep "is needed by" psft-dpk-setup.log | awk '{print $1;}' >> os-packages.list
  4. Run DPK cleanup
  5. Review list, edit if needed
    • vi os-packages.list
  6. Install missing OS packages
    • sudo yum install $(cat os-packages.list)
  7. Install DPK again

Unfortunately, you may have to repeat this process a few times to identify all the missing packages. Once I have gotten through a DPK install on a particular OS, I save off the os-packages.list file for safe keeping. I then make sure I install this list of packages to any new VM that I am doing a fresh DPK install on. Doing this before DPK installs will ensure we don’t see any missing OS package errors. I’m sure this list will need to be updated as time goes on and we see different versions of Puppet, etc in our DPKs.

Hopefully you found this post helpful! This little tidbit was pulled out of the PeopleSoft Deployment Package QuickStart course. Feel free to enroll in this FREE course today for more DPK goodness.

os-packages

Extending psadmin with psadmin-plus

I have created a helper menu script to extend the delivered psadmin program. The script is called psadmin-plus and I have created a repository for it on psadmin.io’s GitHub account. This was built as a self-study side project while I was on paternity leave this summer. I wanted to learn a little more about bash scripting and how to use git, and at the same time try to make a useful tool for myself and others to use. As of this writing, the tool is usable but far from complete. At the moment it only has support for Linux. I hope to make improvements over time and would invite others to summit issues on GitHub for questions, bugs or enhancement ideas. If anyone wants to contribute themselves, that would be great too!

There are two main uses for psadmin-plus. The first is actually calling the delivered psadmin program. The value add here is that it will auto-discover all your PS_CFG_HOME directories for you and source environment variables as needed. This all assumes you follow a few conventions, which should be documented in the GitHub readme or wiki pages. As mentioned in a previous blog post, this is useful if you use a single user to run your PeopleSoft environments. If you have a different user for each environment and source at login, then this feature doesn’t really help.

The second use is executing actions against multiple PS_CFG_HOMEs and domains at once. An example would be to stop all Process Scheduler domains on a machine. With this tool, you can do this with a few key strokes. You also have the option to execute now or later. If you select later, a script will generate to file. This allows you to run at a future time, maybe during a maintenance window. Again, there are a few assumed conventions that must be followed.

If you want to try it out for yourself, I have created a setup script to run against a PeopleSoft Image(VBox or Linux DPK install only). This will create a few extra PS_CFG_HOMEs and domains for you to play with in the menu. You can find instructions in the GitHub readme.

Below is a quick demo of psadmin-plus in use. For more information please see GitHub.

psadmin-plus-demo

Managing Environment Variables When Using Decoupled Homes

As a reader of this blog or a listener of the podcast, you know I am a user of both Linux and decoupled homes. Traditionally with a Linux PeopleSoft installation you need to source the delivered psconfig.sh to set your environment variables. When an entire environment was contained under its own PS_HOME, you could tweak this psconfig.sh file if customizations were needed without fear of impacting other environments. Now with decoupled homes, the PS_HOME directory will likely be shared, so changing the psconfig.sh file located there is a bad idea.

When switching to decoupled homes, I was looking for a good way to manage sourcing the psconfig.sh file and the different environment variables. While attending Alliance 2015, I saw a presentation given by Eric Bolinger from the University of Colorado. He was talking about their approach to decoupled homes and he had some really good ideas. The approach I currently use is mostly based on these ideas, with a few tweaks. The main difference is that he has a different Linux user account specific to each environment. With this approach, he is able to store the environment specific configuration file in the users home directory and source it at login time. This is similar to the approach Oracle suggests and uses with their PIs(see user psadm2). My organization didn’t go down the road of multiple users to run PeopleSoft. Instead, we have a single user that owns all the environments and we source our environment specific configuration file before we start psadmin. We use a psadmin wrapper script to help with this sourcing(which I will discuss and share in a future post). The main thing to keep in mind is regardless of how these files are sourced, the same basic approach can still be used.

The idea here is to keep as much delivered and common configuration in psconfig.sh as possible and keep environment specific customization in their own separate files. I like to keep these config files in a centralized location, that each server has access to via a NFS mount. I usually refer to this directory as $PSCONFIGS_DIR. What I do is copy the delivered psconfig.sh file to $PSCONFIGS_DIR and rename it psconfig.common.sh. I then remove any configurations that I know I will always want to set in our custom environment specific file, mainly PS_HOME. I then add any needed configuration that I know will be common across all environments (Another approach would be to create a new psconfig.common.sh file from scratch, set a few variables and then just source the delivered file cd $PS_HOME && . psconfig.sh. Either way works, but I like the cloning approach). This common file will be called at the end of every environment specific file. Remember to take care when making any changes to this file, as it will impact any environment calling it. It is also a good idea to review this file when patching or upgrading your tools.

Next for the environment specific files, I create a new file called psconfig.[env].sh. The environment name is listed in its filename. An example would be psconfig.fdev.sh. You could really choose any name for this, but I found this approach to be handy. In this file you will set the environment specific variables as needed, then end with calling psconfig.common.sh. Here is an example file:

This approach allows you to be a little more nimble when patching or upgrading. You can install new homes or middleware, then update the psconfig.[env].sh file and build new domains. When you get to go-live for Production, you can have the domains all built ahead of time. When ready, just update the config file, upgrade the database, and you are good to go!

One final note, regarding directory naming conventions. My organization tends to always have our PS_CFG_HOME directory match the environment or database name exactly, ie. fdev. I’m considering changing this, however. During our last Tools patching project, I found it a little awkward to prebuild the domains and still end up with same directory name. It seems to make much more sense to include the PeopleTools version in the directory name. That way you can prebuild the domains in a new PS_CFG_HOME, and when you go-live just blow the old home away. Another great idea I took away from Eric’s presentation was how to dynamically generate a PS_CFG_HOME directory name:

export PS_CFG_HOME=/opt/pscfg/$ENV-`$PS_HOME/bin/psadmin -v | awk '{print $2}'`

If you use this technique, you will want this to be the last line in your config file – after sourcing the common file. What this does is concatenate your environment name with the PeopleTools version, using the psadmin version command – ie. fdev-8.55.03. This will give you more clarity on what tools version the domains under this PS_CFG_HOME were built with and it will make it easier to prebuild your domains.

Switching File Attachment Storage

In previous posts I have talked about the File Attachment storage options in PeopleSoft. The three basic options are Database, FTP or HTTP. My organization initially went with HTTP, but now we are looking to move to Database storage. The requirement is to move not only future file attachments, but attachments from the past as well. At first I thought this would require a lot of work, including custom conversion AE programs, etc. However, there is a delivered process called Copy File Attachements that does all the heavy lifting for you. There is an online and batch method to run this, but I highly recommend the batch mode. There are a few other steps needed to fully convert all attachments, but it is fairly straight forward. Below are the steps I used in FSCM 9.2.17, PeopleTools 8.54.18.

Create new File Attachment Server

  1. Navigate to Main Menu > Set Up Financials/Supply Chain > Common Defictions > File Attachments > Administer File Attachments
  2. Click Add Database Server
  3. Record Name will default to PV_ATT_DB_SRV. The default is fine, but change if you would like.
  4. Set Pick Active Server to match the ID of your new server.
  5. Save. Future File Attachments will now be stored in your database.

Copy File Attachments

  1. If needed, create new URL for Database storage – PeopleTools > Utilities > Administration > URLs. Values URL Identifier: PSA_ATT_DB, URLID:record://PV_ATT_DB_SRV
  2. Navigate to run control page PeopleTools > Utilities > Administration > Administer File Processing > Copy File Attachments (Batch)
  3. Values Source: URL.PSA_ATT_HTTP, Destination:URL.PSA_ATT_DB
  4. Run the App Engine COPYATTS through Process Scheduler.
  5. This will take a long time to run, possibly hours depending on your attachment count.
  6. After completion, you will want to review the trace file AE_COPYATTS_[PIN].trc file. The AE should produce this automatically.
  7. I used this trick to generate a log of ONLY errors from the trace:

    grep "EvalCopyAttachments: failed getting file" AE_COPYATTS_*.trc > AE_COPYATTS.errors

  8. Take action on any errors that occurred.

Update previous File Attachment URL

  1. These steps point the old HTTP server setup to point to the new DB server.
  2. Navigate to PeopleTools > Utilities > Administration > URLs. Values URL Identifier
  3. Open the URL used for the previous File Attachment Server setup. Example: PSA_ATT_HTTP
  4. Change the URLID value to the new Database server URL values. Example: record://PV_ATT_DB_SRV
  5. I recommend adding comments describing this conversion process – especially if HTTP or FTP is in the URL ID and it now points to the DB. This can help avoid confusion in the future.

After we completed these steps, we decided to keep our old storage location around just in case we found any issues after the fact. We ended up renaming the directory path, just to make sure nothing was still referencing the old location. After a few weeks of no issues, we went ahead and destroyed this old storage location. As mentioned above, this was done in the FSCM application which has its own File Attachment framework built on top of the one Tools delivers. You should be able to take a similar approach with other applications, but the Create new File Attachment Server section above won’t be relevant. Instead, you can simply complete the Update previous File Attachment URL steps after your copy is complete.

Disabling the App Engine Server

I was having a discussion with another admin recently, and he questioned why we were running our process schedulers with Application Engine Server(PSAESRV) enabled. He pointed out that there is very little gained by running PSAESRV, so why have it turned on?

Now, you aren’t completely without loss in turning this off. Depending on what applications you are running and what your system’s needs are, you will have to make that decision for yourself. Check out Doc ID 651970.1 in MOS for a run down on the pros and cons for each method of running App Engines on your process schedulers.

 

Maintenance Page with Backdoor Login

A standard requirement when doing PeopleSoft maintenance is to change the normal sign in page. When the system is offline, we often like to display a maintenance page that informs users that the system will be down and prevents them from logging in. This is done easily enough by changing the signin.html page, adding a message and removing the login form.

However, there is one more requirement that is a little tricky. Once the maintenance is complete, there are often tasks that need to be completed before handing the system back to the end users. These can be configuration changes, running batch processes or simply completing validation that everything was applied correctly and is in working order. How can the core team sign on to the system and complete these tasks, all while preventing end user access?

I recently came up with a bit of JavaScript that gave us a backdoor to the system during our last PeopleTools upgrade. What the script does is hide the login form on the sign in page, preventing login.  The trick was that our core team knew the key to the backdoor. After the sign in page would load, when they pressed Ctrl+Space the login form would be reveled. The combination of keys was only known to the team, so they were the only ones able to get in. Keep in mind if you had some crafty end users this could obviously be worked around, but it did the trick for us.

The JavaScript and HTML to accomplish this is listed below.  Adding the script and HTML changes to your signin.html page is pretty straight forward. First include the file containing the JavaScript, or write it inline. You then need to wrap your login form elements in a <div> with an id=loginbox. I would suggest starting before the <div> containing ptLabelUserid and ending after the <div> containing the submit button. Lastly, you need to add hideLogin(true); to the body onload attribute.

Keep in mind the key doesn’t have to be Ctrl+Space, it can be any key combination really.You will see in the comments of the script a link to information about other keycodes that can be used.

Updated: 10/11/2016
Instead of hard coding a true value for your hideLogin() parameter, why not use a Custom Property set in the Web Profile? You can create any Custom Property you would like, for example: login.isLoginHidden and set to true. Then reference the property in your signin.hmtl page like this: hideLogin("<%=login.isLoginHidden%>"). This will allow you to toggle the hide login functionality by updating the web profile and bouncing the server.

Click here to see a working demo. Enjoy!


#1 – Decoupled Homes

I’m really happy to announce a new project we are launching today:

The PeopleSoft Administrator Podcast

Kyle Benson and I sit down and talk about all things PeopleSoft Admin. Kyle and I would meet for lunch every few weeks to talk about our latest issues at work and the latest features in PeopleTools. After a year of this, we thought it would be fun to record our conversations and share them.


In episode 1 of the PeopleSoft Administrator Podcast, we talk about decoupled homes and how to use them. They discuss the advantages of using them, how they manage each home and why you should look at using decoupled homes.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Links from this episode:

Rebuilding Tuxedo Domains

When using the command line to rebuild a Tuxedo domain, there is a new flag to keep your settings:

The –keepfeatures switch in the following command was introduced in PeopleTools 8.54.

For example:

psadmin –c import [PS_CFG_HOME] –d [old domain name] –n [new domain name] -keepfeatures

This came from the Best Practices guide for Mid-tier Deployment White Paper. At the end of the document, there is a good overview for the steps needed with a PeopleTools upgrade and the commands to help you script the upgrade.