#88 – Fast and Furious w/ Brad Carlson

This week Brad Carlson from the University of Minnesota joins us to talk about some recent projects he worked on. Brad explains why and how they consolidated their PeopleSoft database, an IB Monitoring tool that was recently open-sourced, and how the U of M can deploy CPU patches in 21 days.

Show Notes

  • Introducing Brad @ 1:00
  • Database Consolidation @ 5:30
  • Benefits of a Single Database @ 11:00
  • IB MonitorService @ 34:30
  • Distributing Notifications @ 53:30
  • Fast CPU Patching (21 Days!) @ 58:30

#81 – Database Statistics w/ David Kurtz

This week David Kurtz joins us to talk about Database Statistics, his PS_GFC_STATS package, the PSCBO package, identifying redundant indexes and more. David also gives insight into why he created PS_GFC_STATS and how it compares to the PSCBO package.

Show Notes

Refreshes with Data Guard and Pluggable Databases

A PeopleSoft refresh is one of the more common tasks for a PeopleSoft Administrator or DBA. There are many ways to accomplish this task, but it usually involves a database restore from backup, custom SQL refresh scripts and potentially ACM steps. Depending on the level of effort put into the refresh scripts, there can also be manual steps involved. This approach is tried and true, but tends to lack the speed and flexibility that we are starting expect with the delivery of the PeopleSoft Cloud Architecture toolset. Nightly or Ad-hoc refresh environments and quickly provisioned temporary environments are just a few use cases that would benefit greatly from refresh process improvements. I have been doing some exploring in this area recently and would like to share a few thoughts. First, a quick overview of some Oracle tools and features that I have been leveraging.

Data Guard

Oracle Data Guard is a tool that gives you high availability, data protection and disaster recovery for your databases. At a high level, it consists of a primary database and one or more standby databases. These standby databases are transactionally consistent copies of the primary database. Therefore, if the primary database goes down, the standby can be switched to primary and your application can keep on rolling.

Physical vs. Snapshot Standby

There are multiple types of standby databases that can be used with Data Guard. I’d like to briefly explain the difference between Physical Standby and Snapshot Standby. A physical standby is a database that is kept in sync with a primary database via Redo Apply. The redo data is shipped from the primary and then applied to the physical standby. A snapshot standby is basically a physical standby that was converted to a snapshot, which is like a point in time clone of the primary. At this point we can use the snapshot to do development, testing, etc. When we are done with our snapshot, we can then convert it back to a physical standby and it will once again be in sync with the primary database. This is accomplished by taking a restore point when the snapshot conversion happens. The whole time the standby is in snapshot mode, the redo data is still being shipped from the primary. However, it is NOT being applied. Once we convert back to physical, the restore point is used to restore and then all waiting redo is applied.

Pluggable Databases

With Oracle 12c, we have the introduction of multitenant architecture. This architecture consists of Container(CDB) and Pluggable(PDB) databases. This setup makes consolidating databases much more efficient. It also gives us the ability to clone a PDB very easily. Cloning a PDB between different CDBs can even be done via a database link. Having a true multitenant setup does require additional licensing, but you can have a CDB-PDB setup without this extra licensing cost if you use a single instance(Only one PDB per CDB). Here is a great video overview of multitenant.

Refresh Approach

Now that we have an idea of what these tools and features gain us, let’s think about how to put them to use with database refreshes. Both of these approaches assume the use of Data Guard and PDBs. Having a true multitenant setup would be most efficient but a single instance setup will work just fine. I would recommend you have a dedicated standby database for your refreshes, versus using the same standby you rely on for HA\DR. It would also make sense for the standby to be located on the same storage as the PDBs you will be refreshing. Neither of these are requirements, but I think you will see better performance and lessen the risk to your HA\DR plan.

The use case we will use for this example is a sandbox PeopleSoft database. This sandbox will be scheduled to refresh nightly, giving the business an environment to test and troubleshoot in with data from the day before. The refresh could also be run adhoc, if there is a need during the business day. So the goal is to have this fully automated and complete as fast as possible.

Clone Standby Approach

This approach will be to take a snapshot of our refresh standby database and clone it, overlaying our previous sandbox PDB. After this is completed, we will need to run custom SQL scripts or ACM steps to prepare the refreshed PDB. Finally, we will restore the refresh standby back to a physical standby database. This blog post by Franck Pachot gives a quick overview of the SQL commands needed to accomplish most of these steps.

  1. Convert the refresh source physical standby to a snapshot standby.
  2. Open the refresh source PDB as read only.
  3. Create database link between the sandbox target CDB and the refresh source PDB.
  4. Drop the sandbox target PDB and create a clone from the refresh source PDB.
  5. Open the new clone sandbox PDB.
  6. Cleanup the sandbox PDB.
    • Check for errors.
    • Patch the PDB to the patch level of the CDB, if needed.
  7. Run custom SQL scripts or ACM steps against sandbox PDB for PeopleSoft setup.
  8. Convert the refresh PDB back to physical standby.

Snapshot Standby Approach

This approach is somewhat similar, except we won’t be doing any cloning. Instead, we will be using the actual snapshot standby itself as our database. Since we know this sandbox database will be refreshed nightly, we can stay in snapshot standby mode all day and then switch to physical standby mode briefly at night, applying redo data to sync up with our primary production database. After that is done, we will then switch back to snapshot mode and run our custom SQL scripts and ACM steps. This will require a dedicated standby database and should only be used with a frequent refresh schedule. Since the redo data continues to ship during snapshot standby mode, the redo data will start to backup. The volume of this redo data backing up could become an issue if it gets too large, so you will need to do some analysis to make sure you can handle it based on your refresh interval.

  1. Create a sandbox PDB as a physical standby, with primary database being production.
  2. Convert sandbox to a snapshot standby.
  3. Run custom SQL scripts or ACM steps against sandbox for PeopleSoft setup.
  4. Use the snapshot standby sandbox PDB as your normal database; connecting app and batch domains, etc.
  5. Wait until next refresh interval.
  6. Convert sandbox from snapshot standby to physical standby.
    • Restore point will be used and redo data applied, syncing up with current primary database state in production.
  7. Covert sandbox from physical standby to snapshot standby.
  8. Run custom SQL scripts or ACM steps against sandbox for PeopleSoft setup.
  9. Repeat.

Conclusion

Those are just two ideas, but you can see that there are probably many variations of these approaches that will work. Leveraging Data Guard and PDBs really gives you many options to choose from. I have been using the Clone Standby approach recently and have packaged up the scripts, including bouncing app\batch domains, in Oracle Enterprise Manager as a job. This gives me push button refreshes with a turn around time under 20 minutes. I have been able to provide adhoc refreshes for emergency production troubleshooting to the business multiple times in just a few months since implementing this approach. This is a very powerful tool to have and is well worth the effort to get your refreshes fast, efficient and automated.

Automate PeopleTools Database Patches

With PeopleTools 8.55, applying database PeopleTools patches with Change Assistant is easy to automate. In 8.54, Change Assistant gained the ability to apply patches from the command line. But with 8.55 and the PeopleTools Client DPK, the process is even better.

There are three main steps to getting your PeopleTools 8.55 database patched:

  1. Install the new PeopleTools Client (and Change Assistant version)
  2. Configure your target environment
  3. Run the PeopleTools patch command

Let’s look at each step.

PeopleTools Client and Patch Files

The first step is to install the updated PeopleTools client using the Client DPK (PeopleTools Patch Client Install section). When you install the client, make sure you select “PeopleTools Patch” or “Full PeopleTools Upgrade”. Both of those options will create a PTP folder under the client home. The PTP folder will contain the PeopleTools Patch (PTP) .zip files and Change Assistant expects those files in the PTP folder. Make sure you install the new Change Assistant version too. With 8.55, you can install more than one version of Change Assistant; I used the convention C:\Program Files\PeopleSoft\Change Assistant-8.55.xx for the install path.

1-ptclient.gif

Change Assistant Configuration

In the updated version of Change Assistant, you need to create an environment definition for the target database. If the environment existed in an older version of Change Assistant, use the new Import feature under “Tools > Export/Import Change Assistant Settings”. If you don’t have an environment defined, to to “File > New Database” to create it. For your target database, change the PS_HOME directory to the new PeopleTools Client folder. Verify and Save the environment definition.

2-caenv

Automate

Open a command prompt and navigate to C:\Program Files\PeopleSoft\Change Assistant. We’ll kick off the PeopleTools Patch using the changeassistant.bat file. There are 4 parameters we specify:

  • -MODE: The Change Assistant mode to run (UM for Update Manager mode)
  • -ACTION: The change action to run (PTPAPPLY for a patch or PTUAPPLY for an upgrade)
  • -TGTENV: The target environment name
  • -UPD: PeopleTools Patch ID (PTP855xx)

3-ptpapply

Check the softwareupdatePTP855XX{-DBNAME}IP output folder (defined under Tools > Options > Output Directory in Change Assistant) for the logs. Change Assistant will create the usual logs, but you will also find a job.log file. job.log is the same output that you normally see in the left-pane inside Change Assistant.

I applied PeopleTools 8.55.08 (to an 8.55.05) database and the patch job took less than 10 minutes.

#38 – Automation and more w/ Nate Werner

This week we interview Nate Werner. Nate talks about many of the unique solutions he created to make this PeopleSoft environments secure, standardized and automated. We cover a lot of topics and Nate shows why he is a top-notch PS Admin!

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

  • Supporting CS and Integration Hub setup @ 3:00
  • Shibboleth and Oracle Traffic Director @ 7:30
  • Sign-in As @ 10:30
  • Experience with GH ERP Firewall @ 12:30
  • 2 Factor Auth @ 14:30
  • Ansible Automation and Standardization @ 18:15
  • Automating PeopleSoft Image Deployment @ 26:00
  • Stacking Application Architecture @ 30:00
  • Supporting Students v. Business Users @ 35:00
  • Supporting Mobile and Branding @ 37:00
  • Database Consolidation @ 38:45
  • Managing Releases and Downtime @ 43:30
  • Managing Files without Server Access @ 47:00
  • Everything as a Share/Mount @ 48:45
  • Smart Automation @ 56:00
  • Learning Puppet/Embracing DevOps @ 58:15