#86 – Usability Testing

This week on the podcast Kyle talks about Usability Testing with his PeopleTools upgrade and moving to Fluid Navigation. Dan and Kyle also talk about Unified Navigation and the Node Network, where the Portal fits in and beefing up the Integration Broker.

Show Notes

  • Excited about Fluid @ 1:00
  • Discovering Homepage Tabs @ 3:15
  • How to Approach Usability Testing @ 9:00
  • Usability Participants @ 13:45
  • Accessibility Testing @ 18:00
  • Comparing Usability Testing before Fluid @ 22:45
  • Personalizing Homepages @ 32:00
  • Unified Navigation @ 38:30
  • Getting Lost with Unified Navigation @ 47:00
  • Integration Monitor @ 50:30
  • Making Search a Critical Component @ 55:00
  • Beefing Up the IB with Unified Nav @ 57:30

#79 – UMRUG 2017 Spring Recap

This week Charlie Sinks joins Kyle and Dan to recap the 2017 Spring UMRUG meeting. We discussion transitioning to Fluid, selective adoption strategies, PTF improvements, Cloud Manager, Interaction Hub and Elasticsearch. The spring meeting was a great User Group meeting and generated excellent conversations.

Show Notes

  • Robin Velayedam (PeopleSoft HCM Strategy) Keynote @ 1:45
  • Workcenters or Nav Collections? @ 9:00
  • Going to 8.55 with or without Fluid Nav @ 12:00
  • Related Actions and Fluid Navigation @ 16:00
  • July HCM Image on 8.56 @ 20:15
  • Selective Adoption Strategies @ 22:00
  • Technical Roundtable @ 31:30
    • Elasticsearch Clustering
    • How easy to set up Elasticsearch?
  • PTF Improvements in 8.56 @ 35:30
  • Oracle 12c patch and tuning recommendations @ 38:15
  • How long does it take to apply images? @ 39:30
  • Security Identity Validation Framework and Secure Transactions @ 40:45
  • Interaction Hub Panel @ 43:15
    • Unified Navigation
    • Federated Search and Notifications
  • Cloud Manager in Interaction Hub @ 53:00
  • Applying Maintenance to Cloud Manager @ 56:00
  • Elasticsearch and Alternative Uses @ 59:00
  • Taking Control of Logs with Elasticsearch @ 62:00
  • Archiving IB Data @ 67:15

#63 – Revisiting PS_APP_HOME

This week, Dan and Kyle talk about writing PTF tests for PeopleTools, running multiple IB domains and trying A CM again. Then, we revisit our strategies for managing PS_APP_HOME when applying selective maintenance.

Show Notes

#54 – Elasticsearch w/ Wayne Fuller

This week on the podcast we welcome back Wayne Fuller to talk about Elasticsearch. Wayne gives an excellent introduction into using Elasticsearch with PeopleSoft, some lesson’s learned from his testing, and highlights the benefits of switching to the new search tool.

One quick note – after recording, Wayne got clarification that you cannot cut over to Elasticsearch by index, only by user. We’ll talk with Wayne in an upcoming episode again about that change so everybody has accurate information. -dan

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

  • Testing Elasticsearch @ 1:00
  • Elasticsearch and IB Changes @ 5:00
  • Shards and Replicas @ 8:00
  • Elasticsearch Infrastructure @ 13:00
  • Elasticsearch Speed @ 17:15
  • Search Framework Architecture Changes @ 18:00
  • Integration Broker Impacts @ 25:00
  • Attachment Processing @ 28:00
  • Planning an Elasticsearch Deployment @ 31:30
  • Improving the Search Framework @ 38:30
  • Elasticsearch REST Interface @ 42:45
  • Health Center and Elasticsearch @ 46:00
  • Delete Processing @ 50:00
  • Beta Testing Experience @ 55:30

#32 – UMRUG 2016 Spring Meeting Recap

This week Dan gives a recap of the spring Upper Midwest Regional User Group meeting. The talks and roundtable discussions were great and we learned why PS_CUST_APP_HOME is gone in 8.55. Kyle and Dan also discuss IB development and the need for better App Class documentation.

We want to make this podcast part of the community discussion on PeopleSoft administration. If you have comments, feedback, or topics you’d like us to talk about, we want to hear from you! You can email us at podcast@psadmin.io, tweet us at @psa_io, or use the Twitter hashtag #psadminpodcast.

You can listen to the podcast here on psadmin.io or subscribe with your favorite podcast player using the URL below, or subscribe in iTunes.

Podcast RSS Feed

Show Notes

  • Fun Twitter Accounts @ :30
  • Inconsistent Fluid Nav @ 3:45
  • IB Development @ 6:30
    • Helper Classes and JavaDoc
    • HCR_PUBLICATION_RULES:PersonBasicPublication
    • Undocumented App Class – %metadata
  • Kyle’s Oh No! Story @ 13:30
  • UMRUG Recap @ 20:00
    • New Development Tools Talk and Supplemental Data
    • NativeOS DPK Image upgrades
    • Security Automation Tool
    • Interface Data Stores
    • QAS for interfaces
    • PeopleTools/PUM Support
    • 8.56 – PT Patching will support Continuous Delivery
    • 8.56 Release date?
    • Refresh Script strategies
    • PS_CUST_APP_HOME Answers
    • Data Masking
  • PeopleTools 8.55 Certifications @ 49:00
  • Custom Type and Resources for Puppet? @ 53:00
    • pt_deploy and pt_config
  • SUSE Linux Support @ 55:00
  • 8.55 PT Upgrade Panel @ 57:00

File Upload Listening Connector

Javier Delgado has a nice overview of building a custom File Upload Listening Connector for the Integration Broker:

One of my customers recently had the need of allowing a third party web application to attach files into PeopleSoft. After trying a number of different approaches (the integration had to be done at the web application client level, which significantly reduces the options to manipulate the request to PeopleSoft before sending it, particularly when dealing with old web browsers), I gave up and came to the conclusion that I needed a custom listening connector in Integration Broker to implement such integration.

Limit PeopleSoft Vulnerabilities

Last week a presentation at Hack in the Box, “Oracle PeopleSoft Applications are Under Attack”, focused on vulnerabilities in PeopleSoft applications. The presentation showed a number of ways that hackers could get access to a system. While some of the issues need to be fixed by Oracle (like poor encryption), there are a number of steps you can take to reduce the risk of someone gaining unauthorized access to your system.

For reference, here is the presentation and here is the white paper.

Database Security

The first part of the presentation talks about back-end attacks (gaining access to the database). For someone who is on the internal network, there are a few ways to grab the Access ID.

Access ID Attacks

First, the Connect ID and password are often left to the default people and peop1e. The Connect ID has access to the PSACCESSPRFL table (by design), but the values in PSACCESSPRFL are not securely encrypted. According to the presentation, PeopleSoft uses a simple XOR operation to encrypt the values. XOR encryption is fairly simple to decrypt compared with newer encryption technologies. Update 5/11/2016 I have been informed that this is incorrect. XOR is not used for the Access ID and Password. The encrypt is 3des-168-cfb, which means it requires a 24 byte key, and a 64bit Initialization Vector, which is far harder to decrypt. End update The presentation also mentions that people often choose a Connect ID password that is the same as the Access ID. DO NOT DO THAT! Passwords should be unique and complex (as possible) so that one account doesn’t compromise other accounts.

This attack vector does bring up a concern that Oracle should address in a future PeopleTools release. Starting with 8.53, PSORPDEFN added a OPERPWSDSALT field that increased the complexity of password encryption. This functionality should be extended to the Access ID Password. Additionally, the limitation of an 8 character password for the Access ID password needs to be fixed.

Second, if someone has access to a developer machine there a two ways to grab the Access ID and password. One is to grab the configuration from the Windows Registry and decrypt the password from there. The encryption in the config file appears to be XOR based. The other method is to connect App Designer via 3-Tier connection and grab the network packets because the Tuxedo protocol is a plain text protocol. When developer tools communicate with the application server, their User ID and password are sent with each request. Plain text SQL is also sent via Tuxedo, and also executed under the Access ID, which is another potential issue.

In reality, the second attack requires far more access that most users have because it requires a 3-Tier connection along with App Designer access. It would be good for Oracle to start updating the Tuxedo protocol, but addressing the issues in the first attack would be the top priority. Another way to grab the Connect ID, is to open the psappsrv.cfg or psprcs.cfg files used by the Tuxedo domains. If a user has file server access to the process scheduler (very common for developers), they often times have read access to the process scheduler domain directory. If an administrator does not encrypt the Connect ID password when building the domain, anyone with read access can find the password.

Locking Down Database Access

  • Stop using default passwords
    Change the Connect ID password for your environment. You can even change the Connect ID itself, but changing the password from peop1e is the most important step.
  • Make the Access ID password unique
    While we can only have an 8 character password, make the password unique and complex. Don’t use dictionary words to make it memorable. Passwords need to be secure and complexity helps make them more secure.
  • Encrypt all passwords in config files
    When creating Tuxedo domains (application servers and process schedulers), make sure to encrypt any passwords in the file. The psadmin utility can do this for you when you configure the domain.

Web Security

The second part of the presentation focused on front-end attacks, or attacks through the web server. Front-end attacks are targeted at WebLogic, the PIA and other servlets delivered by PeopleSoft.

WebLogic Security

Attacks When deploying a Single Server PeopleSoft site, the WebLogic console is accessible on the same server & port as the PIA. Using http://server:port/console, anyone can access the WebLogic console login page. The default user and password for the console is system and Passw0rd in newer PeopleTools versions. If you do not change the password, a quick Google search can let anyone into the WebLogic console.

If you create a multi-server site, the Admin server will run on a separate port (or even server). You should still change the default passwords, but external hackers will have to discover the admin server and port before they can attempt an attack. There are two more accounts in WebLogic that can also expose a vulnerability: monitor and operator. These accounts are used for monitoring your WebLogic instance, but they offer enough access to the console that a hacker could install a .war application to run.

Locking Down WebLogic Access

  • Use a complex system password
    When creating a new PIA site, use a complex password for the system account. This account has full access to WebLogic (and subsequently the server file system) so do not use the default password.
  • Disable monitor and operator accounts
    If you are not using these accounts for monitoring, it is best to disable them. No need to have active accounts if they are unused. If you are using the accounts, make sure you change their password. Give these accounts unique passwords too; you don’t want to make them the same as the system account either.
  • Restrict access to the WebLogic console
    This post has an overview on how to limit access to the WebLogic console. If you are using a load balancer or RPS for external access (which is a good idea), you can block the IP address for those devices from the console. To access the console, you can VPN into your network and access the console from inside the network.

Servlet Attacks

Next in the list of web-based attacks are the servlets installed on the web server. There are a number of servlets installed on the web server like PSIGW, xmllink, etc. If you open up your web server directory structure (or through the WebLogic console) you can see the default servlets.

Some servlets handle XML processing (like PSIGW) and are susceptible to XXE (XML External Entity) attacks. XXE vulnerabilities allow for external code to be run on your web server.

Another attack vector (via XXE) is to read config files for passwords stored on your web server, like the integrationGateway.properties file for the PSIGW servlet. Passwords in this file can be encrypted, but the encryption uses the default 3DES key. You can tell this by looking at the beginning of the encrypted password string and seeing {V1.1}.

The last servlet attack mentioned is against the Integration Broker directly. The IB supports remote configuration (e.g, editing the integrationGateway.properties file via the PIA) and with a specially crafted URL an external system could read or overwrite the IB config file. Many organizations don’t change the default password (administrator and password) for the integrationGateway.properties file. This leaves the system vulnerable to an attack.

Locking Down Servlet Security

  • Patch WebLogic and PeopleTools
    The XXE vulnerability was fixed in WebLogic 10.3.6.0.5 and also in these version of PeopleTools:

    • 8.51.24
    • 8.52.17
    • 8.53.05
  • Generate a new PSCipher key
    Before you encrypt values in the web server config files, run this command on the web server to generate a new key: pscipher -buildkey When you encrypt values on the web server, you’ll see {V1.2} at the beginning of the string. That will tell you that a new (and non-default) key was used to encrypt the password.
  • Give service accounts unique and complex passwords
    For accounts that you assign different parts of the application, like the Node Default User, or SES Callback User, give those accounts unique and complex passwords. (Have you picked up on a theme yet?)
  • Change the password IB Administrator password
    You can change the IB Administrator password by selecting the “Change Password” box under under the Gateway Setup Properties link on the IB Gateways page. You an also edit the gatewayUserProfile.xml file on the web server.

Single Signon Attacks

The last topic of the presentation focused on the Single Signon feature between PeopleSoft systems. PeopleSoft SSO is handled by the PS_TOKEN browser cookie. If a browser session contains a valid PS_TOKEN, and the issuing system is trusted by other PeopleSoft applications, users are considered authenticated and will have access to the application. The PS_TOKEN is generated using these values:

  • User ID
  • Language Code
  • Node Name
  • Date and Time
  • Signature (SHA1 Hash of User ID + Language Code + Node Name + Date and Time + Node Password)

By default, PeopleSoft uses standard node names for HR (PSFT_HR), FS (PSFT_EP) and the other applications. Since all but the Node Password are potentially known ahead of time, the SHA1 Hash can be calculated and compared against a PS_TOKEN to identify the Node Password.

While you could remove the trust from any nodes (PSTRUSTNODES table), that would break many features in the application (like the Report Repository and Single Signon). Removing node trust isn’t a great option.

Another way to get a PS_TOKEN is to open a page where the system automatically logs in as a public user. For example, a landing page or the PeopleSoft “Forgot My Password” page require the Web Profile to have a public user configured. If the public user has lots of security, any knowledge user can navigate into pages inside the application using the PS_TOKEN.

Locking Down SSO Access

  • Use a complex Node password
    Use a complex node password for your Default Local Node, and the other trusted nodes. The longer the password, the more time it will take to calculate the SHA1 hash.
  • Use unique node names for all environments
    Use a different node name for each application across all the environments (Development, Test, etc). For example, use HR92PRD for HR Production, HR92TST for HR Test, etc. Also, make your node passwords different between Dev, Test and Production. This will prevent someone from logging in to your Dev environment and using that valid PS_TOKEN to authenticate into Production.
  • Limit Public user security
    For the Public User on any Web Profiles, ensure they only have access to the fewest Components/WEBLIBS/etc needed to function. A PS_TOKEN generated by the user with limited security will prevent users from gaining unauthorized access to other parts of the application.
  • Implement Node Certificate Security
    Select Certificate for the node authentication method instead of Password. You will need to generate and load certificates for the Gateway URL for each node.
  • Use external authentication
    PeopleSoft can plug into different authentication methods, like LDAP and Kerberos. If you are using an external system for user authentication, you can reduce some of the risk with the PS_TOKEN issue by removing other applications’ nodes from the PSTRUSTNODES table.

GreyHeller has an excellent blog post with more details on the PS_TOKEN vulnerability and they show how their own tools can help with this security risk.

Recommendations for Oracle

With proper configuration, many of the vulnerabilities in this presentation can be properly mitigated. The main theme in the presentation, and in this article, is to not use default, shared or simple passwords.

There are some issues identified in the presentation that are out of our control. Oracle will need to make the changes in PeopleTools to address some of these vulnerabilities. To help bring visibility to the issues, filing a feature request SR with Oracle will show that there is user interest in making the product more secure. Here are three suggested SR’s to file:

  • Access ID Password needs to support complex passwords (more than 8 characters)
    The Access ID has an 8 character limit. To ensure the security of the database and its information, this password should only be limited by the RDBMS platform.
  • PSACCESSPRFL encryption support
    Like PSOPRDEFN, PSACCESSPRFL should support additional/newer encryption options (like using a SALT).
  • PS_TOKEN should generate a more secure hash
    The SHA1 hash used to create the PS_TOKEN signature should be updated to use a newer hashing algorithm, or find a more secure alternative. Utilizing the JSSE libraries on the web server would allow for stronger hashes to be built (albeit with some performance impacts).