In my humble opinion, weblogic is full of bugs :) So let's see some common of them.
if you have missing identity file when you try to start the NodeManager as follows:
Trying to start Nodemanager
[oracle@oud bin]$ ./startNodeManager.sh
CLASSPATH=/home/oracle/jdk1.8.0_301/lib/tools.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/server/lib/weblogic.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/modules/thirdparty/ant-contrib-1.0b3.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/modules/features/oracle.wls.common.nodemanager.jar::/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/..:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/modules/features/oracle.wls.common.grizzly.jar
+ /home/oracle/jdk1.8.0_301/bin/java -server -Xms32m -Xmx200m -Djdk.tls.ephemeralDHKeySize=2048 -Dweblogic.RootDirectory=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/common/nodemanager -Dcoherence.home=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../coherence -Dbea.home=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/.. -Djava.system.class.loader=com.oracle.classloader.weblogic.LaunchClassLoader -Djava.security.policy=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.JavaHome=/home/oracle/jdk1.8.0_301 weblogic.NodeManager -v
<Nov 7, 2021 2:50:50 PM UTC> <INFO> <Loading domains file: /opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.domains>
<Nov 7, 2021 2:50:50 PM UTC> <WARNING> <Domains file not found: /opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.domains>
<Nov 7, 2021 2:50:51 PM UTC> <INFO> <Loading identity key store: FileName=/opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/security/DemoIdentity.jks, Type=jks, PassPhraseUsed=true>
<Nov 7, 2021 2:50:51 PM UTC> <SEVERE> <Fatal error in NodeManager server>
weblogic.nodemanager.common.ConfigException: Identity key store file not found: /opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/security/DemoIdentity.jks
at weblogic.nodemanager.server.SSLConfig.loadKeyStoreConfig(SSLConfig.java:225)
at weblogic.nodemanager.server.SSLConfig.access$000(SSLConfig.java:33)
at weblogic.nodemanager.server.SSLConfig$1.run(SSLConfig.java:118)
at java.security.AccessController.doPrivileged(Native Method)
at weblogic.nodemanager.server.SSLConfig.<init>(SSLConfig.java:115)
at weblogic.nodemanager.server.NMServer.<init>(NMServer.java:169)
at weblogic.nodemanager.server.NMServer.getInstance(NMServer.java:134)
at weblogic.nodemanager.server.NMServer.main(NMServer.java:589)
at weblogic.NodeManager.main(NodeManager.java:31)
Then we have to generate again the SSL certificate and the identity file as follows:
1. Export correct variables:
Export Variables:
[oracle@oud bin]$ . setWLSEnv.sh CLASSPATH=/home/oracle/jdk1.8.0_301/lib/tools.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/modules/features/cieCfg_cam_lib.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/modules/features/wlst.wls.classpath.jar: PATH=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/server/bin:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/modules/thirdparty/org.apache.ant/1.10.5.0.0/apache-ant-1.10.5/bin:/home/oracle/jdk1.8.0_301/jre/bin:/home/oracle/jdk1.8.0_301/bin:/home/oracle/jdk1.8.0_301/bin:/bin:/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/modules/org.apache.maven_3.2.5/bin Your environment has been set. [oracle@oud bin]$
2. We can generate the SSL and the identity file as follows:
Re-generate the files
[oracle@oud security]$ pwd /opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/security [oracle@oud security]$ java utils.CertGen -keyfilepass DemoIdentityPassPhrase -certfile democert -keyfile demokey -strength 2048 -noskid Generating a certificate with common name oud and key strength 2048 issued by CA with certificate from /opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/server/lib/CertGenCA.der file and key from /opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/server/lib/CertGenCAKey.der file [oracle@oud security]$ [oracle@oud security]$ java utils.ImportPrivateKey -keystore DemoIdentity.jks -storepass DemoIdentityKeyStorePassPhrase -keyfile demokey.pem -keyfilepass DemoIdentityPassPhrase -certfile democert.pem -alias demoidentity No password was specified for the key entry Key file password will be used <Nov 7, 2021 3:38:33 PM UTC> <Info> <Security> <BEA-090905> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.> <Nov 7, 2021 3:38:33 PM UTC> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG128 to HMACDRBG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true.> Imported private key demokey.pem and certificate democert.pem into a new keystore DemoIdentity.jks of type jks under alias demoidentity [oracle@oud security]$ [oracle@oud security]$
3. We can check if everything is created:
Verify all has been created
[oracle@oud security]$ ls -lart total 24 -rw-r-----. 1 oracle oinstall 80 Nov 7 14:25 SerializedSystemIni.dat drwxr-x---. 3 oracle oinstall 45 Nov 7 15:37 .. -rw-r--r--. 1 oracle oinstall 1260 Nov 7 15:38 demokey.der -rw-r--r--. 1 oracle oinstall 1781 Nov 7 15:38 demokey.pem -rw-r--r--. 1 oracle oinstall 923 Nov 7 15:38 democert.der -rw-r--r--. 1 oracle oinstall 1306 Nov 7 15:38 democert.pem -rw-r--r--. 1 oracle oinstall 2282 Nov 7 15:38 DemoIdentity.jks drwxr-x---. 2 oracle oinstall 139 Nov 7 15:38 . [oracle@oud security]$
4. After that, we can start the node manager normally as follows:
Starting Node manager
[oracle@oud bin]$ nohup ./startNodeManager.sh > nm.out& [1] 6328 [oracle@oud bin]$ nohup: ignoring input and redirecting stderr to stdout [oracle@oud bin]$ [oracle@oud bin]$ [oracle@oud bin]$ tail -100f nm.out CLASSPATH=/home/oracle/jdk1.8.0_301/lib/tools.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/server/lib/weblogic.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/modules/thirdparty/ant-contrib-1.0b3.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/modules/features/oracle.wls.common.nodemanager.jar:/home/oracle/jdk1.8.0_301/lib/tools.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/modules/features/cieCfg_cam_lib.jar:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/modules/features/wlst.wls.classpath.jar::/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/..:/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/modules/features/oracle.wls.common.grizzly.jar + /home/oracle/jdk1.8.0_301/bin/java -server -Xms32m -Xmx200m -Djdk.tls.ephemeralDHKeySize=2048 -Dweblogic.RootDirectory=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../oracle_common/common/nodemanager -Dcoherence.home=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/../coherence -Dbea.home=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/.. -Djava.system.class.loader=com.oracle.classloader.weblogic.LaunchClassLoader -Djava.security.policy=/opt/oracle/Oracle/Middleware/Oracle_Home/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.JavaHome=/home/oracle/jdk1.8.0_301 weblogic.NodeManager -v <Nov 7, 2021 3:43:42 PM UTC> <INFO> <Loading domains file: /opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.domains> <Nov 7, 2021 3:43:42 PM UTC> <WARNING> <Domains file not found: /opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.domains> <Nov 7, 2021 3:43:43 PM UTC> <INFO> <Loading identity key store: FileName=/opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/security/DemoIdentity.jks, Type=jks, PassPhraseUsed=true> <Nov 7, 2021 3:43:43 PM UTC> <WARNING> <NodeManager configuration properties file '/opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.properties' not found. Using default settings.> <Nov 7, 2021 3:43:43 PM UTC> <INFO> <Saving NodeManager configuration properties to '/opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.properties'> Node manager v12.2.1.4.0 Configuration settings: DomainsFile=/opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.domains LogLimit=0 DomainsDirRemoteSharingEnabled=false AuthenticationEnabled=true LogLevel=INFO DomainsFileEnabled=true ListenAddress= NativeVersionEnabled=true ProcessDestroyTimeout=20000 ListenPort=5556 LogToStderr=true weblogic.StartScriptName=startWebLogic.sh SecureListener=true LogCount=1 LogAppend=true weblogic.StopScriptEnabled=false StateCheckInterval=500 CrashRecoveryEnabled=false weblogic.StartScriptEnabled=true LogFile=/opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager/nodemanager.log LogFormatter=weblogic.nodemanager.server.LogFormatter coherence.StartScriptEnabled=false ListenBacklog=50 NodeManagerHome=/opt/oracle/Oracle/Middleware/Oracle_Home/oracle_common/common/nodemanager RestEnabled=false isLogLockingEnabled=true weblogic.startup.JavaHome=/home/oracle/jdk1.8.0_301 weblogic.startup.MW_Home= coherence.startup.JavaHome=/home/oracle/jdk1.8.0_301 coherence.startup.MW_Home= Domain name mappings: <Nov 7, 2021 3:43:43 PM UTC> <INFO> <12.2.1.4.0> <Nov 7, 2021 3:43:43 PM UTC> <INFO> <Server Implementation Class: weblogic.nodemanager.server.NMServer$ClassicServer.> <Nov 7, 2021 3:43:44 PM UTC> <INFO> <Secure socket listener started on port 5556>