You are here: Home / oracle / WebLogic 12c: Node Manager Best Practices

WebLogic 12c: Node Manager Best Practices

January 31, 2012 by 24 Comments

During the last couple of years (and the last couple of WebLogic versions) I collected a number of best practices  regarding WebLogic nodemanager. All of them hold true for WebLogic 12c as well. This posting is not a step-by-step beginners guide and it will not save you from attending some training or studying the Oracle documentation regarding node manager yourself. Anyway, here are some suggestions, check if the apply for your environments:

Node Manager Best Practices

 

  • At first, take a decision to start servers with or without NM. Note, that  is not absolutely necessary. You can always start your servers with the scripts generated by the config wizzard. I personally know rather big companies building lovely cars who took the decsicion not to use node manager.
  • Would I use nodemanger myself? For an “average” project: yes! Only after configuring node manager you can use the WebLogic admin console to start and stop managed servers and node manager will restart you failed servers as well. However, if you consider restarting you servers automatically because of out-of-memory problems, better read this article about “surviving generations”  to understand how to track down memory leaks and fix them. Anyway, you still want to use node manager.
  • Make sure you understand that nodemanger will use default values to start your servers unless you specify them yourself in the admin console under server startup parameters.
  • Make sure you always start your servers with same startup parameters! This is really important. You end up in deep trouble if you don’t. Believe me.
    Imagine somebody is starting a managed server using the admin console and the provided values there. Next day somebody else starts a server using the provided scripts (which – at least in real life – will never be identical to the startup values configured in the admin console). Now depending on the way the server was started it will behave differently and show erratic behaviour or not.
  • Document and communicate the usage of node manager. Write it down in the operations manual. If you ever hire me as a consultant for some performance tuning it helps to know if you are actually using node manager or not.
  • Don’t forget to enroll new machines for NM usingnmEnroll()
  • A good way to overcome the potential problem with  startup parameters configured in admin console is to use: 
    [file: nodemanger.properties]startScriptEnabled=true
stopScriptEnabled=true

    Then node manager will use the generated start script and you do not need to configure startup values in the admin server console.

  • If you are not using SSL for your domain the default option for node manager to use encrypted communication does not make that much sense for you. Disable it. On the admin server site switcht to “plain text” for node manager communication and in the node manager.properties located in WL_HOME/common/nodemanager set 
    secureListener=false
  • If you decide to use SSL for the node manager communication, get correct certificates! The demo certificate will not work in a distributed system. Make sure the hostnames in the certificates are correct. If they are not correct, you may want to consider disabling host name verification on the admin server (which is the client for the node manager).
  • Remember that node manager is not part of the domain. Still you can check the node manager status and and see the logs directly from the WebLogic admin console.

Some homework for Oracle 😉 Here is my personal wishlist for WebLogic Server 13f:

  1. Enable plain text communication for node manager as default. Why should it be SSL?
  2. Set start/stopScriptEnabled as default. This will cause less confusion.

Any settings you would like to share? Something to argue? Let me know.

 

Filed Under: oracle Tagged With: , , , , ,

Comments

  1. Jan van Zoggel says
    January 31, 2012 at 16:21

    Hi Frank, like your blog but I think you are not correct regarding the remark about the new node manager status feature. It has been around since BEA Weblogic 10 (I’m looking at it now) and I think it has been available since at least release 8.1 (if I remember correct).

    I agree with the SSL/demo default from Weblogic. By creating this as default many production environments are out there still running with demotrusts which is very sad indeed. It’s a bit of a comprimise I guess to enable a domain config from start which uses SSL and just keeps triggering administrators to get decent certificates and keep spamming their weblogic logging on startup as a reminder. 🙂

    Reply
    • Prithviraj says
      December 29, 2015 at 20:55

      Hello guys ,
      I need some help in nodemanager setup.

      I am using weblogic 12.1.3. i have configured nodemanager so that i can start and stop managed servers from weblogic console.
      i can successfully start the managed server from weblogic console by using NodeManager.
      When i try to start the Managed server from console nodemanager invokes the start up script which i have configured and it get started properly.

      BUt whiole stopping any managed server from console , the configured stop script is not get invoked and my managed server proccess get killed.
      i want to invoke my stopscript instead of managed server getting killed .

      My node manager properties file is as below

      DomainsFile=/product/test/gmpp/ISW/user_projects/domains/GMPP/nodemanager/nodemanager.domains
      LogLimit=0
      PropertiesVersion=12.1.3
      AuthenticationEnabled=true
      NodeManagerHome=/product/test/gmpp/ISW/user_projects/domains/GMPP/nodemanager
      JavaHome=/opt/64bit/jdk1.8.0_60
      LogLevel=INFO
      DomainsFileEnabled=true
      StartScriptName=/product/test/gmpp/ISW/user_projects/domains/GMPP/serverScripts/startManagedServers.sh
      StopScriptName=/product/test/gmpp/ISW/user_projects/domains/GMPP/serverScripts/stopManagedServers.sh
      ListenAddress=sinrndvud252
      NativeVersionEnabled=true
      ListenPort=5550
      LogToStderr=true
      SecureListener=false
      LogCount=1
      StopScriptEnabled=true
      QuitEnabled=false
      LogAppend=true
      StateCheckInterval=500
      CrashRecoveryEnabled=false
      StartScriptEnabled=true
      LogFile=/product/test/gmpp/ISW/user_projects/domains/GMPP/nodemanager/nodemanager.log
      LogFormatter=weblogic.nodemanager.server.LogFormatter
      ListenBacklog=50

      Can you please help me in this ?

      Reply
  2. frank says
    January 31, 2012 at 17:36

    Jan,

    many thanks for your feedback! Of course you are right, being able to see NM status and logs from admin server is not at all new. My intention was to mention that you can do it since I personally missed this feature for a long time. I changed the initial posting to make it less confusing.

    cheers,

    Frank

    p.s. Just realized you participated in the OSB cookboock. Congrats! Time for me to get the final version of the prelim. ebook that I bought.

    Reply
  3. Jan van Zoggel says
    February 1, 2012 at 09:55

    Your welcome and I enjoy your blog so keep those posts coming 😉
    And thanks for the congrats, I hope you like the book.

    cheers,

    Jan

    Reply
  4. Radu Dobrinescu says
    February 2, 2012 at 19:31

    Hi Frank,

    Thanks for the great post. I would also add something to the wishlist for future versions: Node Manager credentials not to be randomly generated by the Configuration Wizard when a production mode domain is created. This can cause issues when using NM in WLST:

    http://oraclemiddleware.wordpress.com/2012/01/15/cannot-connect-to-node-manager-access-to-domain-for-user-weblogic-denied/

    Regards,
    Radu

    Reply
  5. Edwin Biemond says
    February 25, 2012 at 17:05

    Hi,

    the ssl on the nodemanager works by default ( no need for any change ) , you only need to disable the hostname verification on all the weblogic servers ( go to the server , ssl -> advanced ).

    Enroll is also not always necessary , restart after a new domain is enough ( I think it has something with the locations of the wls domains )

    it is always a good plan to start first the managed from the script and after that from the nodemanager.

    Regards

    Reply
  6. Jacco H. Landlust says
    July 10, 2012 at 08:54

    Hi Frank,

    Some more feedback 🙂 I think I know some stuff about nodemanager which I like to share:

    I think you have mistaken the usage of stopScriptEnabled. Bij default there is no stopscript and it has little to do with the actual stopping of the managed server. The stopScript is called *after* the managed server is stopped from the administration server.

    Personally I really dislike disabling hostname verification like Edwin suggests. Either run SSL properly (with hostname verification) or don’t. Disabling hostname verification sort of beats the whole idea of using SSL for me.

    Next up I think Oracle should encourage to run SSL on NodeManager and managed servers and (more importantly) remove the default certificates so one is forced to install custom ones. Like Jan said: there are loads of domains running production on default certificates. That is a bad bad bad practices, since everyone has those default certs.

    Radu suggested that the username for NodeManager should not be created randomly. This is *only* the case when you create a domain from config.sh and in production mode. Jan and I discussed this and Jan posted the results here: http://jvzoggel.wordpress.com/2011/08/19/oracle-weblogic-nodemanager-bea-300033/

    Commandline parameters passed to the start nodemanager command override settings in the nodemanager.properties file.

    Finally I have some extra “best practices” from my own usage:
    – do not use the script based nodemanager, always the java based (don’t know if the script based is still there in 12.1)
    – do not run the nodemanager as root (and therefore do not use the post bind uid/guid features)
    – I like to have different users for the software and the domain. So software owner by oracle, the domain (+ jvm) an other user (e.g. weblogic).
    – One user per domain is even better (if you consolidate multiple domains on one server), that does force you to use one nodemanager per domain (instead of one per user).
    – do not store your nodemanagers configuration and logging (the nodemanager home) at the default location, but move them elsewhere (e.g. with the domain on a shared device, etc)
    – In my opinion the username for nodemanager should not be weblogic, but some other name. This makes brute forces attacks harder.
    – Backup SerializedSystemIni.dat (nm_data.properties for 11g) and nm_password.properties
    – use keyfiles instead of username/password for starting the nodemanager from a script (weblogic.Admin STOREUSERCONFIG) and protect these keyfiles with proper file privileges
    – Whenever you use nodemanager to manage your domain, set CrashRecoveryEnabled to true. If you use a shared device (e.g. NFS) for you domain, please set DomainsDirRemoteSharingEnabled to true too (parameter valid from 10.3.4 and newer, see http://oraclemva.wordpress.com/2011/02/16/new-parameter-in-nodemanager-properties/)
    – Start your adminserver from nodemanager too (to have it protected by nm + crashrecoveryenabled). Check http://oraclemva.wordpress.com/2011/02/07/wls-nodemanager-and-startup-properties/ to find out how without having to start the adminserver from a script first.
    – do not start managed servers from nodemanager if you have used rlwrap with wlst (see http://oraclemva.wordpress.com/2011/04/04/rlwrap-wlst-and-the-nodemanager/)

    Well… that sort of covers nodemanager 🙂

    Cheers,

    Jacco

    Reply
    • frank says
      July 25, 2012 at 12:58

      Hi Jacco,

      many thanks for your detailed and valuable comments.
      Some of your items I already had in the NM recipe that will be published some day, but there are also quite some new items that I didn’t think of, e.g. DomainsDirRemoteSharingEnabled or rlwrap etc.
      Thanks for sharing your knowledge and experiences and for making this posting more helpful for everyone!

      best,

      Frank

      Reply
  7. Anirban Ghosh says
    September 17, 2012 at 22:56

    Hi Frank,

    Thanks for the informative post. Here’s one more to add to the wishlist: There should be a provision in the Admin Console to associate a NM credential with a machine. Right now the Admin Server uses the same NM credential for all the machines in the domain. Which indirectly makes the NMs somewhat dependent on the domain.
    Since it is possible to have multiple domains on a machine connecting to the same NM process using different credentials, I think it is only logical that a domain should be able to connect to NM processes on multiple machines using different credentials.

    Regards,
    Ani

    Reply
  8. Ken says
    August 6, 2013 at 16:17

    Thank you for the great information, some of it was new to me and also really helpful.

    As for something one of the commenters said, I’m hoping somebody can shed some light for me. I know this post is old but I hope someone will respond.

    Jacco said:
    – do not run the nodemanager as root (and therefore do not use the post bind uid/guid features)

    If one does not use the post bind feature, how can you have your managed server listen on a privileged port like 80 or 443 in Linux? Is there something I am missing? Why is running NM as root then using post bind a problem?

    Thanks,

    Reply
    • frank says
      August 8, 2013 at 10:28

      Ken,

      running NM as root can be a potential security risk. There is loads of security with NM, such as credentials, SSL, nodemanager.domain etc, but still NM is a process accepting commands from remote and then starting / stopping processes on the machine.

      Using managed servers with ports less than 1024 is independent from that. Actually it is not that common. Typically your load balancer can listen to 80 or 443 then forward to a port in the range of 7000 or so. Even the default port of a WLS server is 7001.

      thanks for your comment,
      keep posting!

      Frank

      Reply
  9. subhani says
    February 2, 2014 at 05:49

    What are default memory arguments of weblogic Nodemanager?

    Reply
  10. hari says
    September 3, 2014 at 17:26

    Hi,
    I have a query. I have 2 machines – mac1 & mac2. Both machine has weblogic 10.3.6 installed. Now, is there some way to Enroll nodemanager on both machines only from mac1??
    I know nmEnroll is to be done individually on both machines.

    Any suggestion.

    Thanks,
    Hari

    Reply
  11. Andrew says
    February 5, 2015 at 17:49

    We have been using node manager to start managed servers under 11g for a while and we use the startScriptsEnable=true setting. However we are now upgrading to 12c and we’re finding that on restart node manager is starting the servers in admin mode rather then in the running mode. Is there a new property that controls this?

    Thanks for any help,
    Andrew

    Reply
    • frank says
      February 23, 2015 at 12:39

      Andrew,

      sorry for the late reply. Do you still see this issue?
      I am not aware of a new property rgd what you describe.

      Are you restarting from admin server or rebooting machines?
      Also are you using the new per domain NM?

      kind regards,

      Frank

      Reply
  12. Kumar says
    March 27, 2015 at 13:23

    Hello
    If your domain home is on shared location (nas) and then y ou mount this nas on multiple boxes, in this case, should node manager configuration be local to one box or on shared location.
    From my understanding, I put this on shared location. But wanted to confirm.

    Thank you
    Kumar

    Reply
  13. Mrcin says
    October 2, 2015 at 11:39

    Hi All,

    I have a question: Why Admin Server is located on a separate machine? Most of WLS deployments I’ve seen so far are having Admin Server running on a primary node of a cluster, having configured VIP IPs to enable admin server to be migrated to the secondary node..

    Thanks in advance for hints…
    Marcin

    Reply
  14. N Kalyan L says
    January 29, 2016 at 13:29

    Hello Sir,
    I am using wls 11g in RHEL 5.8 Env.
    i have starting Admin Server , Node Manager & Unable to Start the Managed Server from my Admin Console.
    My Node manager Status was Reachable in Console.
    Previously i was able to start from console , it self ..Now i can’t.
    I am getting error like
    BEA–300048 : unable to start managed server : exception while starting Managed Server from console.
    – Weblogic.Nodemanager.server.handler handle start warning ;
    exception to start server
    java.io.IOException :: Exception while starting server .
    please provide me solution & replly to “nlkalyan.434@gmail.com”

    Reply
  15. Gaurav says
    June 15, 2016 at 00:59

    Hi,
    I have a question. Which configuration in Weblogic will be better for performance. More domains with less number of managed servers in each domain or less number of domains with more number of managed servers in each domain.

    I am using Weblogic 12c and have a requirement to host many applications which can run independently on the same machine.

    You can mail me at tripathigaurav23@gmail.com

    Regards,
    Gaurav

    Reply
    • frank says
      June 20, 2016 at 11:01

      Hi Gaurav,

      the answer is it depends.

      Did you read the chapters about performance and the ones about domains e.g. in Oracle doc (if you like) or my book?
      What is your definition of performance? what you want to optimize for?
      If you follow a proper tuning process, can you report your initial testing results?
      How much performance would be worth an additional administraion overhead?

      please consider these questions,

      F.

      Reply
  16. Rohit says
    November 10, 2017 at 11:38

    How to start nodemanager

    Reply

Trackbacks

  1. localhost is mostly wrong (btw that’s why it’s called Distributed System) — munz & more says:
    February 5, 2012 at 21:43

    […] listen address (for all manged servers and the admin server as well, ah and don’t forget the node manager either). Unlike your private Mac or PC,  multi-project, multi-domain machines may have dozens of […]

    Reply

Speak Your Mind

*