Hi,
We have solarwinds HA Environment and looking to have an additional polling engine,
Could somebody please advise what would be the best way, it can be done.
if there is a documentation we can follow to install it, will be greatly appreciated.
Thanks
Is it possible to multihome the SW agent. Basically have the agent (windows server) communicate to separate SW environments (not HA or APE's).
Thanks
Hi,
I'm trying to set up reporting for cluster DSR, I can't seem to find anything to configure reporting for this? Any help or guidance would be appreciated! I did find a paper but I can't access it.
Hello,
I have a service running that is being monitored by SolarWinds and every night it goes down so backups can take place. I want to mute the alerts on the service but not the node, and I don't want to amend the alert because it's used for several different services. Is this possible or will I need to just mute the node?
Many thanks.
Newbie here...
I've created a SQL-based Application Monitor Template. It has a Component Monitor of type SQL Server User Experience Monitor. Within the Component Monitor I have a SQL Query which returns a value. This value is evaluated in the Statistics Threshold and set to Warning at X and Critical at X+4000.
I'm lost at how to email alerts on these Statistic Threshold evaluations. Can someone point me in the right direction?
Thx.
When you're writing your powershell scripts, or trying to interrogate your devices and applications for data, what authentication do you need for those REST API endpoints? For example, Microsoft gives you guidance on how to protect your APIs using OAuth 2.0 here Protect an API by using OAuth 2.0 with Azure Active Directory and API Management | Microsoft Docs
Netscaler polling – Solarwinds API Edition
So, the new API polling has opened up a lot of doors. Everything has a REST API! Likewise, so do Netscalers. By definition I was determined to figure out how to make this work. It took a bit of effort and a bit of reference knowledge.
This was tested on SAM 2019.4 and Netscaler 14K’s and virtual appliance Netscalers.
Some information can be found at https://developer-docs.citrix.com/projects/netscaler-nitro-api/en/12.0/statistics/load-balancing/lbvserver/lbvserver/ - where they provide some examples of how to poll the data.
We have information available to us from solarwinds at https://support.solarwinds.com/SuccessCenter/s/article/SAM-2019-4-feature-API-Poller . https://documentation.solarwinds.com/en/Success_Center/sam/Content/SAM-API-Poller-Add.htm . This is the actual picture example similar to below, for reference. What I added is the special sauce for netscaler.
A valid HTTPS certificate on your load balancer’s web interface (if not, you’re using HTTP and well…yeah – that’s not good)
A login to the netscaler, both for you and the credential you intend to use (as far as I know, someone who knows netscaler probably knows how to get your service account more granular).
Names for the netscalers you want to poll.
Log into netscaler, go to Traffic Management -> Load Balancing -> Virtual Servers
Search and find the name of the load balancer you want. The name there is what you’re looking for. This is what you’re going to be pointing the API poller to.
Go to the Node in Orion and go to “create API”. You’ll notice that after this point looks like the solarwinds reference document a bit.
Go to configure and add the appropriate credential you have for the Netscalers to pull this information. That’s under configure in the top right after you go to “create” for the API poller. Also, whatever you name it in the credentials area (UX!!!!! *shakes fist*) is what will show up under the image above – so if you call it in the credential/config area “random API poller test” then under this dropdown of “API Poller” you’ll see “random API poller test” as the name.
For the query, you’re looking for https://{NETSCALER-FQDN}/nitro/v1/stat/lbvserver . See reference at the start of this document. So, Citrix has a “Nitro” API for when we’re polling data and not changing anything.
Replace {NETSCALER-FQDN} with the name of your netscaler. So if your netscaler is booboo1, then the link is https://booboo1/nitro/v1/stat/lbvserver to test seeing ALL of the load balancer groups.
As referenced in the citrix api:
get (all)
URL:
http://<netscaler-ip-address>/nitro/v1/stat/lbvserver
This is how you make sure it works, at a basic level. When it works, you get this when you send a query. Which should be loaded differently, but each number there starting from zero is incrementing and listing each of the load balancer groups you have.
Now, you need that netscaler name to make the rest work. You don’t have to get too granular. The API references a lot of options, but you can pick what you want out of the results anyway. In my case:
https://{NETSCALER-FQDN}/nitro/v1/stat/lbvserver?args=name:{VIRTUALSERVERNAME}Citrix Reference document says: args
http://<netscaler-ip-address>/nitro/v1/stat/lbvserver?args=name:<String_value>,detail:<Boolean_value>,fullvalues:<Boolean_value>,ntimes:<Double_value>,logfile:<String_value>,clearstats:<String_value>,sortby:<String_value>,sortorder:<String_value>,sortorder:<String_value>
Use this query-parameter to get lbvserver resources based on additional properties.
However, we don’t need to say I need value a/b/c because solarwinds already does this for you.
So, here's the metric in question from one of my netscalers. I monitor the VSLB health, which tells you when something has dropped (percentage value).
When this is done, you can then poll the Netscaler and pick what metric you want when you click on the little monitor icon on the right side next to the metric you want to obtain! Happy monitoring!
A few side-points here that are confusing:
Once you click the monitor icon, you have to name what you’re getting/displaying. So, I’d name one of mine VS_LB_CASMAPI.
As a result, you can put the load balancers in a group and the API pollers into a child like this:
Oh, and at a bonus: you get to put all these tasty metrics straight into perfstack. Just click on the results from the node itself. (This needs to be it's own page in the future, IMO).
Have you read all this? Are you exhausted? Should this be a SAM template? Maybe. Do we need more darktheme ? yes.
Let me know how this goes! Tell me what else you’ve found? What doesn’t work and does work? Does this work for you?
edit: added photo of VSLB health
You ask, we listen. Many of the top features being worked on in SAM are generated through your feedback, your participation in our user sessions and your votes in our Server & Application Monitor Feature Requestsforum.
We actively refine the product roadmap to solve your problems. Participate in user sessions for THWACK points and personalized input into the future of SAM.
Hello All,
Does anyone know of a way to discover a server mapping showing the communication for servers? Ultimately, we would want to select a node and see what other servers it's communicating with. We are trying to automate a process of using SW to help find what servers coincide with different applications. Some of our application platforms can consist of 20 servers, the challenge is how we would use SW to help with the discovery.
Thanks!
I cannot find the noc rotation view tab. Are there any suggestions please. Thank you.
Hello to the THWACK community,
I am currently having trouble upgrading from SAM 6.7.1 (Orion Platform 2018.4 HF3) to SAM 2019.4.1
The actual software install seems to go fine, but a problem arises during the configuration wizard.
I get to this point, and the wizard hangs and runs transactions continually on the DB until the transaction log fills up at 100GB.
Overall Progress: 72.2%
Configuring general components for plugins 98.8% - Configuring Cortex Integration General
It sits here for approximately 30 minutes, then will fail.
According to a DBA that looked in to the database during the 'hang' period:
"There is a common table expression running that is filling up the log"
I can see a repeating pattern in the configuration wizard log during this time:
2020-01-09 20:27:29,918 [147] DEBUG SqlHelper - SQL: IF EXISTS(SELECT * FROM [dbo].[sysobjects] WHERE id = OBJECT_ID(N'[dbo].[HA_PoolMembersView]') AND type in (N'V'))
AND EXISTS(SELECT * FROM [dbo].[sysobjects] WHERE id = OBJECT_ID(N'dbo.HA_PoolMembers') AND type in (N'U'))
BEGIN
SELECT PoolMemberId, PoolMemberType, PoolId, HostName, ElectionPriority, Priority, PreferredStatus, PreferredStatusTimestamp, PreferredStatusRevision, Status, StatusMessage, ReasonOfFail, ReasonOfFailRevision, HeartBeat, LastHeartBeatTimestamp, PoolIdRevision
FROM dbo.HA_PoolMembersView
WHERE PoolId<>0 AND (PoolMemberType='MainPoller' OR PoolMemberType='MainPollerStandby') ORDER BY HostName
END
2020-01-09 20:27:29,918 [147] DEBUG SqlHelper - SQL: SELECT TOP 1 ServerName FROM dbo.Engines WITH (NOLOCK) WHERE ServerType = 'Primary' ORDER BY KeepAlive DESC
2020-01-09 20:27:29,918 [147] DEBUG MessageBusTopologyProvider - No message bus host change detected.
2020-01-09 20:28:05,981 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:28:05,981 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:28:05,981 [151] DEBUG ScheduledTask - Running SolarWinds.Orion.LogMgmt.RuleProcessing.Rules.RuleDataSynchronizerLoadRetry
2020-01-09 20:28:05,981 [151] DEBUG ScheduledTask - SolarWinds.Orion.LogMgmt.RuleProcessing.Rules.RuleDataSynchronizerLoadRetry succeeded
2020-01-09 20:28:05,981 [Scheduler] DEBUG Scheduler - Sleeping for 1.00:00:00.0200000
2020-01-09 20:28:05,981 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:28:05,981 [Scheduler] DEBUG Scheduler - Sleeping for 00:01:00.0200000
2020-01-09 20:28:11,153 [156] DEBUG SqlHelper - SQL: SELECT CAST((CASE WHEN SERVERPROPERTY('edition') = 'SQL Azure' THEN 1 ELSE 0 END) AS INT)
2020-01-09 20:28:11,153 [156] INFO CwActiveInstanceChecker - Extending expiration for active instance of ConfigWizard on machine ITIS-SOLWIND1 (138.26.53.177)
2020-01-09 20:28:11,153 [156] DEBUG SqlHelper - SQL: SELECT CAST((CASE WHEN SERVERPROPERTY('edition') = 'SQL Azure' THEN 1 ELSE 0 END) AS INT)
2020-01-09 20:29:06,013 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:29:06,013 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:29:06,013 [Scheduler] DEBUG Scheduler - Sleeping for 1.00:00:00.0200000
2020-01-09 20:29:06,013 [168] DEBUG ScheduledTask - Running SolarWinds.Orion.LogMgmt.RuleProcessing.Rules.RuleDataSynchronizerLoadRetry
2020-01-09 20:29:06,013 [168] DEBUG ScheduledTask - SolarWinds.Orion.LogMgmt.RuleProcessing.Rules.RuleDataSynchronizerLoadRetry succeeded
2020-01-09 20:29:06,013 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:29:06,013 [Scheduler] DEBUG Scheduler - Sleeping for 00:01:00.0200000
2020-01-09 20:29:29,936 [157] DEBUG SqlHelper - SQL: IF EXISTS(SELECT * FROM [dbo].[sysobjects] WHERE id = OBJECT_ID(N'[dbo].[HA_PoolMembersView]') AND type in (N'V'))
AND EXISTS(SELECT * FROM [dbo].[sysobjects] WHERE id = OBJECT_ID(N'dbo.HA_PoolMembers') AND type in (N'U'))
BEGIN
SELECT PoolMemberId, PoolMemberType, PoolId, HostName, ElectionPriority, Priority, PreferredStatus, PreferredStatusTimestamp, PreferredStatusRevision, Status, StatusMessage, ReasonOfFail, ReasonOfFailRevision, HeartBeat, LastHeartBeatTimestamp, PoolIdRevision
FROM dbo.HA_PoolMembersView
WHERE PoolId<>0 AND (PoolMemberType='MainPoller' OR PoolMemberType='MainPollerStandby') ORDER BY HostName
END
2020-01-09 20:29:29,936 [157] DEBUG SqlHelper - SQL: SELECT TOP 1 ServerName FROM dbo.Engines WITH (NOLOCK) WHERE ServerType = 'Primary' ORDER BY KeepAlive DESC
2020-01-09 20:29:29,936 [157] DEBUG MessageBusTopologyProvider - No message bus host change detected.
2020-01-09 20:30:06,039 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:30:06,039 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:30:06,039 [173] DEBUG ScheduledTask - Running SolarWinds.Orion.LogMgmt.RuleProcessing.Rules.RuleDataSynchronizerLoadRetry
2020-01-09 20:30:06,039 [173] DEBUG ScheduledTask - SolarWinds.Orion.LogMgmt.RuleProcessing.Rules.RuleDataSynchronizerLoadRetry succeeded
2020-01-09 20:30:06,039 [Scheduler] DEBUG Scheduler - Sleeping for 1.00:00:00.0200000
2020-01-09 20:30:06,039 [Scheduler] DEBUG Scheduler - Checking if Scheduler should shutdown
2020-01-09 20:30:06,039 [Scheduler] DEBUG Scheduler - Sleeping for 00:01:00.0200000
This continues until the database transaction log fills up.
Before the upgrade, the database mdf was 34GB allocated with 22GB data inside. The log file was 45GB allocated with 1GB of data inside.
After the wizard fails. the DB is 67GB and the log is 96GB
I contacted support 4 weeks ago, and since then it has been a continual cycle of "I see X error in the log, try this", with the "try this" part taking several hours at the least, and leaving me where I started every time. The latest thing to try is to stand up a new server and install Orion there. This seems like it will be a painful process to me.
I definitely appreciate anyone taking the time to read through this. By posting here, I am hoping to get some help or advice that will lead me forward.
I have a few questions:
1. Is reinstalling Orion on a new server a common troubleshooting step?
2. If reinstalling on a new server is the best path forward, what can I expect to lose in the move? I am looking for any gotchas like data that is not stored in the databse. I don't *think* we have anything custom, but what should I look for?
I currently have the following in Orion:
~550 servers
~2000 volumes,
111 Application monitors (1331 component monitors)
20 universal device pollers
67 alerts
Hello All,
I'm trying to set up an alert action that reboots a service when its down. The issue is after settings things up, it doesn't restart the services. I've added a trigger action to send a email alongside the restart but I would only get the email.
I've set up the alert exactly how it's directed in this article: Success Center
I've attached how the trigger condition and actions look on my side.
Any assistance would be appreciated.
Also i
Greetings THWACK. Question to see how others are monitoring your instance of Service Now.
I was going to setup a HTTPS monitor in SAM to check to see if our instance of Service Now was up which is simple and easy to do. I was just wondering if anyone out in THWACK world has done anything different. If not no big deal and I will setup the HTTPS monitor to check for words on the web page.
Thanks!
Dave
When you're writing your powershell scripts, or trying to interrogate your devices and applications for data, what authentication do you need for those REST API endpoints? For example, Microsoft gives you guidance on how to protect your APIs using OAuth 2.0 here Protect an API by using OAuth 2.0 with Azure Active Directory and API Management | Microsoft Docs
Hi All,
I'm working on something that should be simple but can't seem to make it work.
Wanting to show only active alerts for a subset of servers in a resource (view tile) in a console.
Let's say I want to make a console showing information for the Domain Controllers in my environment. Domain Controllers all have '-DC-' in the host name (node caption).
In my Solarwinds system I've made a console for Domain Controllers. I have set up a View Limitation defined for System Name like '-DC-'
O.K. I add a Node List resource to the console and it only shows me Domain Controllers. Groovy.
Now I add the Active Alert resource to my console. But... it shows me every active alert in the entire system, not just those for DCs. And... the 'edit' functionality does not allow the addition of any kind of filter.
Same with the 'All Alerts' resource. Shows all alerts and seems to ignore view limitation on the console. Also no filtering capability that will help me.
Next idea. Custom Table resource. I choose the 'Active Alerts' data source. But what field to filter for hosts with '-DC-' in the name? "Node Caption" and "Related Node" caption to not return anything. The Active Alerts resource has a field called "Triggering Object" that looks like what I want but that does not seem to be present in the WHERE clause fields for Active Object. I think the field name is "ObjectName" in the SQL table.
So... kind of stuck.
Can anyone in the community offer some pointers on how to get Active Alerts filtered by host name pattern to show up in a console view? Thanks in advance for any assistance.
BTW Orion Platform 2019.2, NPM 12.5, SAM 6.9.0, running on Windows Server 2016 and SQL 2016.
Hi,
Recently, we have upgraded SolarWinds from 2019.2 to 2019.4.
We have observed that SolarWinds agents went in auto update state after upgrade even disabling auto update globally as well as disabled auto update at each agent level by querying in database.
Disabling agent auto update option does@ not works with SolarWinds.
I'm using the o365 templates from application monitors and our servers have a GPO with Basic Authentication set to false, anyone know another authentication method that can be used?
Hello Thwackers,
I thought I would share my rough and ready PowerShell script which helps when you have a known list of ICMP devices to import.
This is very useful if like me, you are going through an onboarding process from one system to another.
The script takes input from a CSV file and creates ICMP polled nodes from the data. It supports setting custom properties across all rows, and custom properties per row.
Just change the parameters between lines 8 and 35 with values that are appropriate for your installation. if you are using custom properties, add them at lines 94 and 158. Comments in the script will guide you.
It should also help as a springboard for further development with the API.
Enjoy!
I created a python script to validate an XML file on a web site. The script works when ran from the main polling engine with no issue, however when i plug the script into the monitor i get the following error:
When i go to that path, that temp file doesn't even exist there. why is it looking at a temp file?
I have followed Using a Python script as a SAM component and SAM Script Component Monitors - Everything you need to know, but i'm not sure what i'm missing here.
Any help would be greatly appreciated!