Servicenow Discovery

This article is a supplement to the ServiceNow documentation. For full documentation please refer ServiceNow official website

Checkout our NEW Video Channel you can like and subscribe too!

Introduction:

The Management, Instrumentation, and Discovery (MID) Server is a Java application that runs as a Windows service or UNIX daemon on a server in an enterprise network. Service Now interacts with external system and applications running on an enterprise network through this MID-Server. Mid Server can be used for various reasons like

Discover new/modified CIs in the enterprise network or in an amazon vpc.
Query LDAP information from an enterprise network
Use Import Set and Mid Server together to load data from an external system to Snow

Internally how it works:

The MID Server opens a persistent connection to the instance through the Asynchronous Message Bus (AMB) Client and listens on the /mid/server/<mid_sys_id> AMB channel. When an output record is inserted into the Queue [ecc_queue] table, an AMB message is sent to the MID Server’s channel. The MID Server receives this message and immediately polls the ecc_queue table for work.

The MID Server polls the ECC queue on the regular interval defined in the mid.poll.time configuration parameter, regardless of AMB message activity. The default polling interval is set to 40 seconds, but can be reconfigured. This polling of the ECC queue at a regular interval is done in case the AMB connection is dropped.

MID Server ECC queue polling process

snowdiscovery07052020 (2).png

MID Server Capabilities

Mid Server can interact with various systems/application throughout the enterprise network using protocols like snmp, wmi, ssh, powershell, aws, jdbc etc. snowdiscovery07052020 (3).png

Setup Mid Server

We will use guided ITOM setup to install mid server. ITOM guided setup helps to setup MID server in a step by step manner. We can setup mid server manually also. Search with ITOM In the left navigation pane snowdiscovery07052020 (4).png Click on continue in the next screen snowdiscovery07052020 (5).png

In the next screen we can see various Guided setups like Mid server setup, Discovery etc.For now select Mid Server snowdiscovery07052020 (6).png On the right side we can see the task we need to configure to setup

Create a MID user
Download and Install MID
Validate MID
Add SNMP credentials (optional)
Auto-Assign MID server IP Ranges(optional)

1.Create a MID user:

In the first step we need to create a user that we are going to use to communicate from MID server to the ServiceNow Instance. Click on Create Mid User from the above task list

snowdiscovery07052020 (7).png

In the next screen we need to add user name, password for the new user which we are going to use as mid_server_user snowdiscovery07052020 (8).png Once use gets created we can view the user under User application in Servicenow.

Important: In this user creation process, in the backend snow assigns a mid_server_role to this user so that it can communicate between servicenow and the midserver.

We can see that if we scroll down to roles under this user

snowdiscovery07052020 (9).png Now let’s mark this step complete snowdiscovery07052020 (10).png

2.Download and Install MID server:

snowdiscovery07052020 (11).png In this step we are going to install and start the mid server. In our example we are going to setup the mid server in a EC2 Instance.

First we need get the MID server binary from ServiceNow instance

Click on Configure button and the download page for MID server binary will be displayed as shown below.

Next right click on Linux 64 bit(as we are installing in EC2 linux instance)

snowdiscovery07052020 (12).png

And copy the link address as shown in the diagram above. Now let’s login to the EC2 instance as root user snowdiscovery07052020 (13).png Create a folder under temp as mid_server where we are going to download the agent snowdiscovery07052020 (14).png snowdiscovery07052020 (15).png

Now we will use the wget command to download the binary link which we copied from the above step snowdiscovery07052020 (16).png Once the download finishes we need unzip the package

Now we will see a folder below after unzipping snowdiscovery07052020 (17).png let’s go inside the folder and see the content snowdiscovery07052020 (18).png Now let’s configure this agent using the config.xml file snowdiscovery07052020 (19).png snowdiscovery07052020 (20).png

Once done save this file and start the server snowdiscovery07052020 (21).png Our agent sever is installed correctly now. let’s go back to service now instance and mark this step as complete snowdiscovery07052020 (22).png

3.Validate MID

As a next step we need to validate the MID server that we installed and started. In this step SNOW validates the mid server so that only authorized MID server can get access to snow. This is done of security reasons Now let’s see our MID server listed, search in the left navigation pane snowdiscovery07052020 (23).png

Click on MID server and in the right pane we can see the list of MID servers

When we first see the mid server it will appear as below snowdiscovery07052020 (24).png We have to click on the mid server and click on validate on the screen below snowdiscovery07052020 (25).png Once validated, we will see both status and validate as green

We now mark this steps also complete snowdiscovery07052020 (26).png

Step 4 and Step5 is optional

This is required if we have multiple MID server and multiple subnets in our environment.MID server will query the SNMP router and check the host ips from the router table. It will then try to discover those IPs under that router automatically.

snowdiscovery07052020 (27).png snowdiscovery07052020 (28).png We are going to skip this for now. As we are going to assign the IP of the host that the MID server needs to scan manually. This completes our MID server configuration steps.

Disovery Guided step

Now as we have completed MID server setup and installation .As a next step we have to setup the discovery part of it.

There are 4 phases in which discovery happens

PortScanning: This is the first step where SNOW instance will send a Shazzam probe to the list of IP /Ip Ranges defined in the discovery schedule. For example, if probe see an active response from port 135 it understands that this node is type of windows. Then discovery will launch the windows classification probe.
Classification: Now as Discovery knows that this a windows instance, it sends a WMI probe to gather OS related information like OS versions.
Identification: In this phase sends more probe to uniquely Identify this device. Discovery then uses the CI identification rule to check with CMDB for a matching CI. For example a Windows server CI example WIN00012 is already discovered in previous discovery schedules.It will check with cmdb for matching CI and update the CI instead of creating a duplicate CI
Exploration In this phase, Dicovery try to gather detail information about this device.For example cpu,memory usage,the process running etc.

Now let’s configure the discovery process Go to the ITOM guided setup and click on discovery snowdiscovery07052020 (30).png

Step 0: In this step we have activate the discovery plugin first Go to** https://developer.servicenow.com**

Click on Actions>Activate Plugin snowdiscovery07052020 (31).png And then activate the discovery plugin snowdiscovery07052020 (32).png Once the activation is successful we will get a confirmation in mail

Step1: So as first step we need set the discovery targets.

In this step we are going to define what the things we want to discover and exclude are.
snowdiscovery07052020 (33).png snowdiscovery07052020 (34).png

For software filter let us add a specific filter for java Write Java in the text box and click on the New key button under s/w filter snowdiscovery07052020 (35).png snowdiscovery07052020 (36).png

So we added a filter for Java discovery We can now mark this step as complete and move to next step

Step2: Add credentials

In this step we are going to specify the credential that the mid server will use to connect to the discovery node. snowdiscovery07052020 (37).png

In our example we will create another EC2 instance as a node that we want to discovery In our ec2 console we can see this environments as below snowdiscovery07052020 (38).png

Now to configure credentials click on configure This will open up the credential page Click on new to create a new credentials In our case we will have given the Private Key and IP in the credential form. Note: the Node doesn’t require to have a public IP, because MID server is in the same network so it will be able to communicate. snowdiscovery07052020 (39).png Click on New In the next page, select the type of credential, in our case it will SSH Private key credentials snowdiscovery07052020 (40).png In the next page we configure the details of the new credential form snowdiscovery07052020 (41).png Once done we can test it by click test credential and the IP of the machine to test snowdiscovery07052020 (42).png A pop appears asking for IP of the node which we want to use to test this credentials snowdiscovery07052020 (43).png After entering the details we click ok. If successful we will get the below confirmation snowdiscovery07052020 (44).png

So we are done validating the credentials we can mark this step as complete.

Step3. Setting up the discovery schedules

snowdiscovery07052020 (45).png In this step we are going to setup the discovery schedule. This process can be automate with Auto-Assign MID Server IP task (a part of the MID server guide steps) or we can create schedule manually. In this example we are going to create it manually. Head over to Discovery schedule. snowdiscovery07052020 (46).png

Create a new discover schedules snowdiscovery07052020 (47).png Now in the discovery form page we need to update the IP range, the job run time etc.

snowdiscovery07052020 (48).png Also we scroll down below we can see the list of IP it is going to scan. We can add a new range or a single IP also

snowdiscovery07052020 (49).png Here we have given a single IP of the node which we want to scan To create a new range, click on new and range form will get displayed as shown below

snowdiscovery07052020 (50).png We can also exclude the Ips which we don’t want to scan snowdiscovery07052020 (51).png Now we are set, we can now click on Discover Now to start the discovery process snowdiscovery07052020 (52).png This will star the discovery process, we can see the discover status progress in Discovery Status section

The description field indicates that the discovery has been started by user manually We can click on the discovery job number to see the detail discovery process

We can also see the timeline discovery showing all the 4 phases of discovery we mentioned earlier snowdiscovery07052020 (53).png Once the discovery is completed we can check the status under Discovery Schedule page

If we click on this completed task we can see • Discovery Log: Logs generated while discovering • Devices: list of devices discovered in this process Here we can also see whether the CI go discovered is an create or update of a CI snowdiscovery07052020 (54).png • ECC Queues: The Queue messages used during the discovery process

Let’s see the CI which this discovery just created/modified Our CI name is ip-172-31-29-25 We go to device type and click on the CMDB CI snowdiscovery07052020 (55).png

This will open up the CI view window snowdiscovery07052020 (56).png

Scroll down below to see the list of discover software,nework adapter, file system etc snowdiscovery07052020 (57).png snowdiscovery07052020 (58).png snowdiscovery07052020 (59).png snowdiscovery07052020 (60).png