Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Troubleshooting TCP/IP Communication Issues

News

Linux Network Troubleshooting

Recommended Books

Recommended Links

Network Utilities

Redhat Network Configuration Suse 10 network configuration
Ethernet Protocol ifconfig ethtool How to change IP address in RHEL netstat Xinetd Remote Syslog
route command Linux Routing DNS nslookup hostname Changing hostname Procedure for installing Qlogic QLE2460 cards
NFS vsftp pure ftpd rsync NTP RHEL NTP configuration Troubleshooting NTP on Red Hat Linux
Telnet Protocol VNC on Linux   SSH Autonegotiation Samba Sendmail on RHEL Postfix
TCP Performance Tuning Bonding Multiple Network Interfaces Linux multipath InfiniBand Installing Mellanox InfiniBand Driver on RHEL 6.5 Setting up a basic infiniband network Troubleshooting InfiniBand connection issues using OFED tools
USB to Ethernet Adapter in Linux USB to wireless adapter Network Manager overwrites resolv.conf Tips Admin Horror Stories Humor Etc
Neil Cashell
Proactive Resolution Team
Novell Worldwide Support
[email protected]
 

15 May 2000

This document addresses communication issues that generate about a third of the support calls coming into the TCP/IP group at Novell Technical Support. We recommend that anyone who is implementing TCP/IP in a NetWare 5.x environment read and understand the information presented here.

This article is divided into two parts: understanding the concepts behind IP routing, and troubleshooting common TCP/IP problems. A follow-up article will explain some of the TCP/IP tools that are available for use in troubleshooting problems in a TCP/IP environment.

Concepts Behind TCP/IP Routing

The majority of connectivity issues involve problems with routing table entries. Every packet being processed by a TCP/IP host has a source and destination IP address. Upon receiving each packet, the IP protocol examines the destination address of the packet, compares it with entries in its local routing table, and then decides what action to take:

  1. If the destination IP address is itself (that is, to a local application such as GroupWise, BorderManager Proxy Server, etc.), the packet is passed up to a protocol layer above IP.
  2. If the packet is destined for another known network, the packet is forwarded through one of the locally-attached network adapters. (This assumes that the TCP/IP host has multiple interfaces and has routing enabled.)
  3. If neither of the above apply, the packet is discarded.

The TCP/IP routing table can maintain four different types of routes, listed below in the order that they are searched for a match:

IP compares the destination IP address of the packet that it is processing with the entries in the table. If IP finds that a host entry exists and matches the destination IP address, it will forward the packet to the next hop associated with that host entry. Host entries are usually found in routing tables when ICMP (Internet Control Message Protocol) has added the entry because of the pathMTU algorithm, or from an "ICMP redirect" call. To check this, load the TCPCON utility at the server console prompt and look at the IP Routing Table option to verify if the protocol associated with that route is ICMP.

IP has three classes of addresses: Class A, Class B and Class C. Each class contains a default subnet mask (for instance, Class A has 255.0.0.0. as a default subnet) until a class of addresses is broken into extra networks (i.e., subnetted). However, once the network is subnetted, the IP address will not have the default subnet mask.

So if IP doesn't find a host entry, but does find a subnet entry that matches the packet's destination IP address, IP will forward the packet to the next hop associated with that subnet entry. Subnet entries exist when RIP2 (Routing Internet Protocol v2), OSPF (Open Shortest Path First), or static entries have been added to the routing table through a non-default subnet mask.

If IP doesn't find a subnet entry in the TCP/IP routing table but does find a network entry that matches the destination IP address, IP will forward the packet to the next hop associated with that network entry. (Customers running in default NetWare TCP/IP mode will have network entries.)

Finally, if IP doesn't find a network entry, but does find that a default route entry exists, IP will forward the packet to the next hop associated with that default entry. The default route is most commonly inserted as a static route through NetWare's server console INETCFG utility. However, the route may also be learned via RIP or OSPF. Failure to at least have a default route can often lead to communication problems on the network.

If an IP packet match has not been found in the TCP/IP routing table at this stage, the packet is simply dropped and an ICMP "destination unreachable" message is triggered to notify the sender that the host or network is unreachable.

When a TCP/IP communication problem occurs, the most common reason is that a route entry doesn't exist for the network or host with which you are trying to communicate. When this is the case, you can either add a route entry or try to figure out why the route is missing.

Troubleshooting Common TCP/IP Problems

When troubleshooting any networking problem, it is helpful to take a logical approach. Some questions to ask are:

Troubleshooting a problem "from the bottom up" is often a good way to quickly isolate what's wrong and come up with a solution. The "bottom up" approach from an IP routing perspective is to start by verifying that the problem is not related to the physical layer (cabling, hubs, switches, and so on) or ARP (Address Resolution Protocol). Next, you ensure that the IP routing table is functioning correctly. Finally, you check to see whether the problem is at a generic TCP/UDP or application level.

To better understand the TCP/IP troubleshooting scenarios covered in this article, we'll use a small example network to illustrate some of the most common IP problems. This example network is shown in Figure 1.

Figure 1: Example network for TCP/IP troubleshooting scenarios.

 

 

In this network, Workstation 1 accesses the Internet/WAN through a NetWare server which contains two network adapters, each with its own IP address: 137.65.43.1 and 137.40.3.1. Workstation 2 accesses the Internet/WAN through the Internet Router with the IP address of 137.40.3.4. The NetWare server also communicates to the Internet/WAN through the Internet Router, as well as the Unix box (whose IP address is 137.40.3.3), which also communicates to the Internet/WAN through the Internet Router (137.40.3.4). The Internet Router's IP internet address is 137.30.1.254.

It's also important that you understand the terms "local host" and "remote host" in an IP network environment:

From the point of view of Workstation 1 in Figure 1, the NetWare server is considered a local host because its network adapter is attached to the same IP subnet as Workstation 1. Workstation 2, whose IP subnet address is different than that of Workstation 1, can be considered a remote host.

The following scenarios, which represent six of the most common IP problems, use the example network in Figure 1 as a reference. The most common solutions are given for each of these problems. While this is not a comprehensive list of solutions, they cover most of the routing issues that customers face.

Scenario 1: Cannot PING or Communicate with Local Router

Symptom: The user cannot PING from Workstation 1 (137.65.43.2) to the local segment side of the NetWare server (137.65.43.1).

Solutions: If two nodes on the same subnet cannot PING each other successfully, you can use the "ARP _A" command at a Windows workstation to check the ARP table entries. The -A parameter displays the ARP entries by interrogating the current protocol data. If more than one network adapter uses the Address Resolution Protocol, you'll see entries for each ARP table.

You can also use the TCPCON utility on the NetWare server to view the IP Address Translations Table. Select the Protocol Information | IP | IP Address Translation options, and see if the computers have the correct MAC addresses listed for each other.

Note: You can use the IPConfig utility (for Windows NT), the WINIPCFG utility (for Windows 95/98), or type CONFIG <Enter> at the NetWare server console to determine a host's MAC address (displayed as Node Address).

  1. If an ARP entry exists for the default router's IP address, perform the following troubleshooting steps.
  2. If no ARP entry exists for the default router's IP address, this usually indicates that there is a hardware problem with the devices on the network. Perform the following troubleshooting steps.

Scenario 2: Cannot PING or Communicate with Remote Interface of Local Router

Symptom: The user can PING from Workstation 1 (137.65.43.2) to the local segment side of the NetWare server (137.65.43.1), but not from Workstation 1 to the other side of the NetWare server (137.40.3.1).

Solutions:
  1. In this scenario, Workstation 1 needs to know which IP router to send the IP packet to when the destination network is on a different subnet (to a remote host, according to our earlier definition). This procedure is not required if Workstation 1 wants to communicate with hosts only on its local subnet (local host). Each TCP/IP stack configuration (whether client or server) has a parameter for a default router or gateway. (See TID #10018660 for information on configuring and troubleshooting client issues on Windows 95/98 and NT.)

    In this scenario, Workstation 1 would need to configure as its default router the IP address of the server's network adapter that is local to the workstation. The IP address would be 137.65.43.1. This implies that any packets that Workstation 1 will transmit to any remote hosts will be sent through this IP address.

  2. The NetWare server must be configured as an IP router so that it can forward packets from one network interface board (137.65.43.1) to the other (137.40.3.1). For this to happen, TCP/IP must have been loaded with the parameter "forward=yes" as part of the configuration.

    The best way to verify that TCP/IP has been loaded with forwarding enabled is through the TCPCON utility. Load TCPCON at the server console. You will see the "IP Forwarded: numbers" entry in the lower left-hand corner of the top window. If this entry has numbers after it (even if it is 0), then this server is configured as an IP router. If this entry has DISABLED after the statistic, it is not set to gateway mode. To enable this, load the INETCFG utility at the server console, select the Protocols entry, the TCP/IP entry, and then ensure that the "IP Packet Forwarding" parameter is set to ENABLED. (See TID #10013002 for more details.)

  3. Check to see whether the "Local Errors" field in TCPCON | Statistics | IP entry increases as your PING requests fail. This field increments anytime IP drops an incoming packet for any reason. If this field is increasing, perform the following diagnostic steps:

Scenario 3: Cannot Ping or Communicate with Internet Router

Symptom: From Workstation 1 (137.65.43.2) the user can ping both IP addresses that are bound to the network adapters in the NetWare server (137.65.43.1 and 137.40.3.1), but cannot ping the Internet Router (137.40.3.4).

Solutions:
  1. By default, the NetWare server uses RIP as its routing protocol. However, most IP Routers use either OSPF (Open Shortest Path First) or IGRP (Interior Gateway Routing Protocol) as the routing protocol of choice. Since the routing protocols are different on both routers, they will not update each other's routing table. The IP Router will not have a route back to the 137.65.0.0 segment, and will therefore not know how to respond to Workstation 1's PING.

    To fix this problem, insert a static route entry at the IP Router. On a NetWare server, this can be done using INETCFG by selecting Protocols| TCPIP| Static Routes. This entry tells the IP Router that in order to get to the 137.65.0.0 subnet, packets must go through the 137.40.3.1 gateway, which is the IP address of the NetWare server for the segment local to the IP Router. This implies that any time the Internet Router has a packet destined for 137.65.0.0, it will send it to the 137.40.3.1 gateway.

  2. Another possible solution is to synchronize the routing protocols at the NetWare server or IP Router so that they both understand either RIP or OSPF. You do this by enabling the same routing protocol on all routers in the network. This will guarantee that the routes being advertised by both sides will dynamically enter the necessary routing tables. Note that an ASBR (Autonomous System Boundary Router) can also be set up on either router to act as a conversion gateway between OSPF and non-OSPF (static, ICMP, or RIP) routes.
  3. Check to see whether an ARP entry exists for the Internet Router (137.40.3.4 ) in the NetWare server. To do this, go to the TCPCON | Protocol Information | IP | IP Address Translations screen.
  4. One other possible problem is that another device is responding to the ARPs using the Internet Router's IP address. In this case, there is either an IP address conflict or a bad switch.

Scenario 4: Cannot PING or Communicate with Remote Workstation

Symptom:

From Workstation 1 (137.65.43.2), the user can PING both IP addresses that are bound to the network adaptersin the NetWare server (137.65.43.1 and 137.40.3.1), and the Internet Router (137.40.3.4), but cannot PING Workstation 2 (137.40.3.2).

Solutions:

As described in Scenario 2, the workstation must have its default router or gateway set in order to reply or send packets to segments other than its local segment gateway. (See TID #10018660 for information on configuring and troubleshooting client issues on Windows 95/98 and NT.)

  1. From a Windows workstation DOS prompt, type the "NETSTAT -R" command. This command displays protocol statistics and current TCP/IP network connections, while the -R parameter displays the routing table. You can use this information to verify whether a default route exists from Workstation 1 to Workstation 2, and whether the route points to the next correct hop router for that subnet.
  2. Configure the default gateway on Workstation 2. In this scenario, the default route should point to the IP address of the Internet Router (137.40.3.4), or to the server network adapter that is local to Workstation 2's segment (137.40.3.1) Then reboot the workstation (unless you used the "ROUTE ADD" command as mentioned in TID #10018660 to insert the static route at the workstation).

Scenario 5: Cannot PING or Communicate with Remote UNIX Host

Symptom:

From Workstation 1 (137.65.43.2), the user can ping both IP addresses that are bound to the network adapters in the NetWare server (137.65.43.1 and 137.40.3.1) and the Internet Router (137.40.3.4), but cannot PING the UNIX box (137.40.3.3).

Solutions:
  1. At the UNIX box, use the "NETSTAT -R" command to see if a default route (0.0.0.0) exists on that box. If no static route exists, you must enter one in order for the UNIX box to have a route to the 137.65.0.0 subnet. The syntax for adding a static route on the UNIX box in this scenario should be similar to the following:
    route add net 137.65.0.0 137.40.3.1 1

    (For more information on the route command for UNIX, refer to the documentation that comes with your UNIX software.)

  2. Synchronize the routing protocols at the NetWare server or at the Unix box so they will both understand either RIP or OSPF protocols (as explained in Scenario 3, Step 2). This will guarantee that the routes being advertised by each side will dynamically enter both routing tables. Note that an ASBR (Autonomous System Boundary Router) can also be set up on either the NetWare router or the UNIX box to act as a conversion gateway between OSPF and non-OSPF (static, ICMP, or RIP) routes.
  3. Check to see whether an ARP entry exists for the UNIX box (137.40.3.3 ) in the NetWare server. To do this, go to the TCPCON | Protocol Information | IP | IP Address Translations screen.
  4. One other possible problem is that another device is responding to the ARPs using the Unix box's IP address. In this case, there is either an IP address conflict or a bad switch.

Scenario 6: Cannot PING or Communicate with Remote Hosts Beyond the Internet Router

Symptom:

From Workstation 1 (137.65.43.2), the user can ping both IP addresses that are bound to the network adapters in the NetWare server (137.65.43.1 and 137.40.3.1) and the Internet Router (137.40.3.4). The user can also ping Workstation 2 (137.40.3.2) and the UNIX box (137.40.3.3), but cannot PING past the Internet Router.

Solutions:
  1. In this scenario, the NetWare server knows about both the 137.65.0.0, and the 137.40.0.0 subnet segments, but it does not know where to route the packet if the destination is not on either of these segments. To fix this, you must add a default route to the NetWare server. Load the INETCFG utility at the server console and go to the Protocols | TCP/IP | Static Routing entry (be sure it is Enabled), then go to the Static Routing Table entry. Press <Insert> to add Default Route with an IP address of Network/Host 0.0.0.0 and with the Next Hop Router on Route (Gateway) of 137.40.3.4 Metric 1 Passive. This information is then written to the SYS:ETC\GATEWAYS file. (See TID #2911404, "Set LAN Default Route NW 4.x, 3.x, WEB, Proxy," for more details.) Then use the Reinitialize System command from INETCFG's initial "Internetworking Configuration" window to add the static route to the routing tables located in the server's memory.
  2. Run commands, such as the "show ip route summary" for Cisco IOS at the Internet Router's console prompt to view the Internet Router's routing table and see if Workstation 1's network (137.65.0.0) has an entry. Because of dynamic routing protocols such as RIP and OSPF, this should normally be the case. Problems here may indicate that the Internet Router's routing table is not being updated correctly. If no route exists, insert a static route for the 137.65.0.0 network and investigate why the 137.65.0.0 network is not being advertised by the dynamic routing protocols.

    To troubleshoot this problem, you first need to understand the network layout. Having the layout in mind will enable you to identify other routers in the network that should be advertising the route. You can use LAN traces to verify whether or not these other routers are advertising the missing network, and if so, with the proper parameters, such as hop count. In some cases, invalid hop counts may be advertised and the routes are being dropped accordingly.

  3. Verify that the packet is not being blocked through some filtering mechanism, such as the IPFLT.NLM. If this NLM is loaded, type "Unload IPFLT.NLM" at the server prompt, then check to see if the behavior is the same.

In the next column, we'll look into an extension of this troubleshooting scenario dealing with subnets and a couple of the more common problems that users face with subnetting.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Feb 07, 2017] Troubleshooting method for networking problems: work up the OSI model - layer 1 - check the cabling

Troubleshooting method - work up the OSI model - layer 1 - check the cabling. After checking the cabling, check the cabling again. Before you're ready to escalate, ask for help, check the cabling again.

pseudocyber Jul 28, 2015 9:20 AM (in response to adcast)

Troubleshooting method - work up the OSI model - layer 1 - check the cabling. After checking the cabling, check the cabling again. Before you're ready to escalate, ask for help, check the cabling again.

[Feb 06, 2017] The way to keep senior management informed

rbrickler Jul 17, 2015 11:52 AM

I was working for Network Operations in a company several years back. It was a small company and we had a VP that was not tech savvy. We were having an issue one day, and he came running into the Network Operations Center asking what was going on.

One of our coworkers looked at him and said, relax, it is no big deal, we have everything under control. He asked what was the problem.

Our coworker said, "the flux capacitor stopped working, but we got it restarted." The VP said OK, turned around and left the room to go report to the execs about our Flux Capacitor issue....

[Feb 05, 2017] Cutting yourself from the networked server by putting down and then up eth0 interface

jemertz Mar 30, 2016 10:26 AM
When working in a remote lab, on a Linux server which you're connecting to through eth0:

use: ifdown eth0; ifup eth0

not:

ifdown eth0 
ifup eth0

Doing it on one line means it comes back up right after it goes down. Doing it on two lines means you lose connection before you can type the second line. I figured this out the hard way, and haven't made the same mistake a second time.

[Feb 05, 2017] Cutting yourself from the networked server by putting down and then up eth0 interface

jemertz Mar 30, 2016 10:26 AM
When working in a remote lab, on a Linux server which you're connecting to through eth0:
use: ifdown eth0; ifup eth0

not:

ifdown eth0 
ifup eth0

Doing it on one line means it comes back up right after it goes down. Doing it on two lines means you lose connection before you can type the second line. I figured this out the hard way, and haven't made the same mistake a second time.

[Aug 04, 2015] My 10 UNIX Command Line Mistakes by Vivek Gite

Both article and the discussion are well worth reading in full

3ToKoJ June 21, 2009, 9:26 am

I can add, being trapped by aptitude keeping tracks of previously planned - but not executed - actions, like "remove slapd from the master directory server"

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019