Change MTU size in a 3750 port

The Catalyst 3750 cannot change the MTU per port. It can only be done globally in the switch with the command "system mtu 1526". For it to take effect you have to reload the switch.
10/100 ports: Max is 1998
1G: 9000 bytes



C3750#sh system mtu

System MTU size is 1524 bytes
System Jumbo MTU size is 1524 bytes
System Alternate MTU size is 1524 bytes
Routing MTU size is 1500 bytes

Unable to read configuration. Try again later.

CISCO6500#wr
Unable to read configuration. Try again later.

CISCO6500#


This is caused by high activity on CLI. Probably, a sh run is taking place or any other action that locks the executation of the CLI cmd.

high CPU usage on cisco catalyst 6500 with itasca process

In a Cisco 6500 switch with ACE, if the process itasca is consumming too much CPU, it can be caused by logging supervisor command on ACE.

This command will dump the logging of ACE into the supervisor and consumes RP resources.

To resolve this remove the cmd "logging supervisor " from ACE configuration and make ACE send the logging to a log server directly.
 
--------------
 
The same problem was seen with logging generated by the switch itself.
The fix was: 
no logging on

At least, until the problem is found, the cpu was back to normal.

Before:
SWITCH# proc cpu sort
CPU utilization for five seconds: 99%/1%; one minute: 99%; five minutes: 99%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  62     2749624    243589      11287 86.82% 83.39% 84.82%   0 itasca
 131       11100     21481        516  0.47%  0.60%  0.67%   4 SSH Process
 499   5563701363409673472          0  0.39%  0.44%  0.45%   0 Port manager per
 
After: 
SWITCHh #sh proc cpu sort
CPU utilization for five seconds: 5%/1%; one minute: 9%; five minutes: 30%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
 499   5563783763409717288          0  0.79%  0.94%  0.87%   0 Port manager per
 560     40374923186106614          1  0.31%  0.31%  0.31%   0 HSRP Common
 273   2786680721058539466        263  0.31%  0.36%  0.41%   0 IP Input

 

ACE30 Monitoring with SNMP

I have been preparing some KPI for ACE30 blades which can give me a macro status of all services provided in several contexts inside ACE.

The goal is to measure these KPIs with SNMP using MRTG for example, builing a graph.
The interesting mibs that can be monitored are:
- Number of connection currently in use per context
- Traffic in and out of the ACE30 blade (16G connection to the backplane)
- CPU

Regarding the number of connections, you have to do it per context based. So, you will need an IP address in each context for snmp pooling proposes.
In my MRTG I am using the mib:

enterprises.9.9.480.1.1.2.1.8.7.100.101.102.97.117.108.116.3

As for traffic, the best way (till now) is to browse your interfaces with "snmp mib ifindex ifmib" and get the interface id to the ACE30. It will come up as a TenG interface.

How port-channel works in cisco switches

Port-channel is a feature to aggregate 2 links in an interconnection to another switch. It is very usefull in the following cenarios: - Increasing the throughput by adding another link. For example, in a port-channel of 2 Gbps, if you need more throughput, you just have to add aother link to the port-channel. There is no downtime altough the hashing of the port-channel will change. - Very efecient for link upgrade cenario as you will not have STP rotation.

Things you should know about port-channel that are not very clear in documentation: - The maximum flow in a port-channel is the maximum link throught that you have. This is beacause the best Cisco equipments can do is IP Hashing. This means, that each IP will be associated to only one port. This means that any flow from any to that IP behing it will always have 1Gbps throughput. - Aggregation protocols are very important such as LACP. It will add keeplive checks up to the port-channel to garantee that it is still up. LACP fast can detect a failure really fast. - High CPU can bring down aggregation protocols, which could end in link down catastrophic cenarios.

CSM force failover

To perform CSM failover, execute the following comand on active unit: clear module contentSwitchingModule 4 ft active

CSM Redirect serverfarm

In some cenarios, you might want to convert your loadbalancer into a redirect engine. Some reasons for that are: - Controlling http to https redirect - During maintenance period you can redirect the website to another location during webserver down time. - Migration cenarios. The script to execute this is: module ContentSwitchingModule 3 serverfarm MYFARM_80 nat server no nat client redirect-vserver MYFARM_RED webhost relocation https://newsite.com inservice ! vserver MYFARM_80 virtual 10.1.1.140 tcp 80 serverfarm MYFARM_80 persistent rebalance inservice !

NX-OS - Initial Setup

I am currently doing my first steps in NX-OS world, setting up a couple of 7000's.

My first goal was to setup a management ip address for remote management:
1-Initial login is admin/admin

2-Configure the interface mgmt. This is the interface on the supervisor

interface mgmt0
ip address 10.2.2.10/24

3-Configure a default gateway for this ip address. Remember that NX-OS has by default 2 vrfs: default and management. The mgmt0 interface can only be used with the management vrf. To configure routing for management do the following:

vrf context management
ip route 0.0.0.0/0 10.2.2.1

4-Diagnose

sh ip route vrf management
ping 10.2.2.1 vrf management

5-Other diagnostic commands
show ip arp
show ip traffic
show tcp statistics udp4
show ip client
show tcp client
show ip fib
show ip process
show ip route
show pktmgr interface
show frame traffic
show platform fib
show platform forwarding
show platform ip
show vrf
show vrf interface


more to came about nexus os ...

Check Optical Signal in Cisco SFP

Optics have to be DOM Compliant.

http://www.cisco.com/en/US/docs/interfaces_modules/transceiver_modules/compatibility/matrix/OL_8031.html
http://www.cisco.com/en/US/docs/interfaces_modules/transceiver_modules/compatibility/matrix/OL_6981.pdf

Some of them will not work.
The command is show interface transceiver. If the transceiver supports DOM, it will show something like this:


show interface transceiver
Transceiver monitoring is disabled for all interfaces.

If device is externally calibrated, only calibrated values are printed.
++ : high alarm, + : high warning, - : low warning, -- : low alarm.
NA or N/A: not applicable, Tx: transmit, Rx: receive.
mA: milliamperes, dBm: decibels (milliwatts).

Optical Optical
Temperature Voltage Current Tx Power Rx Power
Port (Celsius) (Volts) (mA) (dBm) (dBm)
---------- ----------- ------- -------- -------- --------
Te10/1 27.6 ++ 0.00 7.8 ++ -2.0 ++ -2.9 ++
Te10/2 27.4 ++ 0.00 7.8 ++ -2.0 ++ -3.1 ++
Te10/3 24.4 ++ 0.00 0.0 ++ N/A -40.0
Te10/4 23.6 ++ 0.00 0.0 ++ N/A -37.0 ++
Te10/5 25.3 ++ 0.00 0.0 ++ N/A -40.0
Te10/6 25.0 ++ 0.00 0.0 N/A -40.0
Te10/7 24.3 ++ 0.00 0.0 N/A -28.8 ++
Te10/8 26.0 ++ 0.00 0.0 N/A -30.0 ++

Trunking Dot1q - Applying allowed vlan in port - STP recalculation?

Some cisco switch ports have only the following configuration:

interface G2/1
switchport
switchport trunk encapsulation dot1q
switchport mode trunk

Well, this configuration is simple, because you only have to create the VLAN in the switch and it is automatically passed throught the trunk. Well, good for implementation, bad for security, performance and growth:
- In a security persepective, you have an uncontrolled trunk, specially if you are connecting to another switch that is not managed by you or your team.
- In a performance perspective, you have STP calculations permanently running and is case of a failure is this port can cause a downtime in other areas in the network.
- In a growth perspective, it is not scalable to use switchports without allowed in a growing scenarios such as Datacenters.

In a situation where you have uncontrolled trunk switchports (without the switchport trunk allowed vlan XXXX,...) and you want to limit the vlans passed in this trunk, there is always the question:
- Will this bring any downtime or STP recalculation.

Actually, I had to convert some of these links to allowed vlan mode, with hundreds of Vlans passing in a switchport, in a 6500, and when applying the allowed there was no recalculation of STP or any cut in the traffic.

My 2 cents for the ceptics ...

IP_SNMP-4-NOTRAPIP: SNMP trap source VlanXXXX has no ip address

The following error:

%IP_SNMP-4-NOTRAPIP: SNMP trap source VlanXXXX has no ip address

is caused by the change in the trap source. XXXX is the VID.
You can correct it with the following command:

snmp-server trap-source vlan XXXX

startup-config file open failed (Device or resource busy)

Error message:
"startup-config file open failed (Device or resource busy)"

The write to flash processo might have hang or might be in use by another user.

How to fix:

systat -> check which users are logged in
clear line vty X -> clear all the users from the router
write -> now you can write your config to flash

clear all lines. Any of the sessions is running a process that is writing to flash that might have been hang.

Running SDM 2.4 with Java 1.6.0_16

There is an exception when running SDM with Java version 1.6_16. The Java exception dump in the console is related with the class awt-eventqueue-2.

I was not able to correct the problem in this version. I have unselected the version 1.6_16 on my Java console control panel and I used a previously installed version 1.5 that I had. This corrected my problem.

How to Check a Cisco Interface Last Change

Sometimes, customers ask:
"I would like to know when was the last time this port went down!"

Well, our aswer will always be, "we will investigate and let you know."

The problem is, how can I get this information.
Actually, you will find it, if you have a syslog server, but what if the syslog rotates de logs earlier then your think !

Well, there is another way to get some info. Although Cisco says that it is valid, all my tests have not give me any confidence for this data:
Anyway, here it is how to get it.

Cisco provides a MIB for the last change of an Interface:
snmpwalk -v 1 -c .iso.3.6.1.2.1.2.2.1.9

IF-MIB::ifLastChange.1 = Timeticks: (10995) 0:01:49.95
IF-MIB::ifLastChange.2002 = Timeticks: (14524) 0:02:25.24
IF-MIB::ifLastChange.5001 = Timeticks: (12082) 0:02:00.82
IF-MIB::ifLastChange.5002 = Timeticks: (96967443) 11 days, 5:21:14.43
IF-MIB::ifLastChange.10101 = Timeticks: (97498708) 11 days, 6:49:47.08
IF-MIB::ifLastChange.10102 = Timeticks: (97497760) 11 days, 6:49:37.60
IF-MIB::ifLastChange.10103 = Timeticks: (97497340) 11 days, 6:49:33.40
IF-MIB::ifLastChange.10104 = Timeticks: (97496849) 11 days, 6:49:28.49
IF-MIB::ifLastChange.10105 = Timeticks: (97495422) 11 days, 6:49:14.22
IF-MIB::ifLastChange.10106 = Timeticks: (97494025) 11 days, 6:49:00.25
IF-MIB::ifLastChange.10107 = Timeticks: (97494995) 11 days, 6:49:09.95
IF-MIB::ifLastChange.10108 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10109 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10110 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10111 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10112 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10113 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10114 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10115 = Timeticks: (11332) 0:01:53.32
IF-MIB::ifLastChange.10116 = Timeticks: (97498710) 11 days, 6:49:47.10
IF-MIB::ifLastChange.10117 = Timeticks: (97497765) 11 days, 6:49:37.65
IF-MIB::ifLastChange.10118 = Timeticks: (97497353) 11 days, 6:49:33.53
IF-MIB::ifLastChange.10119 = Timeticks: (97496854) 11 days, 6:49:28.54
IF-MIB::ifLastChange.10120 = Timeticks: (97495448) 11 days, 6:49:14.48
IF-MIB::ifLastChange.10121 = Timeticks: (97494037) 11 days, 6:49:00.37
IF-MIB::ifLastChange.10122 = Timeticks: (97494996) 11 days, 6:49:09.96
IF-MIB::ifLastChange.10123 = Timeticks: (11333) 0:01:53.33
IF-MIB::ifLastChange.10124 = Timeticks: (11333) 0:01:53.33
IF-MIB::ifLastChange.10125 = Timeticks: (11333) 0:01:53.33


Use it in your own risk!

Serial Interface Sync

There serial interface can be in a state where it is up but looped. This means that the circuit has a loop on the other side, but not connectivity to any other point.

In order to correctly read the status of the interface, use the option:

interface Serial0/0
down-when-looped

This will bring the status to:

Serial0/0 is up, line protocol is down (looped)
Hardware is PowerQUICC Serial

When the V.35 Layer 1 connection is actually estabilhed, the HDLC will also sync with the logs:


Serial0: HDLC myseq 3354903, mineseen 3354903*, yourseen 25701, line up
Serial0: HDLC myseq 3354904, mineseen 3354904*, yourseen 25702, line up
Serial0: HDLC myseq 3354905, mineseen 3354905*, yourseen 25703, line up
Serial0: HDLC myseq 3354906, mineseen 3354905, yourseen 25703, line up

May 28 14:12:54: %IP_SNMP-4-NOTRAPIP: SNMP trap source FastEthernet0/0 has no ip address
May 28 14:14:03: %IP_SNMP-4-NOTRAPIP: SNMP trap source FastEthernet0/0 has no ip address
6d03h: Serial0/0: HDLC myseq 53150, mineseen 53149*, yourseen 53150, line up (looped)
6d03h: Serial0/0: HDLC myseq 53151, mineseen 53150*, yourseen 53151, line up (looped)
6d03h: Serial0/0: attempting to restart
6d03h: PowerQUICC(0/0): DCD is up.

After this log, the serial will be in the state:

Serial0/0 is up, line protocol is up