chrisp3

updated

17 years ago

C

chrisp3

111 Posts

0

2584

April 13th, 2010 17:00

Network Configuration NS480

We are purchasing an NS480 to replace and NS702 and I have been looking at possible network configurations

We are getting a dual xblade NS480.

If I use copper, I was going to a fail safe network of 3 to 1.

Server_2 CGE0 - CGE2 go to switch 1 and CGE3 go to switch 2

Server_3 CGE0 - CGE3 go to switch 2 and CGE3 go to switch 1

The 3 ports CGE0-CGE3 would be in an etherchannel.

Does this copper config make sense?

From what I have read this should work.

From what I read 10 gig does come by default with xblade

On the fibre 10 gig side, I am not so clear.

I would like to think of a config that uses 10 gig

and 1 gig fail safe.

Would you connect 10 gig optical to one switch and then 1 gig copper to another switch?

Where do you plug in the 10 gig optical...in AUX0?

Where would you plug in the 1 gig fail safe....in CGE0?

I appreaciate any thoughts or comments on either config

Chris

  • Rainer_EMC

    6 Operator

    •

    8645 Posts

    893

    0

    Posted May 21st, 2010 09:00

    From what I read 10 gig does come by default with xblade

    actually 10gig isnt the default

    when you order a NS-480 you have a choice of either

    - four 1Gbps copper ports

    - two 1Gbps fiber and two 1Gbps fiber ports

    - two 10Gbps fiber ports and two 1Gbps copper ports

    per data mover

    see  http://corpusweb130.corp.emc.com/ns/NS120_NS480/hardware/components/NS480_overview.pdf

    Rainer

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    700

    1

    Posted April 13th, 2010 18:00

    Ethernet channel can combine two, four or eight (must be in multiples of two) Ethernet ports,  you might have to go Link aggregation (LACP) route. I only had 3 ports available for this specific connection so i connected two ports to switch A and one port to switch B. Created an LACP trunk on switch A and then combine the trunk and regular cge from switch B into an FSN device where the trunk device is configured as active. This is all with copper ..no experience with optical.

  • IanSchorr

    117 Posts

    700

    1

    Posted April 13th, 2010 21:00

    Aux 0 should be a Fibre Channel port, so wouldn't be used for Ethernet.  I'm not sure offhand which models, if any, come with a 10Gig port by default (my understanding was that you still choose your port configuration, whatever kind you want), though your salesman should definitely know.  It's certainly a configuration that can be purchased

    If you do buy a 10Gig port, though, yes, what you're saying makes sense.  A good configuration is just like you say - set up the 10Gig port in an FSN (for redundancy), with a cluster (aka "trunk" or "channel") of 1Gbps ports as "standby" in the FSN, in case the 10Gig link goes down.  The 1Gbps ports should be plugged into a different switch than the 10Gig port for switch redundancy.  (Or if you bought 2 10Gig ports, just set up the 2 ports as redundant ports with an FSN, and connect each to a different switch).  On your new hardware they may be marked as "cge0-0" through "cge0-3".

    Configuring multiple 1Gbps ports (usually 4) into a single trunk/channel is pretty standard.  Having more ports gives higher bandwidth available (with the gotcha that a single host with a single CIFS/NFS conversation will never use more than one port at a time).

    Just to mention - if possible, we always recommend going with LACP (aka "Etherchannel with LACP" in Cisco terms) than just regular Etherchannel (aka "static Etherchannel" or "unmanaged Etherchannel").

    Since "Etherchannel" is unmanaged, there is no consistency checking or communication about the ports between the DM and the switch.  It's much easier to misconfigure the channel, easier to misconfigure and *not know it*, and more difficult to tell the state of the channel, than with LACP.  And as Dynamox mentioned, some switches have requirements on the exact numbers of ports that can be in a "static Etherchannel".  There's also virtually no downside to LACP, except that there are still a few really old or low-end switches that still don't support it.

  • chrisp3

    111 Posts

    517

    0

    Posted April 14th, 2010 07:00

    I appreciate your input...you are correct.

    We are using LACP.  I should have said

    port channel and not etherchannel.

  • dynamox

    11 Legend

    •

    20419 Posts

    •

    87439 Points

    700

    0

    Posted April 14th, 2010 08:00

    Ian,

    when someone configures ethernet channel or link aggregation, one needs to select one of the 3 methods for statistical load balancing ( TCP port, IP and MAC). Are there any recommendations when to select specific method ?

  • IanSchorr

    117 Posts

    700

    0

    Posted April 14th, 2010 16:00

    Virtually always you'd want to select TCP (which uses a combination of TCP ports and IP addresses to choose the port to use).  It gives the most even distribution of traffic across the ports in the trunk/channel. That's generally true of the Celerra and the switches you're connecting it to.

    In fact, I can't think of a situation at the moment where you wouldn't want to use "tcp".  Even if the switch you're connected to doesn't support that mode, it won't affect the load balancing mode you choose to use on Celerra.

    "IP" mode, which is the default, generally will give a smooth amount of load balancing.  Though like the "tcp" setting, the more clients are talking to Celerra, the smoother the load distribution is between ports.

  • cadencep45

    4 Apprentice

    •

    318 Posts

    700

    0

    Posted May 6th, 2010 06:00

    I have a question about this statement

    In fact, I can't think of a situation at the moment where you wouldn't  want to use "tcp".  Even if the switch you're connected to doesn't  support that mode, it won't affect the load balancing mode you choose to  use on Celerra.

    The reason being is my networking team just told me the lacp load balancing works on the lowest common denominator principle, ie all switches and LACP source need to be able to support the load balancing mode choosen.

    Because cisco switches have most of this data calculated in hardware, it they only support IP or MAC you are restricted to these choices.

    Are you saying that load balancing mode is not a LCD issue and even in a cisco switch in the network link between source and destination does not support TCP Port it can still be used ?

  • IanSchorr

    117 Posts

    701

    0

    Posted May 9th, 2010 02:00

    " The reason being is my networking team just told me the lacp load balancing works on the lowest common denominator principle, ie all switches and LACP source need to be able to support the load balancing mode choosen."

    Nah, that's not true.

    When a switch receives a packet on an EtherChannel (or "port group", or whatever that vendor calls it), it doesn't care what the link partner (in this case Celerra, but usually another switch) sent a packet on.  It receives the packet on any port, associates it with the EtherChannel/port group, and processes it.  Then it makes a TOTALLY INDEPENDENT decision on which port to *transmit* packets on.  Heck, the Celerra COULD send packets for the same conversation on a different port each time, and the switch wouldn't care (though this could cause other problems - mostly out-of-order packet delivery.  But that's not really important here).

    It's very common for packets to be sent on one link in an EtherChannel which travelling one direction, and sent on another link when travelling in the other. And it's absolutely possible for the two link partners to use different load balancing methods.  In fact, even if both systems have chosen to use, say, "IP" load balancing, that doesn't mean that they'll both choose the same port for the same conversation (and frequently doesn't, since different vendors use different algorithms to decide which port to use).

    "Because cisco switches have most of this data calculated in hardware, it they only support IP or MAC you are restricted to these choices."

    That's not true for the reasons above.  And most Cisco switches sold in the last decade support TCP as well anyway =)

  • DanJost

    190 Posts

    701

    0

    Posted May 10th, 2010 07:00

    I posted this on another thread, but not all (unless later IOS releases have changed this) Cisco switches support port-based load balancing:

    http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a0080094714.shtml#catalyst

    But from what I understand of LACP/Etherchannel, you are indeed correct about the algorithm on determining which port to use doesn't need to match.  I'm using HP VirtualConnect Flex-10 and they use a combination of src-ip-dst-ip-src-port-dst-port and I haven't had any issues with my 3750E's which don't support that combination.

    Once a port is chosen, it is my understanding that one "end" sends all of the data down one path.  The more variables involved in the algorithm, there is a better spread (theoretically) over time.  On a routed network, MAC-based balancing is useless on the datacenter side.  On the non-chassis switches, src-dst-ip is the best you are going to get on the Cisco side (haven't had a chance to look at the Nexus switches, perhaps they are better in this respect)... not everyone has the money for a chassis or the high port density requirements in the datacenter.  10Gbit ethernet makes this less of an issue (today at least).

    Dan

  • cadencep45

    4 Apprentice

    •

    318 Posts

    421

    0

    Posted May 10th, 2010 08:00

    fine.  My goal was to understand logically how things worked, improving speed of access is a different ball game. I am currently digesting the RainFinity installaion and deployment guide. Basically I want to be in a position to ask questions that make sense before I call in the enlightened !