Designing the Data Center. Brian Hansen RCDD, CDT. Leviton Network Solutions Specification Engineer BICSI President

Transcription

1 Designing the Data Center Brian Hansen RCDD, CDT Leviton Network Solutions Specification Engineer BICSI President 00

2 Talking points What are Data Centers? Why Data Centers? The components of the Data Center Approaching the Data Center design Breakdown of the components The Applications Nuggets of information from TIA Grounding and Bonding from IEEE 00 Forward looking designs for Data Centers P

3 What is a Data Center? A place to locate computer, storage and networking devices safely and securely A place that provides un interruptable power needed to power a Company s mission critical devices A place that provides a temperature controlled environment for equipment to operate properly A place that provides connectivity to other devices inside and outside the data center

4 Components of the Data Center Physical space Raised flooring Electrical Standby Power Cooling Fire Suppression Physical security Physical access Cabling

5 Why are Data Centers Important? Passive connectivity Pass through Cross connect Wiring Closet CEO s, CFO s, CIO s and other senior mgmt don t spend much time and focus on closets Critical Applications ERP CRM SCM Back office ecommerce Data Center Critical Computing Systems Servers Storage Networking

6 The Approach

7 Designing a Data Center KISS Keep it simple sir Flexibility equipment changes Scalability K sq.ft. or M sq.ft. Modularity logical and master plan for equipment Standardize consistency and uniformity The Human Factor promote good habits Don t lose your sanity! Oops will happen. Work with Data Center Professionals

8 What is tiering? Tier Basic Tier Redundant Components Tier Concurrently Maintainable Tier Fault Tolerant Level of availability Percent Downtime per year Six nines. seconds Five nines. minutes and seconds Four nines. minutes and seconds Three nines. hours and minutes Two nines days, hours and 0 minutes

9 What is N? N = Maximum capacity needs N+ = Max. plus extra device(s) N+ = Max. plus extra device(s) N+ = Max. plus extra device(s) N* (N) = doubles need N* (N) = triples need Balance need versus cost and complexity of redundant systems

10 Flexibility Data Centers are not static use products that can be easily changed or moved. Cabinets that can accept various equipment Configurable Panels Outlets that can be changed with a cord

11 Modularity

12 Understanding the components

13 Physical space Data Center selection should consider: Seismic areas use earthquake resistant technologies Ice storms ensure backup generators can run for extended periods Hurricanes shelter against high winds Tornadoes no windows Flooding build on elevated ground Landslides have retention walls Fires install fire resistant walls and suppression systems

14 Physical space continued Polluted air limit outside air or use high filtration air systems EMI build away from sources of EMI Vibration stay away from railroads, airports, major thoroughfares Political climate labor issues and materials delay Flight paths distribute servers Service provider can you get enough bandwidth

15 Raised flooring Raised floor specification should address: Floor height for plenum is average Ensure there is enough ceiling height for heat exhaust Weight bearing capacity should be rated with same strength as sub floor Sub floor shall be sealed to prevent concrete dust ESD floors shall have static control No perforated tiles in main pathways heavy equipment being transported over them can break them Ramps should have a in incline, be in width and have landings on both ends or square for maneuvering equipment

16 Electrical Electrical specification should consider: Separate power sources building system power feed shall be isolated from server equipment power feed Bypass have maintenance bypass options for normal maintenance without taking down equipment Pre wire cabinet locations adds more upfront cost but less disruptive PDU loads do not load circuits more than 0 percent Use whips flexibility for moving power Receptacles limit to per circuit Heavy gauge wire 0 AWG is better than AWG Run six 0 AWG wires phase conductors, neutral, equipment ground and isolated ground Labeling label the circuit and location in circuit panel Convenience outlets do not tie into equipment power feed

17 Rack / Cabinet mounted PDUs

18 Standby power Standby power should be specified with the following considerations: Sizing of UPS Use peak load and not operating load Uptime should be a minimum of 0 minutes at full load with generator backup or hours without a generator backup Generators should be designed into your Data Center if uptime will be critical for extended outages in the area Battery room ensure adequate ventilation Monitoring lights utilize color coded domes and wire the lights into the various backup systems for instant visual clues such as red for UPS and blue for generator

19 Hot and cold aisles Front Power bank HOT tile can output KW BTU s Front For every degrees above ambient temp. (F-F) reduces equipment life by ½ Intel

20 The Applications

21 High Performance Computer Clustering CISCO CSS 000 SERIES CONTENT SERVICES SWITCH 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X CISCO CSS 000 SERIES CONTENT SERVICES SWITCH 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p System p 0 Gigabit Fiber Link Gigabit Copper Link Gigabit Copper Link Research, financial modeling, digital image rendering and scientific applications Aggregate four 0GBASE-T links to achieve 0 Gig. 0 Gigabit copper link 0 Gigabit copper link 0 Gigabit copper link

22 Network Attached Storage CISCO CSS 000 SERIES CONTENT SERVICES SWITCH 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X CISCO CSS 000 SERIES CONTENT SERVICES SWITCH 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X LAN p p p p p p 0 Gigabit Ethernet Servers 0 Gigabit Ethernet Servers NAS device NAS device Quick transfer of large files All 0GBASE-T links

23 Storage Area Network (SAN) CISCO CSS 000 SERIES CONTENT SERVICES SWITCH 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X 0/00 X X X X X X X X System p System p System p System p System p System p System p System p System p System p System p System p SAN SAN Storage arrays, tape drives Servers Data mining from large repositories of data Upgrade to 0GBASE-T Gig. Fibre Channels GBASE-T links Move to 0GBASE-T using iscsi

24 Cabling

25 TIA Telecommunications Infrastructure Standard for Data Centers What kind of information can I find in the document? Scope Definition Design Overview Cabling system overview Cabling system infrastructure Telecommunications spaces and Related Topologies DC cabling systems DC cabling pathways DC redundancy Multiple Annexes

26 Cabling Topologies Direct Connect Distributed

27 Power Supply Power Supply Catalyst 00 SERIES FAN THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY Sample Application SD SD SD SD SD SD SD SD WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET

28 Power Supply Power Supply Catalyst 00 SERIES FAN THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY Patch Cord into Server Sample Application SD SD SD SD SD SD SD SD WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0

29 Switch STAT Power Power Supply Supply Catalyst 00 SERI Zone Distribution Area FA N US THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY THE SIEMON COMPANY ES SD SD SD SD SD SD SD SD WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 0 WS-X-GE-TX P O R T 0/00/000 BASE-T SWITCHING MODULE PHONE ETHERNET 0 0

30 Data Center topology

31 Labeling the floor X Coordinate AA AB AC AD AE AF AG AH AI AJ AK AL AM AN Right Front Corner Sample Cabinet Y Coordinate Sample Cabinet Identifier: AJO

32 Sample of Panel Identifiers on Rack

33 design guidelines In addition to satisfying today s telecommunication requirements, the horizontal cabling should be planned to reduce ongoing maintenance and relocation. It should also accommodate future equipment and service changes. Consideration should be given to accommodating a diversity of user applications in order to reduce or eliminate the probability of requiring changes to the horizontal cabling as equipment needs evolve. Change your equipment and not your cabling!!! Minimum of Category is recommended. Minimum of Laser Optimized 0 micron is recommended.

34 Application: Frequency spectrum requirements 0GBASE-T GBASE-T 00BASE-T 0BASE-T *Shaded ovals show range & concentration of frequencies for Ethernet protocols MHZ

35 Characteristics of the A cable Larger copper conductor up to AWG Shorter twist ratios on pairs. Larger O.D. (UTP) up to. Grooved offset overall jacket (UTP) 00 Ohm impedance Backwards compatible with existing e and applications Interoperable with other Categories Utilizes and PC plug/jack interface Engineered to eliminate Alien Crosstalk Cat A Cat

36 Cross section of A UTP

37 Cross section of A F/UTP AWG tinned copper solid

38 design guidelines Transmission Differences MM vs. SM Systems

39 Differential Modal Delay

40 design guideline continued Power strips shall be labeled with the PDU/panel identifier and circuit breaker number. Racks that are supported by the access floor shall be bolted to the cement slab or a metal channel secured to the slab by threaded rods that penetrate through the floor tiles. Sharp edges on the top of the threaded rods shall be covered using domed nuts or other method. Exposed threads under the access floor should be covered using split tubing or other method. Power strips with indicators but no on/off switch or breaker reset button should be used to minimize accidental shut off.

41 Forward looking design

42 Overhead conveyance or underfloor?

43 Utilize pre terminated Category A systems Draft 0 was approved in Tampa Great for quick network deployment Excellent Channel to Channel characteristics Ideal for data center Server Row designs Lifetime System Warranty No field testing required at all 00% performance tested at factory to TIA CATA UTP or STP cables secured in braided sleeve Jack to Jack or Jack to Open configurations

44 Utilize pre terminated MTP design Consider OM fiber Great for quick network deployment Cabling harness can be changed for new equipment Excellent Channel to Channel characteristics Ideal for data center connector low loss channels Lifetime System Warranty No field testing required at all 00% performance tested at Factory MTP - SC Duplex MTP - LC Duplex MTP - MT-RJ

45 Advantages of Trunks Direct (Point-to-Point) Jumper Cable Installation Distributed Optical PnP Cabling Solution 00 Jumper Cables Trunk Cables

46 Standard Design Plug & Play Topologies Total Loss =.db

47 Standard Design Plug & Play Topologies PNP Topology Issues: Use of MTP Cassette per Cassette Total Path Loss =.db 0Gig Path Budget =.db Flexibility of MTP Not Exploited Requires use of Hybrid Jumper for Future Migration.dB Over Budget 0 Gig Ready

48 Plug & Play: Alternate Building Blocks Plug & Play is more than MTP Cassettes MTP Brackets: Reduce number of Hops Reduce System Loss Reduce System Cost Work above the Floor

49 Alternative Design Plug & Play Topology Total Loss =.0dB

50 Alternative Design Plug & Play Topologies Alternative Plug & Play Topology Advantages: Use of MTP Bracket & Harness per Interface Total Path Loss =.0dB 0Gig Path Budget =.db Flexibility of MTP Fully Exploited Simplified Future Migration Requires Harness Change Out Head Room = 0.dB 0 Gig Ready Allows for Organized Trunk Pre Cabling for Future Expansion

51 See what your equipment looks like Trunk / Harness Technology Simplifies administration Fan Intake Remains Clear Proper Bend Radius Adherence Simplifies Visual Identification Future Growth Is Built In Allows Pre Cabling to Cabinets for Rapid Deployment (servers) Consistent Channel to Channel Characteristics Trunk Cable Feed From Under floor

52 Switch or Server Cabinet Options x CATA UTP ports in U Minimizes server or switch cabinet rack space panel options Standard Flat Angled Recessed Angled

53 Never worry about wiring cabinets again Reduces cabling/congestion inside of cabinet Allows for copper and/or fiber connectivity Supports 0G connectivity No need to wait on cabinets for your data center projects!! P

54 Copper Switch Topology Foundry Example

55 Copper Topology Comm to Server Row

56 SAN Switch Topology Cisco Example P //00

57 The Data Center Professional

58 Great Communicator!! Value added services Understands your current and future requirements Understands codes and standards Deploys design methods that allows you to change your equipment and not cabling Can provide detailed documentation to all parties Calm and steady head

59 Summary Simplify Build it for scalability Build it using modularity Changes to your design can impact other areas Remove the oops factor by planning Associate and network with other Data Center professionals

60 Thank you for your time!