NYSERNet New York City Metro Fiber - FAQ

PDF Version

1. General Questions About the Dark Fiber Network
1. What is dark fiber?
2. What is the New York City Dark Fiber Network?
3. What is the history of the project?
4. What are the advantages of participation in the DFN for my institution?
5. What services are available to DFN members?
6. What kind of assistance does NYSERNet provide as we move through this process?


2. Questions About Transport Services
1. Will the fiber become obsolete?
2. Is Dark Fiber more or less reliable than a conventional telco circuit?
3. What is the highest bandwidth available over dark fiber?
4. What happens when there is a problem with the fiber?


3. Questions About Colocation Services
1. Please describe the primary colocation facility.
2. What are the specifications for the Colo @32?
3. Is there anyone on-site to help me?
4. What kind of building security is in place at 32 Avenue of the Americas?
5. I understand that there is a secondary PoP optionally accessible.


4. Financing, Billing and Contract Terms
1. What are the costs associated with participating in the DFN?
2. What billing options are available for my institution?
3. Is a ten-year commitment too long given the pace of technological change?


5. Contact Information
1. Program Manager

A1. What is dark fiber?

Dark fiber is industry-standard optical fiber cable that has not been lighted. It may be fiber that is already in the ground or strung on poles, or, as in the case of the NYSERNet DFN, fiber that has been purposely placed to meet the network requirements of our member institutions. It is no different than the type of fiber assets controlled by the major telecommunication carriers, except that it is entirely customer controlled, even to the extent of the customer selecting, acquiring and managing the equipment used for lighting the fiber.

A2. What is the New York City Dark Fiber Network?

The NYSERNet New York City Dark Fiber Network (also referred to as the Dark Fiber Network or DFN in this document) is a sustainable, regional model to provide participating institutions with nearly unlimited bandwidth capacity at a fixed price.

To support the high-speed networking needs of the participating institutions, NYSERNet deployed a dark fiber network, initially spanning Manhattan and portions of the Bronx, along with a data-center-class colocation facility at 32 Avenue of the Americas, to provide the following:

• Economical and high-performance access to the NYSERNet Research & Education Network;
• Economical and high-performance access to NYSERNet lambda services and projects;
• Economical access to commodity Internet service providers;
• Ability to implement private intra- and inter-institutional networks.

The fiber network and the colocation facility are scalable and flexible to accommodate growth from both current and new customers. The Colo@32 is the primary PoP (Point of Presence) for the New York City Dark Fiber Network and houses NYSERNet's and the members' equipment.

NYSERNet's The Colo@32 provides access to the NYSERNet Research and Education Network. It also serves as a peering point for other national and international research networks. Members can light the fiber and access a variety of network services by placing their equipment at their originating site and at their allocated space in the colocation facility. The DFN allows members to hand-off traffic among their own sites, to other DFN members, and to various providers of other network services.

A3. What is the history of the project?

In October of 2000, NYSERNet held a retreat for members of its Boards of Directors. At the retreat, Board members asked NYSERNet to investigate the feasibility of employing dark fiber as a means for providing access to the NYSERNet network in New York City.

In response, NYSERNet conducted a study to determine: (a) the level of interest of NYC-area institutions in participating in a dark fiber network, (b) the network design alternatives that might most cost-effectively fulfill the needs of these institutions, and (c) the costs of implementation of each of the designs.

The conclusion resulting from this study was that a network as described was attainable, and that its deployment could lead to essentially unlimited ability to upgrade bandwidth and genuine costs savings for its members. NYSERNet evaluated the option of leasing existing fiber in Manhattan and found that the cost to do so was comparable to that of building new facilities. By building a new network customized to the needs of its member institutions, NYSERNet created a more survivable and scalable infrastructure while taking advantage of the latest in fiber optic technology.

Subsequent to the study, NYSERNet distributed a Request For Proposals for construction and management of the fiber plant and the colocation facility. The ultimate result of this process was the selection of Lexent Metro Connect as the primary vendor and 32 Avenue of the Americas as the site for the colocation facility.

By August of 2003, the NYSERNet colocation facility at 32 Avenue of the Americas was fully operational, with Internet2's MAN LAN switch and several international research networks as the first tenants. The construction of the fiber backbone throughout Manhattan followed. Throughout 2004 the following institutions connected over 20 sites to the DFN:

• The American Museum of Natural History
• The City University of New York
• Columbia University
• Fordham University
• Mount Sinai School of Medicine
• New York Presbyterian Hospital
• New York University
• Weill Medical College of Cornell University

A4. What are the advantages of participation in the DFN for my institution?

The DFN offers members an effective solution for their current network services needs and positions them to scale gracefully and economically to meet the evolving needs of their end-users. The benefits of the DFN are as follows:

• Capability to upgrade network performance; flexibility to meet evolving needs of new applications for ever-higher bandwidth; and capability to implement private intra- and inter-institutional networks.
• Improved network reliability in a cost-effective manner with diversely routed fiber loops and diverse colocation facilities available.
• Economical and high-performance access to the NYSERNet R&E and Statewide Lambda Networks, to commodity Internet service providers and thus to a variety of network services.
• Allows riding of the declining cost curves with emerging optical network equipment and technologies, and supports new applications of end-users.
• Reduced overall costs for telecommunications services, especially local loop costs. Essentially, the DFN offers unlimited bandwidth capacity for monthly costs that are comparable to those for OC3-type bandwidth services (in the first 10 years), with potential for significantly lower monthly costs in subsequent years.

A5. What services are available to DFN members?

In addition to the transport and colocation services that form the core of the DFN, NYSERNet provides access to two Internet service providers and access to the NYSERNet R&E and Statewide Lambda Networks from The Colo@32, as well as intra-PoP peering among members.

NYSERNet continually evaluates and deploys services that provide members with additional opportunities to leverage their investment in their fiber plant.

A6. What kind of assistance does NYSERNet provide as we move through this process?

NYSERNet's Program Manager for NYC Metro Fiber Services and a dedicated Network Engineer will set up an initial meeting to:

• Review the features and benefits of participation for your institution.
• Review service options and help establish your institution's ultimate requirements.
• Review equipment considerations.

Subsequent to this first meeting, Lexent MetroConnect will conduct a site survey of the locations you plan to connect to the DFN. Once the site surveys are completed, NYSERNet will provide your institution with a proposed network design, the non-recurring and monthly recurring charges for your fiber plant and colocation services, as well as a service agreement.

With receipt of the signed service agreement, NYSERNet will authorize Lexent MetroConnect to begin the construction of your connection. NYSERNet will establish weekly status meetings to review construction progress, resolve any questions / concerns, and coordinate the transition / turn-up of services over the dark fiber network.

Following implementation, NYSERNet's NOC Manager will meet with you to review NYSERNet's comprehensive trouble-reporting and escalation procedures.

B1. Will the fiber become obsolete?

Optical fiber has been referred to as technologically neutral, similar in substance to the air in which wireless signals are transmitted. As the transceivers used to light the fiber evolve, they can be placed on the same fibers to achieve higher throughput, thus providing for nearly limitless upgrade-ability.

Gigabit Ethernet has been the most popular network technology for most metropolitan dark fiber networks due to its low cost, familiar capabilities and management, and simplicity.

B2. Is Dark Fiber more or less reliable than a conventional telco circuit?

There is little data on relative reliability of dark fiber paths versus telco synchronous (SONET) circuits. However, some comparisons can be made:

• Both connections rely on fiber optic paths; the SONET circuit uses dark fiber paths between SONET multiplexers, while the pure dark fiber circuit connects directly to the end equipment. Therefore both are vulnerable to fiber failures (typically cuts by construction equipment). SONET multiplexers typically use dual paths, with one carrying traffic and the other available as a backup, and are able to switch between them very rapidly during a failure. However, this uses twice as much fiber, and since the fibers are usually in the same cable the redundancy is limited unless a full ring topology is used (with three or more SONET multiplexers at separate locations). Similar designs can be built using routers or other network devices connected directly to dark fiber, although they do not usually have the extremely rapid failover characteristics of SONET.
• Dark fiber networks are managed by the network owner (each participating institution in the NYCDF Project manages their own network over their portion of the fiber plant), while telco circuits are managed by the provider. The dark fiber owner determines their own policies on management and maintenance, and knows the exact state of the circuit at all times. It is often the case that the telco provider's management capabilities are limited and do not detect partial failures, since the SONET failover means that the circuit continues to carry traffic. A second failure then leads to a total loss of connectivity, which would have been prevented had the initial problem been corrected earlier.
• Customer termination equipment for dark fiber is similar or identical to that used for SONET circuits, but the SONET equipment in the path introduces additional failure points, as well as increased maintenance and management complexity. Most SONET equipment has very limited remote troubleshooting capability. Customer-owned test equipment can be attached to a dark fiber path to completely test it end-to-end, and often locate faults directly by examining the fiber itself. Some types of test equipment can be left connected to the circuits continuously to provide management information and immediate diagnosis, or additional fiber in the same path can be used for that purpose.
• If SONET features or capabilities are required, customers can place their own private SONET multiplexers on their dark fiber paths, maintaining many of the dark fiber benefits and keeping control over the network while providing the same services available from a telco (typically at much lower cost).

B3. What is the highest bandwidth available over dark fiber?

Commercially available router interfaces currently run up to 10 gigabits per second (Gbps), using either 10 Gigabit Ethernet or OC-192. Future plans call for 40 Gbps, but this equipment is still under development. Higher transmission rates also place much more stringent demands on the fiber path, and not all paths may be usable at those rates. However, with wavelength multiplexing techniques multiple signals can be transmitted across the same fiber, dramatically increasing the total bandwidth available. Coarse wavelength division multiplexing (CWDM) allows for 8 or 16 channels, each of which can carry up to 10 Gbps. Since each channel runs at a lower rate, a fiber span that might not be able to carry 40 Gbps traffic could handle 160 Gbps total across sixteen 10 Gbps channels. Another variation, dense WDM (DWDM), uses very closely spaced channels in a higher wavelength band.

B4. What happens when there is a problem with the fiber?

Lexent monitors the fiber network on a 24x7x365 basis. Together with Lexent, NYSERNet has developed a comprehensive trouble-reporting procedure to facilitate timely resolution to any physical network problems

If the problem is of the nature of a backhoe cut in the backbone or laterals (that is to say, on the network side of the customer's demarcation point), it will be repaired by LMC at no expense to you (unless, of course, you or your agent were the cause of the incident). Incidents on the customer side of the demarcation will be your responsibility. You can engage LMC to perform this repair but it will be at your expense.

C1. Please describe the primary colocation facility.

The NYSERNet primary colocation facility is housed in approximately 1100 sq. ft. of carrier-grade space located on the 24th floor at 32 Avenue of the Americas. The Colo@32 serves as the NYC PoP for the DFN, the NYSERNet Research & Education Network, and the NYSERNet Statewide Network.

As part of the network design, each institution participating in the DFN has a dark fiber connection from its designated hub site to this facility. These connections facilitate access to the variety of services NYSERNet offers as well as peer-to-peer connectivity among members.

C2. What are the specifications for the Colo @32?

The standard dark fiber network design includes a 19 inch two-post rack with 30 amps of -48v DC power.

The Colo@32 also features:

• Separate fiber raceway with fiber management between racks.
• Dual (A+B) -48v DC power available to each rack, with multiple feeds (maximum 100 amps each, with a maximum of 100 amps total per rack) available in 10 amp increments.
• DC power source is NYSERNet-owned and managed battery plant, supplemented by building generators.
• 115 VAC, 20 amps duplex receptacle in alternate racks for convenience use.
• HVAC maintaining nominal temperature of 72o and 60% humidity, non-condensing.
• PSTN and ISDN facilities available.
• Building-maintained pre-action sprinkler system for fire suppression.
• Shipping and receiving services available.
• For authorized individuals - 24x7x365 escorted access provided by building personnel.

C3. Is there anyone on-site to help me?

A NYSERNet Engineer will assist with the initial installation of your equipment at The Colo@32. Although The Colo@32 is not a staffed facility, NYSERNet Engineers are available via phone and are periodically on-site to assist on an as-needed basis. In addition, remote hands services are available via a third-party vendor to provide 7x24 on-call support.

C4. What kind of building security is in place at 32 Avenue of the Americas?

The building manager, Rudin Management Corp., handles site security. Building staff will accompany all personnel during access to the colocation facility, and will remain until work is completed. Access to the building and to the colocation space is controlled by the building staff and, as noted above, only authorized personnel will be admitted. The NYSERNet facility is also monitored by video cameras.

C5. I understand that there is a secondary PoP optionally accessible.

In addition to the primary colocation facility at 32 Avenue of the Americas, NYSERNet's fiber plant extends to the carrier hotels at 111 8th Avenue and 60 Hudson Street. While not PoPs in the same sense as The Colo@32, members choosing to lease fiber from NYSERNet into these facilities have the capability to individually contract with a variety of providers for services.

D1. What are the costs associated with participating in the DFN?

Because the DFN is tailored to the specific needs of the participating institutions, the non-recurring and monthly-recurring charges are developed on an individual case basis following the site survey of the locations to be connected.

Following are the main factors in determining pricing:

• Number of sites to be connected.
• Single or dual entrance for the selected sites.
• Location of those sites vis-à-vis the backbone.
• Number of fiber pairs to be acquired.
• Addition of secondary colocation facilities.

D2. What billing options are available for my institution?

Your institution can elect quarterly, semi-annual or annual billing. If your institution prefers, the billing cycle can be tied to your institution's fiscal calendar.

D3. Is a ten-year commitment too long given the pace of technological change?

This project represents a capital investment by both NYSERNet and the member institutions; the service agreement term of ten years reflects this. Fiber optic cable has an estimated useful life of at least twenty years, and is considered technologically neutral. On the other hand, the technology utilized to transmit signals over fiber continues to evolve rapidly, constantly increasing in performance, while decreasing in price. Members gain increasing value out of the initial fiber investment through the utilization of increasingly more powerful transceivers, providing essentially unlimited upgrade-ability.

E1. Program Manager

Sharon Akkoul is NYSERNet's Program Manager for the New York City Metro Dark Fiber Network. Her contact information is:

Sharon Akkoul
315-413-0345 x5201
sakkoul -at- nysernet . org

Sharon is your primary contact for questions related to service offerings and options, implementation status, payment options, service agreements and general service questions. She will also coordinate the implementation with the NYSERNet Engineer to be assigned to your connection.