NYC Mesh

Zach

Supernode 3 is here!

Supernode 3 has finally landed, and in a big way. This node will help bring major connectivity to Brooklyn and launch the movement of the mesh to lower Brooklyn, extending as far as hopefully Staten Island.

Earlier this month a presentaton was given at the Org Meeting: NYC Mesh Org Meeting - June 11 2019

This Supernode features:

  • Strong internal network

    • 40Gbit Internet donated from Pilot Fiber (Thank you!!)
    • 40Gbit internal network, 10Gb+ to all servers, Copper, Fiber, Singlemode, Multimode..
    • (Coming Soon) VPP router, Faster CGNAT, Redundant DNS, etc

  • Community Colocation for Member Servers

    • Server rack for rack-mounted member servers
    • Small spaces for Raspi’s, NUCs, and other small boxes
    • 100M - 10Gbit connections for servers
    • Public IPs, Private/Mesh IPs, BGP available

  • Rooftop

    • Dual 10Gbit to the rooftop
    • Plenty of DC power on the rooftop, 24v, and 56v antennas
    • Three mounting locations (North, West, South) with ample antenna space
    • Pre-run, Pre-terminated wires for quick additional antenna setup

Highlights:

With 40Gbit connectivity, we have set ourselves up for 40x-50x above our previous network capabilities. Yes, 1x40 is 40x higher, not 50x ( We had 1G before ), but our hardware couldn’t quite hit 1G. With the new design we should be able to not only hit higher numbers, but actually achieve them. Not only does this bring a very stable connection to the Supernode, it unlocks many community benefits. We now have the bandwidth to: Host Sites, Support community livestreams, Open Source mirror projects, Firmware build servers and firmware distribution, and anything else we can put our mind to. We should be able to support other major projects by helping to host public resources.
As a tag-along, Pilot has also offered to support us with a 40Gbit at Supernode 1 as well, growing our entire network from 1G total to 81G total, later this year.
Speedtest

Community Colo is a great opportunity for NYC Mesh Members who want to host a server somewhere for personal use and/or development. Rather than hosting at some company for a high monthly price, and also VPNing back to the mesh, and also supporting the mesh via a donation; simply put your server at Supernode 3 instead. We have a server rack and can acquire whatever space and power is needed as we grow. This helps the mesh pay for itself, and fills a need that some of us think is lacking in the area - affordable non-business server space for fun, experimenting, and personal use. We feel this fits under one of our core missions – Providing affordable and accessible community resources open to all with no limits, filtering, “gotcha’s”, contracts, etc. Rack

Our rooftop position is also a major win. Although Supernode 3 is much lower than Supernode 1, and it can’t see most of our major hubs, the huge capability of network and power to the rooftop should allow us to establish hubs around the area in big way, and, in areas we couldn’t previously reach. We can easily add Point To Point antennas to sites that arise without hesitation nor months of planning. Mount One

New VPN Exit for remote nodes. Supernode 1 was struggling to keep up under the load of the remote VPN traffic. The huge new connection offers a new place to bounce the traffic off a Supernode without contention. Latency has already dropped in a major way for many members.

Struggles

The journey wasn’t always easy. Here are some highlights of struggles and lessons along the way.. Of course, it’s not failure unless we give up, and persistence won in the end!

  • Lost our initial major donor. They backed out of funding (for the whole project), just after we signed the lease.
  • Lost our initial internet donor. (10Gbit offer)
  • Power wasn’t available, and was the wrong type. (Waiting, additional purchases)
  • Our account rep left the company.
  • Our rooftop mounting was taken by another company.
  • Rooftop concrete drilling was more challenging, and expensive, and time-consuming, than expected.
  • Weekend rooftop access is challenging at the site.
  • Couldn’t find an internet donor for months, though we were set-up.
  • Had to pay the lease even with no connection and no members connected.
  • 40Gbit-LR optics aren’t too cheap. :)

Major Contributors

Of course all the struggles are good lessons learned. Through them the community pulled together and made it happen.
Big thanks to:

  • Pilot Fiber for a very generous 40Gbit internet donation, saving us many thousands of dollars.
  • Cologuard Brooklyn datacenter for a fair price, and riser and roof access.
  • wizardishungry for donating a Cisco Nexus switch ( 40G/10G ), and servers.
  • Peter for donating two beefy servers, one of which is routing the site.
  • Brooks and Tony for donating early and being the first Community Colo members
  • All who helped with the elevator shaft mounting and datacenter cabling ( Tony, Jonathan, Olivier, Brian, Brooks, Jason, Zach )
  • Jason, and others ( Olivier, Duncan ), for learning and executing the antenna mast mountings; This was a big work including concrete drilling, epoxy, bolts, generators, many drillbits, metal plates, etc.
  • All the cabling and crimping ( Olivier, Logan, Tony, Jason, Zach )
  • Logan for beautifying the wiring.

Join us via Supernode 3 if you’re in the area, or if you’d like to host a server inside.

On Slack the channel #sn3 has tons of information and is a great way to get involed.