vestasys.org Network Architecture

This document describes details of the vestasys.org network and established methods which allow authorized users to connect other networks to it. The goals of the configuration described are:

1. Protecting the repository that has mastership of the Vesta sources. Secure authentication in the repository has not yet been implemented, and we have not performed any security auditing of the repository's code. For the time being, we're going to err on the side of caution and rely on traditional firewalling and VPN techniques to restrict access to secure channels for authenticated clients. A separate repository (with mastership of nothing) will be used for allowing anyone on the Internet to replicate new versions of the Vesta source code.
2. Enabling continued collaboration. While we want to protect the repository, it must be possible for contributors to access it and perform remote checkouts to and checkins from their local repositories running at other locations.
3. Protecting contributors from each other. Allowing remote checkout/checkin means that the central repository must be able to contact the repositories running at participating sites where development takes place. However, the access should be limited to that which is necessary to facilitate collaboration work on Vesta itself. It must not provide any means by which malicious users could gain unauthorized access to participating sites.

VPN Connections

The method for connecting to the central repository and other services behind the vestasys.org firewall is essentially what's described in the Linux VPN HOWTO: PPP over ssh. This method was chosen for several reasons:

• Linux can be used for both ends of the connection. Linux is fairly ubiquitous. We don't intend to exclude other platforms, and this method can probably be adapted to others. If desired, Linux can simply be used for routing and mangling packets.
• The neccessary software is often already installed. Most distributions include the ssh client in the default install. pppd is also commonly installed.
• It's easier to set up than IPSec. IPSec may be a cleaner solution overall, but the major implementation (FreeS/WAN) requires building a custom kernel after patching the source.
• It's more secure than PPTP.

We provide a standard script for initiating a VPN connection to vestasys.org: vestasys-vpn. It has its own man page. Anyone wishing to connect to vestasys.org should send e-mail to vpn-admin@vestasys.org.

IP Address Space Allocation

The vestasys.org Intranet uses the class A private range address space (10.0.0.0/8).

All interfaces on the vestasys.org physical network use addresses within 10.86.0.0/24. (86 is ASCII 'V'.)

Each contributor needing access to the central repository will be allocated an 8-bit (class C size) address range. Initially these allocations will have the same first two bytes (10.86.1.0/24, 10.86.2.0/24, ...).

Before anyone asks, no, we do not yet have a plan for migration to IPv6.

Network Address Translation

Because vestasys.org uses a private range address space (10.0.0.0/8) and we intend to allow contributors to connect other networks which may use the same private range address space, network address translation is a necessity. The NAT duties are distributed such that each endpoint of the VPN connection only needs to perform translations for the hosts on its local network.

Contributor sites should address the vestasys.org machines by their published IP addresses. (We assume that those addresses will not be co-opted by others on their private networks.) Packets coming to vestasys.org over a VPN connection get their destinations translated from the public addresses to the corresponding private range ones. Packets leaving vestasys.org via a VPN connection get their sources translated from the private addresses to the public ones.

vestasys.org machines will address contributor sites by addresses within their assigned address ranges. (See "IP Address Space Allocation" above.) The contributor's VPN endpoint is responsible for handling appropriate source and destination NAT rules. Any repositories at the client site will need to be assigned a specific IP address. (See "Repository Name Resolution" below.) Other hosts can be handled with many-to-one or many-to-many SNAT. (See the Netfilter/iptables FAQ for a few words on many-to-many SNAT.)

Thus, each end of a VPN connection is only responsible for translating the addresses of the machines on the LAN to which it is directly connected. The end at vestasys.org translates between the private and public range addresses for the vestasys.org hosts. The end at a contributor site translates between the local addresses of clients and addresses in the contributor's assigned IP address range.

NAT Example

Suppose there is a contributor site which has a client host addressed locally as 192.168.1.10. This address is translated by one-to-one SNAT at the client VPN endpoint to 10.86.1.2. Below is a table showing the addresses used during a communication between this client and the central repository server (dev.vestasys.org) on the client network, over the VPN ppp link, and on the vestasys.org network.

Routing

Contributor sites should route packets to the central repository machine (dev.vestasys.org) over their VPN link. (Other machines accessible this way may be added in the future.) Contributor sites must not route packets to the vestasys.org firewall (wall.vestasys.org) over the VPN connection, as it is the endpoint for that connection and must therefore be reachable by another route.

vestasys.org will route any packets for a conrtibutor's IP address range over their VPN link (when it's active). (See "IP Address Space Allocation" above.)

Packet Filtering

The packet filtering at vestasys.org implements these policies:

• Any packets coming off of a VPN connection and destined somewhere other than the physical local LAN will be dropped. This prevents one contributor's network from connecting to that of another simultaneously connected contributor. It also prevents packets from escaping out onto the Internet at large.
• Any packets from a VPN connection destined for an IP address other than the central repository server will be dropped.
• Certain ICMP types will be allowed. (Secifically, types 3, 0, 8, and 11, which correspond to "destination unreachable", "echo reply", "echo request", and "time exceeded".)
• New connection attempts to ports other than 22 (ssh), 23 (telnet), and unpriviledged ports (>1023) will be dropped. Once the keep-alive fix is implemented in the SRPC library (eliminating backwards-pointing pinger connections on arbitrary unpriviledged ports), this will be further restricted to ports 22, 23, and 21776 (the repository's SRPC port).

An individual contributor's VPN endpoint may want to add their own packet filtering rules. The suggested packet filtering policies are outlined below:

• Any packets coming off the VPN interface destined for hosts other than the set of client hosts allowed to use VPN connection should be dropped. (Such hosts can usually be recognized by matching against an IP address range, such as 192.168.1.0/24.) Similarly, any packets heading for the VPN interface coming from hosts other than the authorized set of clients should be dropped.
• Packets coming from the VPN interface with a source address other than that of the central repository (dev.vestasys.org) may be droppped.
• Certain ICMP types should be allowed going to and coming from the VPN interface. (Definitely type 3, which is "destination unreachable", and probably also types 0, 8, and 11, which are to "echo reply", "echo request", and "time exceeded".)
• Packets in already established connections should be allowed.
• New connections from the VPN interface to the local repository machine on unpriviledged ports should be allowed. (Once the keep-alive fix is implemented in the SRPC library this can be restrictied to just the repository's SRPC port, which is usually port 21776.) This is neccessary to allow remote checkout/checkin of source code from/to the central repoisitory, as well as mastership transfer and replication.

We also provide an example script for setting up contributor-site filtering rules similar to those outlined above.

Repository Name Resolution

Vesta repositories are referred to by their hostnames. Because of this, any tool which needs to contact a repository (including other repositories for purposes of replication, remote checkout, and mastership transfer) must be able to resolve its name.

For our purposes this is not straightforward, because the hostnames of machines running participating repositories are often unpublished (because they are themselves on private networks behind firewalls). It's further complicated by the fact that the addresses by which the central repository should contact such repositories are not the same as their addresses they are known by on their own local networks. (See "Network Address Translation" above.)

Our current solution is simply to add entries to the /etc/hosts file on the central repository machine for each contributing site's repository (possibly obscuring it's normal IP). This is a bit of a kludge, but it works.

As an example, suppose there is a participating site at foo.com. The Vesta repository at foo.com is bar.foo.com. Their assigned address range is 10.86.13.0/24. Their repository has its address translated to 10.86.13.7 on the vestasys.org network. To facilitate their participation, the central repository is given a host table entry saying that bar.foo.com has the address 10.86.13.7. Then, when the central repository needs to contact the repository at bar.foo.com (for example, when replicating in a newly checked-in version of one of the core Vesta packages), it gets the correct IP address.

Repository Exports and Realm Names

Assigning each contributing site its own IP address range simplifies the matter of writing the repository exports. Each site needs just one line in the central repository's exports file allowing all hosts in its network range to connect using its chosen realm name.

For example, if foo.com is assigned 10.86.13.0/24, their line in the repository export file on dev.vestasys.org would be:

10.86.13.0/24: allow global foo.com

See the "Access Control" section of the repository(8) man page for information on realm names and the format of the repository's export file.