Vesta is intended to be used as your revision control and build system, and (everything else held equal) that's the best way to use it. However, sometimes it's advantageous to use a different program (CVS/Perforce/etc.) for revision control while using Vesta as the builder. I encountered this requirement because we had to support a Visual Studio build from the same sources as our Linux build (and there's no direct Vesta support for Windows) and I wanted a way to maintain a Vesta build without requiring an upfront "let's switch everything to Vesta" argument.

I implemented this strategy using Perforce for revision control, but the concepts should be portable to similar CVS-like systems.

Perforce Basics

Perforce is a pretty standard versioning system. You create a 'view', which is a definition of how to map directories and files from the repository into your working directory. You then sync this view, which copies the latest versions of these files from the repository into your working dir. The versioning is file granular, but there's also a notion of changesets, which is like a version number for the entire repository (i.e., whenever a new version of any file is created, the changeset number increments).

You are supposed to issue a specific command to edit a file, doing so allows the system to alert you if someone is also in the act of editing the file and adds the file to your 'pending changelist'. There are also commands to add/delete files which have similar functions. You issue a 'submit' command to promote all the files on your pending changelist

Perforce has history aware merging and a fancy gui for resolving merge conflicts and performing other tasks (reverting to old versions, creating new clients, etc.).

The Basic Idea

The general idea is that you make an empty Vesta package, do a non-exclusive checkout of it, and then create a Perforce view which sits on top of the empty working directory. You then create your Vesta build files (.ves files), adding them using the Perforce commands and checking them into the Perforce repository. At vmake time, the entire working dir will be snapshotted into the Vesta repository and from then on it's just a normal Vesta build.

You never run vcheckin on this working dir. Top of tree always lives in Perforce and Vesta just captures your intermediate builds.

Creating a shadow package

This scheme has a few major problems for any non-trivially sized source tree. The first is that doing a Perforce sync of a large tree into Vesta's file system takes a long time (it takes a long time on any file system, but longer on top of Vesta), and vadvancing a large amount of new data is slow.

The second is that if there's a lot of users all creating identical views in their own working dirs, then to Vesta that will look like a lot of unique directory trees. The files themselves are usually fingerprinted (so Vesta recognizes their redundancy), but not the directories. For large source trees and lots of users, this can threaten Vesta's limit of total unique directories.

And finally, at vadvance time Vesta does some amount of work for each file which has changed since vcheckout, which under this naive scheme leads to an annoying delay on each vmake.

The solution to all these problems is simple but works remarkably well. Instead of using an empty Vesta package to sync a Perforce client to, we make a Vesta package that we keep reasonably up-to-date (within a half-hour or so) with the Perforce repository . To create a new client working dir, we do a nonexclusive checkout of this package, create a Perforce client on top of it, tell Perforce that it matches a given Perforce changeset #, and then just do an incremental sync up to top-of-tree.

This approach makes creating a new client view pretty fast (must faster than using Perforce by itself), and largely eliminates the other problems as long as the user creates a new client every week or so (instead of sync'ing an old client).

I accomplish this with two scripts:

update-top

This script will update the Vesta package such that its latest version mirrors the Perforce repository. The Vesta package uses version numbers which match the Perforce changeset numbers. So basically we just vcheckout -N the current version, create a Perforce client on top of it. The we run a special 'p4 sync -k ...@<changeset_num>' Perforce command. This does a sync, but skips actually copying the files, so all it does is tell the Perforce server that you now have a working dir which matches a given changeset. This is a very fast operation since now files are copied.

Once we do that, then we do a regular 'p4 sync' to update the client to the latest changeset, and then we do a vcheckout -n <new_changeset_num> followed by a vcheckin to complete the operation. My actual script (which I'll attach to this page at some point) does some other stuff to make sure the operation happens atomically so that multiple instances of it can be run at the same time.

Generally the script runs as a cron job every 30 minutes. It takes about 20-30s if there's a new changeset that needs to be mirrored.

make-client

This script is similar to update-top in that it does a non-exclusive checkout of the Vesta mirror package, creates a Perforce client on top of it, does the 'p4 sync -k ...@<changeset_num>' trick, and then a 'p4 sync' to update to the top of tree. At this point, though, it just leaves the working dir there for the user to modify, sync, etc., just as he or she would with a normal Perforce client.

All files left writable

By default, Perforce leaves all files in your client view read-only--this reminds you that you need to 'p4 edit' the file before modifying it. However, Vesta just insists on leaving all files writable on a vcheckout.

I suggest using the 'allwrite' P4 client option, so that the two are compatible. However, this creates an annoying problem that there's nothing to warn you if you edit a file without doing a 'p4 edit'.

The solution to this problem is a little script called chk-client. It is run as a vadvance pre-trigger, and its job is to (a) run update-top, and (b) vdiff the working dir against the latest version of the mirror package, and (c) filter all those differences by the list of opened files provided by the 'p4 opened' command. The net result is that it reports to the user any file in his or her working dir which is different from the top-of-tree version AND which hasn't been explicitly added/edited/deleted.