• ReposUIPackageClass

ScottVenier and KenSchalk were talking about the possibility of adding a class to ReposUI to represent a package. This would provide some higher-level operations than VestaSource and wouldm make it easier to write applications like VestaWeb (e.g. a GUI). This page contians some notes on how we could do this.

Classes

We probably need several classes:

• Package: represents a package

• Version: represents an individual version; could be in a package, branch, or session directory

• Branch: derived from Package

• Session: represents a session directory

• Reservation: represents a version reservation stub

• VersionContainer is simply a base class for both Package and Session

• WorkingDir: represents a working copy (a mutable direcotry under /vesta-work)

Things that we don't need more classes for:

• Contents of versions (files, sub-directories): VestaSource is good enough

• Ghosts: a derived class wouldn't have anything to add, so just use VestaSource

• Latest link: it's not a Reservation but it is a stub

• Generic mutable directories (/vesta-work/jnsmith): again, VestaSource is fine

• Generic appendable directory (/vesta/vestasys.org): use VestaSource

• Volatile directories: only the evaluator uses these

Base class

We probably want a base class for all of these. Maybe this should be VDirSurrogate, or maybe it should be a new class.

• findMaster: finds the master repository for this package. returns another object of the same type which refers to the master copy.

  • See ReposUI::filenameToMasterVS

• remote: returns a boolean indicating whether this object refers to the local repository or a remote repository

• isPackage/isBranch/...: determines whether this is one of the other types

  • Does this simply use RTTI, or VestaSource tests like inAttribs("type", "package")?

• castToPackage/castToBranch/...: returns an object of the requested type

• pathname: returns a Text of the pathname, similar to ReposUI::vsToFilename().

VersionContainer class

Abstract base class (should never actually be instantiated). It's concrete sub-classes are Package, Branch, and Session.

Member functions:

• versions: returns all the Version objects in the package

• ancestors: returns the Version representing what this collection of versions is based on (i.e. the old-version attribute (if any) and/or ReposUI::prevVersion of version 0)

• latest: returns the Version representing the highest version in the package

  • See ReposUI::highver

Other notes:

• Some way to find ghosts?

Session class

Derived from VersionContainer and adds these additional member functions:

• newVersion: returns the new-version of the session directory, if any

  • Returns a Version if the session has already been ended by checking in

  • Returns a Reservation if the session is still active

• exclusive: returns a boolean indicating whether this was an exclusive checkout (i.e. whether it has a non-empty new-version attribute)

• workingDir(goToMaster=false): returns the WorkingDir associated with the session

  • goToMaster=false means find the working directory in the same repository, goToMaster=true means find the working directory in whatever repository it's in

  • This is a strange case, but we may want a way to differentiate between a session which never had a working directory (i.e. was created with vcheckout -W) from a session which had a working directory but doesn't have one any more (i.e. a session closed by checkin or removal of its working copy).

• ownedBy: returns the owner of this session as a text string

• ownedByMe: determines whether this session is owned by the user running the program

Package class

Derived from VersionContainer and adds these additional member functions:

• branches: returns all the Branch objects contained in the package

• sessions: returns all the Session objects contained in the package

• reservations: returns all the Reservation objects contained in the package

• isCheckedOut: returns a boolean indicating whether the package is currently checked out (i.e. would vcheckout fail with "already checked out")?

Branch class

Derived from Package. Does it add anything?

We think we'll keep this because branches may become more interesting in the future. (See vupdateandbranches.)

Version class

• session: returns the Session object for the session-dir of this version (if any)

• content: returns the Version object for the content of this version (if any)

• nextBranches: returns all the Branch objects derived from this version

  • Should use the next-branches attribute if present, but should also look for branches in the same package

• nextSessions: returns all the Session objects derived from this version

  • Should use the next-sessions attribute if present, but should also look for sessions in the same package

• nextVersions: returns all the Version objects derived from this version

  • Should use the next-versions attribute if present, but should also look for versions in the same package

• parent: returns the Session, Branch, or Package object which is the parent of this version

• ancestors: returns some kind of list of old-version, ReposUI::prevVersion, and any merged-with attributes.

• ?diff: takes some other Version and returns the diff against it. A few issues surrounding this are discussed on DiffIssues.

  • Note: VestaSource::list with deltaOnly=true will provide the file/directory changes over the basis version. We could provide something like this which just tells the caller about the changes on a whole file level without getting into what's discussed on DiffIssues.

• equalTo/operator==: determines whether two versions are identical

  • Should return true for a pair of Version objects that are in different repositories but have the same path

  • Should return true for a version that refer to the same immutable contents (i.e. have the same fingerprint) such as a version in a package and the snapshot it was created from

• equalContents: returns true if this version has the exact same contents as another version, even if they occured at different points in history

• inAncestryOf(otherVersion): returns true if this version appears anywhere in the full ancestry graph of otherVersion

  • Conceptually >

• higherThan(otherVersion): returns true if this version seems to be later than another version based on a simple name comparison

  • pkg/N is higherThan pkg/N-1
  • pkg/checkout/N/M is higherThan pkg/N-1
  • pkg/N.foo/M is higherThan pkg/N

  • Should use the VestaWeb compare_versions when they're in the same package

    • This probably won't work with the word "checkout" (or "scribble")

WorkingDir class

Member functions:

• session: returns the Session object for the session-dir of this working copy (if any)

• oldVersion: returns the Version representing the old-version of this working copy (if any)

• newVersion: returns the Reservation representing the new-version of this working copy (if any)

• exclusive: returns a boolean indicating whether this was an exclusive checkout (i.e. whether it has a non-empty new-version attribute)

• lastSnapshot: returns the Version representing the latest snapshot in the associated session directory

• needSnapshot: returns a boolean indicating whether the working copy has been changed since the last snapshot (i.e. whether vadvance would take a new snapshot)

Reservation class

Member functions:

• session: returns the Session object for the session-dir of this reservation (if any)

• workingDir(goToMaster=false): returns the WorkingDir associated with the reservation (if any)

• parent: returns the Branch or Package object which is the parent of this reservation

• ownedBy: returns the global name of the owner of this reservation (i.e. the checkout-by attribute)

• ownedByMe: determines whether this reservation is owned by the user running the program

Related Functions

There should be a function to give a canonical sorted order of the versions and version reservations within a package/branch/session (VersionContainer) and it's contents(?). This should include any ghosted versions as well with some indication that they are ghosts.

What about methods to perform checkout, checkin etc.?

We've talked before about the idea of turning the command-line tools (vcheckout, vcheckin, vbranch, etc.) into call-able API functions. However, this not at all a simple task for several reasons:

• The caller would have to provide some mechanism to specify how messages and errors should be presented to the user. Ideally this should be flexible enough to allow a GUI to redirect all such output. Currently the code just sends all this to standard output/standard error as plain text which isn't good enough for a callable version.
• Many of these commands have an interactive component (message entry). Again, the caller would have to be able to specify how user interaction would be handled and we'd want this to be flexible enough to allow for a GUI to present questions in a natural way.
• The commands can be a multi-step process with several long-running pieces. (For example, vcheckout can involve up to 4 different repository servers.) The caller should have the option to provide user feedback that a long-running operation is in progress, and should be able to present output about each step as it is completed. This further complicates the API.

• The current implementation of these operations probably has resource leaks, since they're short-running programs. We should use valgrind on the command-line tools first to at least eliminate any memory leaks or other memory problems.

• Soon we'll have triggers for repository changes. Those are external programs which may expect to print information to the user and/or interact with them, all through their standard input/output/error streams. To be completely general, this probably means that a GUI would have to provide some sort of a simple terminal interface for the triggers to interact with the user.

We do believe that having an API for performing the operations is desirable, but because of these complications we don't think it should be a requirement for the first version of the interface which this page is about. Simply providing information about the revision control objects in a meaningful way with common code is a good first step.

Other notes

• These classes all represent objects in the repository. They should be derived from the VestaSource class. (Really derived from VDirSurrogate.)

  • It's worth noting that a VestaSource object is specific to one repository. That means that each of these objects would be too. Some programs may want to create some of these objects referring to other repositories (e.g. to inspect the master copy of a package).

• This should, of course, all become part of the interfaces wrapped with SWIG

• How should multiple objects (package versions/branches/sessions) be returned? A callback like VestaSource::list? An array/Sequence of all of them?

  • Callbacks are probably no an option across SWIG, which suggests that such functions should just return a list of all the objects.

• Some of these functions can return multiple different possible types (e.g. Version::parent). In C++ we could handle this with run-time type identification, but handling this with the scripting languages is an important implementation detail.

• The multiplicity of functions for answering the question "what is this version derived from" (oldVersion, prevVersion, ancestors) is a bit annoying. This is due in part due to a bug in vcheckin which sets old-version incorrectly, and in part to the unknowns around for merges will be recorded. It would be nice if we could simplify this. Maybe oldVersion is neccessary and prevVersion could be used in all cases?

• We'll need some sort of test suite for these new classes.

• What are the constructor arguments for these obejcts? A path? A VestaSource? Optionally a specific repository server host:port? What if you want information from the master (ReposUI::filenameToMasterVS)? What if you just want any copy (ReposUI::filenameToRealVS)?

  • ScottWeiland asks: What if I want to find the current working dir of the latest checkout of a package that is not mastered here? I can get the master info, but then how can I pass that into the api to acquire the session dir that could be remotely located?

  • All of the above options. In some cases, each of them makes sense.

Development Plan

Not important for the initial version:

• Version::diff: too complicated, too many unknowns

Tricky Issues

Use iterator classes for looping over multiple objects (Package::versions, Version::ancestors, etc.). We think this will work reasonable well across SWIG. These should be lazy iterators (not doing any work until they have to.)

Use type identification functions on base class to handle functions that return one of N different types.

Existing Implementations

• VestaWeb includes implementations of some pieces of this.

  • Released version in straight Perl

  • Re-write using the SWIG interface

• The old Arana-era control panel (which has never been released but could be under the LGPL agreement) included implementations of some of this written in incr Tcl using a much older SWIG interface to some of the Vesta APIs. I've attached the two interesting files:

  • vesta.itcl : most similar to what's described above

  • vesta-gui.itk : includes support for creating a menu for selecting one of several versions

ReposUIPackage Implementation and Concerns

Current version of ReposUIPackage: repos_ui/36.ReposUIPackage/3. It also uses repos/62.ReposUIPackage/1 and basics/os/21.common_parent/1

Classes hierarchy

Some implementation concerns that were brought up previously and theirs current solutions

• Classes are derived from V!DirSurrogate.

  • Problem description: It has been decided that SourceBase class should inherit from V!DirSurrogate so higher level interface objects will inherit V!DirSurrogate functionality. In that case the V!DirSurrogate constructor is called before we know the VestaSource (V!DirSurrogate) which this new higher level object is going to represent. Maybe we should define copy constructor in V!DirSurrogate, call it by SourceBase constructors with ReposUI::filenameToMasterVS or ReposUI::filenameToRealVS as parameter, but there is another tricky issue of freeing the object returned by those functions. Another possibility is to have V!DirSurrogate as a member variable in SourceBase class. In that case we’ll need wrappers around V!DirSurrogate functions in SourceBase class which brings up the maintenance problem.( tracker entry)

  • Current approach: VDirSurrogate::copy() function is used.

  • (KenSchalk): VDirSurrogate::copy() should be pretected. It might also be a good idea to have a protected default constructor (one that takes no arguments) for VDirSurrogate which initializes it to a known invalid state

• isPackage/isBranch/...

  • Current approach: check SourceBase::source_kind member variable.

  • (KenSchalk): Consider using dynamic_cast here. It might make SourceBase::source_kind unnecessary. (Here's a page that briefly describes dynamic_cast and other c++ type casting features.)

• castToPackage/castToBranch...

  • Current approach: performs kind chek by calling corresponded is<Kind> function and then performs casting. Returns NULL if is<Kind> returned false.

  • (KenSchalk): Again, consider using dynamic_cast.

• ownedBy/ownedByMe

  • Current approach: simply reads attributes

  • Better solution: should use new AccessCheck SRPC calls once they are implemented.

• How should multiple objects (package versions/branches/sessions) be returned?

  • Current approach: Enumerators

  • Enumerators that inherit from ContainerEnum (BranchesEnum, VersionsEnum, SessionEnum, ReservationsEnum and GhostsEnum) enumerate only over local objects.

BranchesEnum* Package::branches()
VersionsEnum* VersionContainer::versions()
SessionsEnum* Package::sessions()
ReservationsEnum* Package::reservations()
GhostsEnum* VersionContainer::ghosts()

• AncestorsEnum performs lookup localy and in repositories provided by hints.

AncestorsEnum* VersionContainer::ancestors(Text hints = "")
AncestorsEnum* Version::ancestors(Text hints = "")

• Enumerators that inherit from EnumBase (NextBranchesEnum, NextVersionsEnum, NextSessionsEnum) enumerate over container entries first and perform attributes based lookups next. If object specified by attribute does not exist localy enum should try to find it in repositories specified by hints.

NextBranchesEnum* Version::nextBranches()
NextVersionsEnum* Version::nextVersions()
NextSessionsEnum* Version::nextSessions()

• Some way to find ghosts?

  • Current approach: GhostEnum* VersionsContainer::ghosts()

• The way to differentiate between a session which never had a working directory (i.e. was created with vcheckout -W) from a session which had a working directory but doesn't have one any more (i.e. a session closed by vcheckin or removal of its working copy)

  • Current approach:

// hasWorkDir: returns true if work-dir attribute is set, false otherwise
// (case vcheckout -W)
bool hasWorkDirAttrib() throw (SRPC::failure)

// workingDir: returns WorkingDir object if it's an active session, and 
// NULL if session is not active or work-dir attribute is not set. 
// if go_to_master is false it looks for working directory in the same 
// repository, if go_to_master is true it looks for the working directory 
// in other repositories. 
WorkingDir* workingDir(bool go_to_master = false)
      throw (ReposUI::failure, VestaConfig::failure, SRPC::failure);

• (KenSchalk): The description of workingDir seems a little strange. There is only ever one repository in which the working directory for a session directory exists. Saying that it "looks ... in other repositories" makes it sound like it's searching multiple repositories which doesn't make sense.

• Fixed version:

   // hasWorkDir: returns true if work-dir attribute is set, false otherwise
    // (case vcheckout -W)
    bool hasWorkDirAttrib() throw (SRPC::failure)
    { return isAttribSet("work-dir"); }
    
    // workingDir: returns WorkingDir object if it's an active session, and 
    // NULL if session is not active or work-dir attribute is not set. 
    // If go_to_master is false the work dir is only looked up in local 
    // repository. if go_to_master is true then master copy of the 
    // session is looked up and working dir is looked up in the repository 
    // where session's master copy was found.   
    WorkingDir* workingDir(bool go_to_master = false)
      throw (ReposUI::failure, VestaConfig::failure, SRPC::failure);

• sortContents:

  • Current approach: Not only versions, reservations and ghosts, but also branches and packages are sorted (VersionContainer::sortContents).

  • (KenSchalk): As far as I can see, "sortContents" doesn't occur in repos_ui/36.ReposUIPackage/3

SWIG concerns

• Cases where the scripting language has two different pointers to the same heap-allocated C++ object may be a problem for memory management. Specifically it could result in a double-free or a use-after-free if the scripting language doesn't know that the two pointers are the same. For example, using Package::sortContents or Session::NewVersion and then using the casting functions (castToVersion, castToBranch, castToReservation) could create this situation.

  • We're pretty sure this is a problem in Perl, it may not be in Tcl where objects must be explicitly destroyed. We're not sure about Python and Java though we expect problems there as well.
  • Instead of the casting functions returning the same object, we could have them copy, but this would be bad for C++ so maybe we should do it in the SWIG wrappers. We might want to change the names of the functions to clarify that they're doing more than just casting to a different class.