Version Selection

Some SDL models which builds use in turn import other models. A good example is std_env which imports versions of different bridges, compilers and other tools, libraries, etc. Such models usually allow the caller to specify overrides as parameters to change the versions of these sub-components which get used. To see how this works, look at:

See also "Transient Package Overrides" in the voverrides(5) man page.

The Problem

The handling of version overrides has been written in SDL code. There are several problems with this:

Version selection is a fairly simple problem. While it is possible to implement it in SDL, you don't need all the flexibility that SDL gives you to solve the problem of version selection.

New version selection

Vesta builds would be easier to configure if the selection of which versions to use (overrides, etc.) was separated from the SDL code that defines the build flow. Removing the selection of versions from the flexibility of a full programming language would make writing tools that understand/generate a configuration much easier.

Most of the key features of how improt-based version selection works should be preserved:

The main thing that should be changed is the use of SDL code to pick the versions to be used. SDL code will obviously still need to get access to the selected versions, but the concept of any variable containing overrides should be eliminated. SDL code should never be checking to see if an override is present and choosing to use it instead of some other version. Instead, we should have simple, well-defined versuib selection semantics so non-SDL programs can understand & manipulate the versions used by a build.

Tricky parts:

SDL still needs some way to access the selected versions of different components. This could be done through a new import-like syntax. Another possibility would be feeding the version selection to the top-level model as its initial dot. Either way, we're creating a new namespace from which the SDL will somehow get the selected versions.

We need to associate the version selection file with the top-level model. (We wouldn't want the user to have to specify both at build time.) Maybe name the version selection file the same as the model, only ending in ".vsel".

Version Selection Files

What might a version selection file look like? It could be essentially the same as the current imports clause of a SDL model. An imports clause defines a hierarchical namespace of variables referring to SDL models from existing immutable versions, and we need the same capability.

There's two significant extensions we'd need:

  1. A method to include another version selection file and start with its namespace as a basis.

  2. A method to alter existing portions of the namespace. Imagine that the variables created by an included version selection file are a binding. Now use ++ to overlay the new imported names on top of that. In fact, maybe each assignment statement should effectively be ++ed into the namespace built up so far.

So, for example, the version selection file in a std_env package might look like this: 

import
  // The std_env model is the build.ves here in the std_env package
  std_env = build.ves;

from /vesta/vestasys.org/bridges import
  // Versions of standard bridges to use
  bridges = [ generics/2,
              c = c_like/4,
              cxx = c_like/4,
              lex/1,
              yacc/1,
              lim/1,
              mtex/1 ];

from /vesta/vestasys.org/libs import
  // Versions of libraries to use
  libs/c = [
    gc/6.4/2,
    z = zlib/1.2.2/1,
  ];

Then an actual package using this might have a version selection file like this: 

include /vesta/vestasys.org/platforms/linux/debian/i386/std_env.sarge/1/top.vsel;

// Use a newer version of the c_like bridge
from /vesta/vestasys.org/bridges import
  bridges = [ c = c_like/checkout/5/3,
              cxx = c_like/checkout/5/3 ];

// Use a newer version of zlib

from /vesta/vestasys.org/libs import
  libs/c/z = zlib/1.2.3/1;

This would overlay versions for three names in the version selection namespace:

  1. bridges/c

  2. bridges/cxx

  3. libs/c/z

Later, some SDL code would access these names and use their contents. All overriding happens in the version selection file, so SDL code doesn't need any concept of override parameters/variables.

Using Selected Versions

How do individual SDL models use the versions selected through this new machanism?

First of all, they would not use the existing imports syntax. That can only be used to refer to specific versions, but what we really need is a way to parameterize an SDL model in terms of the namespace of selected versions. (An SDL model might use normal imports for other models local to the directory where it exists, but it wouldn't use the from syntax of absolute paths for models.)

Probably the best way to do this would be to add some new syntax which allows a model to get imports out of the namespace defined by the version selection file selected when the build starts. This could be an entirely new keyword like this: 

import
  // Local import
  self = build.ves;
use
  // Take a value from the version selection namespace
  std_env;
{
  . = std_env()/env_build();

  return self();
}

Or we could extend the import clause syntax to make the version selection namespace accessible as though it were a subdirectory with a special name, maybe some unused non-alphanumeric chaaracter: 

import
  // Local import
  self = build.ves;
  // Take a value from the version selection namespace
  std_env = @/std_env;
{
  . = std_env()/env_build();

  return self();
}

Either way, the important thing is that the idnvidual SDL model doesn't specify any version for std_env here. It simply uses whatever version the user has selected.

Can we do this without new features?

Would it be possible to set up something that has these properties with the current implementation? Mostly. How would we do so?

What would this look like for the examples above?

A version selection model for std_env: 

import
  // The std_env model is the build.ves here in the std_env package
  std_env = build.ves;

from /vesta/vestasys.org/bridges import
  // Versions of standard bridges to use
  bridges = [ generics/2,
              c = c_like/4,
              cxx = c_like/4,
              lex/1,
              yacc/1,
              lim/1,
              mtex/1 ];

from /vesta/vestasys.org/libs import
  // Versions of libraries to use
  libs = [ c = [
    gc/6.4/2,
    z = zlib/1.2.2/1,
  ] ];
{
  return [ std_env, bridges, libs ];
}

A version selection model for a build which overrides bridges/c, bridges/cxx, and libs/c/z: 

from /vesta/vestasys.org/platforms/linux/debian/i386 import
  // The version selection model from std_env
  std_env.sarge/1/vsel.ves;

// Use a newer version of the c_like bridge
from /vesta/vestasys.org/bridges import
  bridges = [ c = c_like/checkout/5/3,
              cxx = c_like/checkout/5/3 ];

// Use a newer version of zlib

from /vesta/vestasys.org/libs import
  libs = [ c/z = zlib/1.2.3/1 ];
{
  return std_env.sarge() ++ [ bridges, libs ];
}

A client model using this version selection model: 

import
  // The above version selection model
  vsel = lix_i386.vsel.ves;
  // Local import
  self = build.ves;
{
  // Put the selected versions of each component into ./_models_ so that every function we call can get at them
  . = [ _models_ = vsel() ];
  // Preapre the build environment and merge it into dot.  (We're assuming it will leave ./_models_ as-is).
  . ++= ./_models_/std_env()/env_build();

  return self();
}

Problems with this approach:

Discussion