Tasks

A Task may be an objective, goal or Task (or series of Tasks); these terms may be used synonymously.

A Task must define its name; and may define its requirements (other Tasks), a list of steps or Actions to run, input files, and output files.

Each Task has a source folder (or Task Path), and an output folder (or Cache). The Task output path is automatically generated.

Tasks are defined using the task() function.

Source Folder

The Task source folder is the folder of the Makex file in which the Task was defined.

Task Locators

Tasks are referred to using a Task Locator (or “Locator”).

A Task Locator is a Task name, and an optional colon : and path concatenated together. The syntax is:

TaskLocator ← TaskName ( ( ":" Path ) / ":"? ) $
TaskName ← [A-Za-z] [0-9A-Za-z_]*

Examples of Task locators are below:

task_name://path

task_name:path

task_name:

task_name

The path is a folder containing the Makex file defining the Task.

The path may be:

• relative (path/...)

• absolute to the filesystem. starting with one (1) slash. (/path/...).

• absolute to the current Workspace. starting with two (2) slashes. (//path/...).

The path may not contain any parts with current/parent folder operators (. or ..).

The Task name (task_name) must start with an [A-Za-z].

Note

In a previous version of makex, the syntax of locators was path:task_name.

This has since been reversed.

Requirements

Tasks may require other Tasks.

The requires argument to the task() function is used to define the Task requirements list.

For example, the following fragment shows the various ways a task can define these dependencies:

task(
    name="example",
    requires=[
        # File local task, named "task-name"
        "task_name",
        
        # Task in relative path (path/to/task/Makexfile), named "task-name"
        "task_name:path/to/task",
        
        # Task in workspace path (//path/to/task/Makexfile), named "task-name"
        "task_name://path/to/task",
    ]
)

If a requirement changes, the Task requiring it will be considered “stale” or “dirty”. Stale Tasks need to be executed to produce fresh outputs.

A Task with outputs, and without any requirements will always be run.

Task Inputs

A task may define paths to files it requires (the Task’s Input Files).

The inputs argument to the task() function may be used to define these files.

The input files may be a list path names. or it may be a mapping of name to one or more paths (see Naming Input Files). Input files may also be the results of functions that find files.

Paths may be explicit Strings, or they be found using the glob or find functions. For these functions – in the context of the inputs list – paths are relative to the Makexfile.

An example of expressing various forms of paths are below:

task(
    name="example",
    inputs=[
        # absolute path to file
        "/absolute/path/to/file",

        # relative path to file (from the directory of this makex file)
        "file",

        # relative path to file (from the directory of this makex file)
        "path/to/file",

        # absolute workspace path to file 
        "//path/to/file",
    ]
)

Naming Input Files

You may name one file, or list of files.

Input files may be named by passing a mapping to the inputs argument of the task() function. The mapping has keys that are names, and values that may be one or more paths.

Names must be valid identifiers ([a-zA-Z][a-zA-Z0-9_]+ or _).

For example:

task(
    name="example",
    inputs={
        # a single file:
        "name1": "path/to/input1",
        
        # multiple files:
        "name2": [
            "path/to/input2", 
            "path/to/inputs3", 
            ...
        ],
        
        # unnamed files (still accessible by the `_` name)
        "_": ["path/to/unnamed-input"]
    },
    steps=[
        # self.inputs.name1
        # self.inputs.name2
        # self.inputs._
        ...
    ]
)

If you wish to leave files unnamed in the input map, use the empty key ("_").

Actions

A Task may define a list of Actions with the steps argument to the task() function. Actions may be running an executable with arguments (execute()), shell scripts (shell()), or any of the built-in Actions (e.g. print()).

Errors during execution of an Action will cause Makex to stop.

Task Outputs

A Task may define a list of output files. If any files are generated as a result of running the Task, they should be specified in this list. Relative paths in this list are interpreted relative to the Task’s output folder.

If a Task has output files, Makex will use hashes, checksums and caching strategies to see if the Task needs to be run to reproduce output files.

A Task with no defined output files will always be run. If no output files are defined, Makex will be unable to determine if the Task’s outputs are stale; and therefore will always rerun the Task when requested.

Changes to a Task definition, its dependencies, or any of its input files will cause a Task to be executed, reproducing any outputs.

Naming Output Files

Output files may be named by assigning a mapping to the outputs argument of the task() function. You may name one file, or lists of files.

For example:

task(
    name="example",
    outputs={
        "name1": "path/to/output1",
        "name2": ["path/to/output2", "path/to/output3", ...],
    },
    steps=[
        ...
    ],
)

Names must be valid identifiers ([a-zA-Z][a-zA-Z0-9_]+).

Output files explicitly defined as Strings may be referenced using self references.

Output files may contain the result of glob() and find(), however, these are evaluated after the task is run. This means it is impossible to know what the outputs of a task are until the task has been completed.

Self References

Note

NOTE: This feature is experimental and subject to change. See the proposal.

Some of the properties of a task may be referred to within the task itself. This is provided to mitigate duplication and redundancy (and associated errors). These properties may only be used and accessed within a task() definition.

These properties are:

• self.name: The name of the task

• self.path: The output/cache path of the task.

• self.inputs: The inputs mapping of the task.

• self.outputs: The outputs mapping of the task.

For example:

task(
    name="example",
    inputs={
        "input1": "input1.txt" 
    },
    outputs={
        "output1": "output1.txt"
    },
    steps=[
        # Referencing a name in self.inputs and copying input to task output: 
        copy(self.inputs.input1),
        
        # referencing a name in self.outputs writing to task output:
        # will write "example\n" to "output1.txt".
        write(self.outputs.output1, f"{self.name}\n")
    ],
)   

If any named inputs/outputs are defined as a list, accessing self.inputs.name or self.outputs.name will return that list. You may not access members of the list individually (For example, using the index operation self.inputs.example[index]).

You may not reference a named output containing results of glob() or find(). You may not reference a named output assigned to the results of glob() or find().

Optional Requirements

Note

NOTE: This feature is experimental and subject to change. See the proposal.

You may require running Tasks that may have not been defined yet, or may never be defined. This is typically used in a component/module building pattern where modules may or may not require any build steps.

For example:

task(
    name="example",
    requires=[
        optional("optional_task1"),
        optional("optional_task2"),
    ],
    steps=[
        ...
    ],
)

or:

COMPONENTS = [
    "component1",
    "component2",
]

task(
    name="example",
    requires=[
        [optional(f"build:components/{component}") for component in COMPONENTS],
    ],
    steps=[
        ...
    ],
)