Tasks

A Task may be an objective, goal or task (or series of tasks); these terms may be used synonymously.

A Task must define its name; and may define its output path, requirements (files and other Tasks), a list of things to run (steps/actions), and its outputs (files).

Each task has an output path; and an input path (the folder of the Makex file in which the task was defined). The task output path is automatically generated, unless it is specified when defining a task.

Tasks are defined using the task() function.

Task Paths

Tasks have a source path (the Task Source Path) and and output or cache path (the Task Output Path).

The Task source path is folder of the Makex file in which the task is was defined.

The Task output path is automatically generated (though it may be specified per task).

Task Requirements

Tasks may require other tasks and files (including files produced from other tasks).

The task(requires=[]) argument is used to define this list.

For example, the following fragment shows the various ways a task can define these dependencies:

task(
    name="example",
    requires=[
        # absolute path to file
        "/absolute/path/to/file",
        
        # relative path to file (from the directory of this makex file)
        "file",

        # relative path to file (from the directory of this makex file)
        "path/to/file",

        # absolute workspace path to file 
        "//path/to/file",
        
        # File local task, named "task-name" (shorthand)
        ":task-name",
        
        # Task in relative path (path/to/task/Makexfile), named "task-name" (shorthand)
        "path/to/task:task-name",
        
        # Task in workspace path (//path/to/task/Makexfile), named "task-name" (shorthand)
        "//path/to/task:task-name",
    ]
)

If a requirement changes, the Task requiring it will be considered “stale” or “dirty”. Stale Tasks need to be executed to produce fresh outputs.

A Task with outputs, and without any requirements will always be run.

Optional Requirements

Note

NOTE: This feature is experimental and subject to change. See the proposal.

You may require running tasks that may have not been defined yet, or may never be defined. This is typically used in a component/module building pattern where modules may or may not require any build steps.

For example:

task(
    name="example",
    requires=[
        optional(":optional_task1"),
        optional(":optional_task2"),
    ],
    steps=[
        ...
    ],
)

or:

COMPONENTS = [
    "component1",
    "component2",
]

task(
    name="example",
    requires=[
        [optional(f"components/{component}:build") for component in COMPONENTS],
    ],
    steps=[
        ...
    ],
)

Input Files

Input files are any of the Tasks requirements which refer to files.

Additionally, if the task refers to any files generated by other Tasks, those are also input files.

Naming Input Files

Note

NOTE: This feature is experimental and subject to change. See the proposal.

Input files may be named by passing a mapping to the inputs argument of a Task. You may name one file, or list of files.

Names must be valid identifiers ([a-zA-Z][a-zA-Z0-9_]+).

For example:

task(
    name="example",
    inputs={
        "name1": "path/to/output1",
        "name2": ["path/to/output2", "path/to/output3", ...],
    },
    steps=[
        # self.inputs.name1
        # self.inputs.name2
        ...
    ]
)

Actions

A Task may define a list of “Actions” with the task(steps=[]) argument. Actions may be shell scripts, programs/executables and their arguments, or any of the built-in actions.

The list of things to run is executed in order. Any errors during execution will cause Makex to stop.

Outputs

A Task may define a list of output files. If any files are generated as part of running the Task are used, they should be specified. The output files should be specified with paths relative to the Task’s output path.

If a Task has outputs, Makex will use hashes, checksums and caching strategies to see if the Task needs to be run to reproduce the outputs.

A Task with no defined output files will always be run. If no output files are defined, Makex will be unable to determine if the Task’s outputs are stale; and therefore will always rerun the Task when requested.

Changes to a Task definition, its dependencies, or any of its input files will cause a Task to be executed, reproducing any outputs.

Naming Output Files

Note

NOTE: This feature is experimental and subject to change. See the proposal.

Output files may be named by passing a mapping to the outputs argument of a Task. You may name one file, or lists of files.

For example:

task(
    name="example",
    outputs={
        "name1": "path/to/output1",
        "name2": ["path/to/output2", "path/to/output3", ...],
    },
    steps=[
        ...
    ],
)

Names must be valid identifiers ([a-zA-Z][a-zA-Z0-9_]+).

Task Locators

Tasks are referred to using a Task Locator (or “Locator”).

A Task Locator is a name and optional path concatenated together:

{path}:{name}

The path may be:

• relative ({path})

• absolute to the filesystem. starting with one (1) slash. (/{path}).

• absolute to the current workspace. starting with two (2) slashes. (//{path}).

The path may not contain any parts with current/parent folder operators (. or ..).

The name must start with an [a-zA-Z] and may use [a-zA-Z0-9_\-] after that.

Task Self References

Note

NOTE: This feature is experimental and subject to change. See the proposal.

Some of the properties of a task may be referred to within the task itself. This is provided to mitigate duplication and redundancy (and associated errors). These properties may only be used and accessed within a task() definition.

These properties are:

• self.name: The name of the task

• self.path: The output/cache path of the task.

• self.inputs: The inputs mapping of the task.

• self.outputs: The outputs mapping of the task.

For example:

task(
    name="example",
    inputs={
        "input1": "input1.txt" 
    },
    outputs={
        "output1": "output1.txt"
    },
    steps=[
        # Referencing a name in self.inputs and copying input to task output: 
        copy(self.inputs.input1),
        
        # referencing a name in self.outputs writing to task output:
        # will write "test" to "output1.txt".
        write(self.outputs.output1, self.name)
    ],
)   

If any named inputs/outputs are defined as a list, accessing self.inputs.name or self.outputs.name will return that list. You may not access members of the list individually (For example, using the index operation self.inputs.example[index]).