hidet.option

Classes:

OptionContext()

The option context.

Functions:

dump_options()	Dump the options in option context stack.
restore_options(dumped_options)	Restore the options from dumped options.
current_context()	Get the current option context.
context()	Create a new option context.
set_option(name, value)	Set the value of an option in current option context.
get_option(name)	Get the value of an option in current option context.
bench_config([warmup, number, repeat])	Set the benchmark config of operator tuning.
get_bench_config()	Get the benchmark config of operator tuning.
search_space(space)	Set the schedule search space of tunable operator.
get_search_space()	Get the schedule search space of tunable operator.
cache_operator([enabled])	Whether to cache compiled operator on disk.
get_cache_operator()	Get the option value of whether to cache compiled operator on disk.
cache_dir(new_dir)	Set the directory to store the cache.
get_cache_dir()	Get the directory to store the cache.
parallel_build([enabled])	Whether to build operators in parallel.
get_parallel_build()	Get the option value of whether to build operators in parallel.
parallel_tune([max_parallel_jobs, ...])	Specify the maximum number of parallel compilation jobs to do, and the number of GiB preserved for each job.
get_parallel_tune()	Get the option value of whether to build operators in parallel.
save_lower_ir([enabled])	Whether to save the lower IR.
get_save_lower_ir()	Get the option value of whether to save the lower IR.
debug_cache_tuning([enabled])	Whether to cache the generated kernels during tuning.
debug_show_var_id([enable])	Whether to show the var id in the IR.
runtime_check([enable])	Whether to check shapes and dtypes of all input arguments to compiled Graphs or Tasks.
get_runtime_check()	Get whether to check shapes and dtypes of all input arguments to compiled Graphs or Tasks.
debug_show_verbose_flow_graph([enable])	Whether to show verbose information (like task) when we convert flow graph in to human-readable text.

class hidet.option.OptionContext

The option context.

hidet.option.dump_options()

Dump the options in option context stack.

Returns:

ret – The dumped options.

Return type:

Dict[str, Any]

hidet.option.restore_options(dumped_options)

Restore the options from dumped options.

Parameters:

dumped_options (Dict[str, Any]) – The dumped options.

hidet.option.current_context()

Get the current option context.

To get the value of an option in the current context:

ctx = hidet.option.current_context()
cache_dir: str = ctx.get_option('cache_dir')
cache_operator: bool = ctx.get_option('cache_operator')
...

Returns:

ctx – The current option context.

Return type:

OptionContext

hidet.option.context()

Create a new option context.

To set options in the new context, use the with statement:

with hidet.option.context() as ctx:
    hidet.option.cache_dir('./new_cache_dir')               # set predefined option
    hidet.option.set_option('other_option', 'other_value')  # set a custom option
    ...

Returns:

ctx – The new option context.

Return type:

OptionContext

hidet.option.set_option(name, value)

Set the value of an option in current option context.

The option must be registered before setting via hidet.option.register_option().

Parameters:

• name (str) – The name of the option.

• value (Any) – The value of the option.

hidet.option.get_option(name)

Get the value of an option in current option context.

Parameters:

name (str) – The name of the option.

Returns:

ret – The value of the option.

Return type:

Any

hidet.option.bench_config(warmup=1, number=5, repeat=5)

Set the benchmark config of operator tuning.

To profile a schedule, hidet will run the following code:

for i in range(warmup):
    run()
latency = []
for i in range(repeat):
    synchronize device
    t1 = time()
    for j in range(number):
        run()
    synchronize device
    t2 = time()
    latency.append((t2 - t1) / number)
return median of latency

Thus, there will be total warmup + number * repeat times of execution.

Parameters:

• warmup (int) – The number of warmup runs.

• number (int) – The number of runs in a repeat.

• repeat (int) – The number of repeats.

hidet.option.get_bench_config()

Get the benchmark config of operator tuning.

Returns:

ret – The benchmark config.

Return type:

Tuple[int, int, int]

hidet.option.search_space(space)

Set the schedule search space of tunable operator.

Some operators can be tuned in hidet to achieve the best performance, such as matrix multiplication.

During tuning, different operator schedules will be tried and profiled to get the best one.

We call the space of the tried operator schedule schedule space. There is a trade-off between the tuning time and the operator execution time. If we try more schedules, the tuning process would take longer time, and we are likely to find better schedule.

This function allows user to set the space level that controls the search space we tried.

By convention, we have space level

• 0 for schedule space contains only a single schedule.

• 1 for schedule space contains tens of schedules so that the tuning time will be less than 1 minute.

• 2 for arbitrary large space.

Usage

hidet.search_space(2)

After calling above function, all subsequent compilation would use space level 2, until we call this function again with another space level.

Parameters:

space (int) – The space level to use. Candidates: 0, 1, and 2.

hidet.option.get_search_space()

Get the schedule search space of tunable operator.

Returns:

ret – The schedule space level.

Return type:

int

hidet.option.cache_operator(enabled=True)

Whether to cache compiled operator on disk.

By default, hidet would cache all compiled operator and reuse whenever possible.

If user wants to disable the cache, run

hidet.option.cache_operator(False)

Parameters:

enabled (bool) – Whether to cache the compiled operator.

hidet.option.get_cache_operator()

Get the option value of whether to cache compiled operator on disk.

Returns:

ret – Whether to cache the compiled operator.

Return type:

bool

hidet.option.cache_dir(new_dir)

Set the directory to store the cache.

The default cache directory:

• If the hidet code is in a git repo, the cache will be stored in the repo root: hidet-repo/.hidet_cache.

• Otherwise, the cache will be stored in the user home directory: ~/.hidet/cache.

Parameters:

new_dir (str) – The new directory to store the cache.

hidet.option.get_cache_dir()

Get the directory to store the cache.

Returns:

ret – The directory to store the cache.

Return type:

str

hidet.option.parallel_build(enabled=True)

Whether to build operators in parallel.

Parameters:

enabled (bool) – Whether to build operators in parallel.

hidet.option.get_parallel_build()

Get the option value of whether to build operators in parallel.

Returns:

ret – Whether to build operators in parallel.

Return type:

bool

hidet.option.parallel_tune(max_parallel_jobs=-1, mem_gb_per_job=1.5)

Specify the maximum number of parallel compilation jobs to do, and the number of GiB preserved for each job.

Parameters:

• max_parallel_jobs (int) – The maximum number of parallel jobs allowed, default -1 (the number of available vcpu returned by os.cpu_count()).

• mem_gb_per_job (float) – The minimum amount of memory (in GiB) reserved for each tuning job, default 1.5GiB.

hidet.option.get_parallel_tune()

Get the option value of whether to build operators in parallel.

Returns:

ret – Get the maximum number of jobs and minumum amount of memory reserved for tuning.

Return type:

Tuple[int, float]

hidet.option.save_lower_ir(enabled=True)

Whether to save the lower IR.

Parameters:

enabled (bool) – Whether to save the lower IR.

hidet.option.get_save_lower_ir()

Get the option value of whether to save the lower IR.

Return type:

bool

hidet.option.debug_cache_tuning(enabled=True)

Whether to cache the generated kernels during tuning.

Note

This option is only used for debugging purpose. It will generate a lot of files in the cache directory and take a lot of disk space.

Parameters:

enabled (bool) – Whether to debug cache tuning.

hidet.option.debug_show_var_id(enable=True)

Whether to show the var id in the IR.

When this option is enabled, the IR will show the var id with the format var@id, like x@1 and d_1@1732. Variable (i.e., hidet.ir.Var) a and b is the same var if and only if a is b evaluates to True in Python).

Parameters:

enable (bool) – Whether to show the var id in the IR.

hidet.option.runtime_check(enable=True)

Whether to check shapes and dtypes of all input arguments to compiled Graphs or Tasks.

Parameters:

enable (bool) – Whether to check shapes and dtypes of all input arguments to compiled Graphs or Tasks.

hidet.option.get_runtime_check()

Get whether to check shapes and dtypes of all input arguments to compiled Graphs or Tasks.

Returns:

ret – Get whether to check shapes and dtypes of all input arguments to compiled Graphs or Tasks.

Return type:

bool

hidet.option.debug_show_verbose_flow_graph(enable=True)

Whether to show verbose information (like task) when we convert flow graph in to human-readable text.

Parameters:

enable (bool) – Whether to show verbose information when we convert flow graph in to human-readable text.