API Reference

NASBenchAPI: Unified APIs for NASBench-101/201/301.

class nasbenchapi.NASBench101[source]

Unified NB101 API wrapper.

Delegates all operations to the underlying NASBench101 implementation.

__init__(data_path=None, verbose=True)[source]
Parameters:

  • data_path (str | None)

  • verbose (bool)

available_budgets(dataset=None, split=None)[source]

NB101 does not track explicit training budgets.

Return type:

Optional[List[Any]]

Parameters:

  • dataset (str | None)

  • split (str | None)

bench_name()[source]

Short benchmark name.

Return type:

str

datasets()[source]

Available datasets.

Return type:

List[str]

decode(encoding)[source]

Decode architecture from native encoding.

Return type:

Arch101

Parameters:

encoding (Dict[str, str])

encode(arch)[source]

Encode architecture to native format.

Return type:

Dict[str, str]

Parameters:

arch (Arch101)

id(arch)[source]

Stable identifier for architecture.

Return type:

str

Parameters:

arch (Arch101)

is_valid(arch)[source]

Check if architecture is valid.

Return type:

bool

Parameters:

arch (Arch101)

iter_all()[source]

Iterate all NB101 architectures.

Return type:

Iterator[Arch101]

load(data_path=None)[source]

Load NB101 from the provided path or environment.

Return type:

NASBench101

Parameters:

data_path (str | None)

mutate(arch, rng, kind=None)[source]

Mutate architecture.

Return type:

Arch101

Parameters:
op_set()[source]

Operations available in NB101 cell.

Return type:

List[str]

query(arch, dataset='cifar10', split='val', seed=None, budget=None, average=False, summary=False)[source]

Query performance metrics for architecture.

Return type:

Any

Parameters:
random_sample(n=1, seed=None)[source]

Random sample from loaded NB101 architectures.

Return type:

List[Arch101]

Parameters:
splits(dataset)[source]

Supported splits.

Return type:

List[str]

Parameters:

dataset (str)

train_time(arch, dataset='cifar10')[source]

Get training time for architecture.

Return type:

Optional[float]

Parameters:
class nasbenchapi.NASBench201[source]

Unified NB201 API wrapper.

Delegates all operations to the underlying NASBench201 implementation.

__init__(data_path=None, verbose=True)[source]
Parameters:

  • data_path (str | None)

  • verbose (bool)

arch_str_to_index(arch_str)[source]

Convert NB201 arch string to canonical index (0..15624).

Return type:

int

Parameters:

arch_str (str)

available_budgets(dataset=None, split=None)[source]

Return available training budgets for NB201.

Return type:

Optional[List[int]]

Parameters:

  • dataset (str | None)

  • split (str | None)

bench_name()[source]

Short benchmark name.

Return type:

str

datasets()[source]

Available datasets.

Return type:

List[str]

decode(encoding)[source]

Decode architecture (pass-through for now).

Return type:

Any

Parameters:

encoding (Any)

encode(arch)[source]

Encode architecture (pass-through for now).

Return type:

Any

Parameters:

arch (Any)

id(arch)[source]

Stable identifier for architecture.

Return type:

str

Parameters:

arch (Any)

index_to_arch_str(idx)[source]

Convert NB201 index (0..15624) to arch string.

Return type:

str

Parameters:

idx (int)

iter_all()[source]

Iterate all NB201 architectures.

Return type:

Iterator[Any]

load(data_path=None)[source]

Load NB201 from the provided path or environment.

Return type:

NASBench201

Parameters:

data_path (str | None)

mutate(arch, rng, kind=None)[source]

Mutate architecture (no-op for now).

Return type:

Any

Parameters:
query(arch, dataset='cifar10', split='val', seed=None, budget=None)[source]

Query performance metrics for architecture.

Return type:

Dict[str, Any]

Parameters:

  • arch (Any)

  • dataset (str)

  • split (str)

  • seed (int | None)

  • budget (Any | None)

random_sample(n=1, seed=None)[source]

Random sample from NB201.

Return type:

List[Any]

Parameters:
random_sample_str(n=1, seed=None)[source]

Random sample NB201 architectures as arch strings.

Return type:

List[str]

Parameters:
splits(dataset)[source]

Supported splits.

Return type:

List[str]

Parameters:

dataset (str)

class nasbenchapi.NASBench301[source]

Unified NB301 API wrapper.

Delegates all operations to the underlying NASBench301 implementation.

__init__(data_path=None, verbose=True)[source]
Parameters:

  • data_path (str | None)

  • verbose (bool)

available_budgets(dataset=None, split=None)[source]

Return available budgets derived from NB301 learning curves.

Return type:

Optional[List[int]]

Parameters:

  • dataset (str | None)

  • split (str | None)

bench_name()[source]

Short benchmark name.

Return type:

str

datasets()[source]

Available datasets.

Return type:

List[str]

decode(encoding)[source]

Decode architecture (pass-through for now).

Return type:

Any

Parameters:

encoding (Any)

encode(arch)[source]

Encode architecture (pass-through for now).

Return type:

Any

Parameters:

arch (Any)

id(arch)[source]

Stable identifier for architecture.

Return type:

str

Parameters:

arch (Any)

iter_all()[source]

Iterate all NB301 architectures.

Return type:

Iterator[Any]

load(data_path=None)[source]

Load NB301 from the provided path or environment.

Return type:

NASBench301

Parameters:

data_path (str | None)

mutate(arch, rng, kind=None)[source]

Mutate architecture (no-op for now).

Return type:

Any

Parameters:
query(arch, dataset='cifar10', split='val', seed=None, budget=None)[source]

Query performance metrics for architecture.

Return type:

Dict[str, Any]

Parameters:

  • arch (Any)

  • dataset (str)

  • split (str)

  • seed (int | None)

  • budget (Any | None)

random_sample(n=1, seed=None)[source]

Random sample from NB301.

Return type:

List[Any]

Parameters:
splits(dataset)[source]

Supported splits.

Return type:

List[str]

Parameters:

dataset (str)

class nasbenchapi.Arch101[source]

NASBench-101 architecture representation (immutable & hashable).

__init__(adjacency, operations)[source]
Parameters:
to_lists()[source]

Convert back to mutable lists (for APIs that need them).

Return type:

Tuple[List[List[int]], List[str]]

adjacency: Tuple[Tuple[int, ...], ...]
operations: Tuple[str, ...]

Unified API wrappers for NASBench-101/201/301.

This module provides wrappers around the benchmark-specific implementations to expose a consistent interface across all benchmarks.

class nasbenchapi.nb_api.NASBench101[source]

Unified NB101 API wrapper.

Delegates all operations to the underlying NASBench101 implementation.

__init__(data_path=None, verbose=True)[source]
Parameters:

  • data_path (str | None)

  • verbose (bool)

load(data_path=None)[source]

Load NB101 from the provided path or environment.

Return type:

NASBench101

Parameters:

data_path (str | None)

bench_name()[source]

Short benchmark name.

Return type:

str

datasets()[source]

Available datasets.

Return type:

List[str]

splits(dataset)[source]

Supported splits.

Return type:

List[str]

Parameters:

dataset (str)

available_budgets(dataset=None, split=None)[source]

NB101 does not track explicit training budgets.

Return type:

Optional[List[Any]]

Parameters:

  • dataset (str | None)

  • split (str | None)

op_set()[source]

Operations available in NB101 cell.

Return type:

List[str]

decode(encoding)[source]

Decode architecture from native encoding.

Return type:

Arch101

Parameters:

encoding (Dict[str, str])

encode(arch)[source]

Encode architecture to native format.

Return type:

Dict[str, str]

Parameters:

arch (Arch101)

id(arch)[source]

Stable identifier for architecture.

Return type:

str

Parameters:

arch (Arch101)

random_sample(n=1, seed=None)[source]

Random sample from loaded NB101 architectures.

Return type:

List[Arch101]

Parameters:
iter_all()[source]

Iterate all NB101 architectures.

Return type:

Iterator[Arch101]

mutate(arch, rng, kind=None)[source]

Mutate architecture.

Return type:

Arch101

Parameters:
query(arch, dataset='cifar10', split='val', seed=None, budget=None, average=False, summary=False)[source]

Query performance metrics for architecture.

Return type:

Any

Parameters:
is_valid(arch)[source]

Check if architecture is valid.

Return type:

bool

Parameters:

arch (Arch101)

train_time(arch, dataset='cifar10')[source]

Get training time for architecture.

Return type:

Optional[float]

Parameters:
class nasbenchapi.nb_api.NASBench201[source]

Unified NB201 API wrapper.

Delegates all operations to the underlying NASBench201 implementation.

__init__(data_path=None, verbose=True)[source]
Parameters:

  • data_path (str | None)

  • verbose (bool)

load(data_path=None)[source]

Load NB201 from the provided path or environment.

Return type:

NASBench201

Parameters:

data_path (str | None)

bench_name()[source]

Short benchmark name.

Return type:

str

datasets()[source]

Available datasets.

Return type:

List[str]

splits(dataset)[source]

Supported splits.

Return type:

List[str]

Parameters:

dataset (str)

available_budgets(dataset=None, split=None)[source]

Return available training budgets for NB201.

Return type:

Optional[List[int]]

Parameters:

  • dataset (str | None)

  • split (str | None)

decode(encoding)[source]

Decode architecture (pass-through for now).

Return type:

Any

Parameters:

encoding (Any)

encode(arch)[source]

Encode architecture (pass-through for now).

Return type:

Any

Parameters:

arch (Any)

id(arch)[source]

Stable identifier for architecture.

Return type:

str

Parameters:

arch (Any)

random_sample(n=1, seed=None)[source]

Random sample from NB201.

Return type:

List[Any]

Parameters:
random_sample_str(n=1, seed=None)[source]

Random sample NB201 architectures as arch strings.

Return type:

List[str]

Parameters:
iter_all()[source]

Iterate all NB201 architectures.

Return type:

Iterator[Any]

mutate(arch, rng, kind=None)[source]

Mutate architecture (no-op for now).

Return type:

Any

Parameters:
query(arch, dataset='cifar10', split='val', seed=None, budget=None)[source]

Query performance metrics for architecture.

Return type:

Dict[str, Any]

Parameters:

  • arch (Any)

  • dataset (str)

  • split (str)

  • seed (int | None)

  • budget (Any | None)

index_to_arch_str(idx)[source]

Convert NB201 index (0..15624) to arch string.

Return type:

str

Parameters:

idx (int)

arch_str_to_index(arch_str)[source]

Convert NB201 arch string to canonical index (0..15624).

Return type:

int

Parameters:

arch_str (str)

class nasbenchapi.nb_api.NASBench301[source]

Unified NB301 API wrapper.

Delegates all operations to the underlying NASBench301 implementation.

__init__(data_path=None, verbose=True)[source]
Parameters:

  • data_path (str | None)

  • verbose (bool)

load(data_path=None)[source]

Load NB301 from the provided path or environment.

Return type:

NASBench301

Parameters:

data_path (str | None)

bench_name()[source]

Short benchmark name.

Return type:

str

datasets()[source]

Available datasets.

Return type:

List[str]

splits(dataset)[source]

Supported splits.

Return type:

List[str]

Parameters:

dataset (str)

available_budgets(dataset=None, split=None)[source]

Return available budgets derived from NB301 learning curves.

Return type:

Optional[List[int]]

Parameters:

  • dataset (str | None)

  • split (str | None)

decode(encoding)[source]

Decode architecture (pass-through for now).

Return type:

Any

Parameters:

encoding (Any)

encode(arch)[source]

Encode architecture (pass-through for now).

Return type:

Any

Parameters:

arch (Any)

id(arch)[source]

Stable identifier for architecture.

Return type:

str

Parameters:

arch (Any)

random_sample(n=1, seed=None)[source]

Random sample from NB301.

Return type:

List[Any]

Parameters:
iter_all()[source]

Iterate all NB301 architectures.

Return type:

Iterator[Any]

mutate(arch, rng, kind=None)[source]

Mutate architecture (no-op for now).

Return type:

Any

Parameters:
query(arch, dataset='cifar10', split='val', seed=None, budget=None)[source]

Query performance metrics for architecture.

Return type:

Dict[str, Any]

Parameters:

  • arch (Any)

  • dataset (str)

  • split (str)

  • seed (int | None)

  • budget (Any | None)

class nasbenchapi.nasbench101_api.Arch101[source]

NASBench-101 architecture representation (immutable & hashable).

adjacency: Tuple[Tuple[int, ...], ...]
operations: Tuple[str, ...]
__init__(adjacency, operations)[source]
Parameters:
to_lists()[source]

Convert back to mutable lists (for APIs that need them).

Return type:

Tuple[List[List[int]], List[str]]

class nasbenchapi.nasbench101_api.NASBench101[source]

NASBench-101 API.

Expects a pickle with NB101 keys (entries_by_arch, latest_by_arch, num_records).

__init__(pickle_path=None, verbose=True)[source]
Parameters:

  • pickle_path (str | None)

  • verbose (bool)

get_statistics()[source]
Return type:

Dict[str, Any]

op_set()[source]

Operations available in the NB101 cell.

Return type:

List[str]

decode(encoding)[source]

Decode adjacency/operations strings into an Arch101 object.

Return type:

Arch101

Parameters:

encoding (Dict[str, str])

encode(arch)[source]

Encode Arch101 into native strings.

Return type:

Dict[str, str]

Parameters:

arch (Arch101)

id(arch)[source]

Stable identifier for NB101 architectures.

Return type:

str

Parameters:

arch (Arch101)

get_index(arch)[source]

Get the stable hash identifier for an architecture.

Parameters:

arch (Arch101) – Arch101 architecture object.

Return type:

str

Returns:

String hash identifier for the architecture.

Note

For NB101, architectures are identified by hash rather than numeric index. This method is equivalent to id() but provided for API consistency.

random_sample(n=1, seed=None)[source]

Random sample from the loaded NB101 latest entries.

Return type:

List[Arch101]

Parameters:
iter_all()[source]

Iterate all NB101 architectures (latest entries).

Return type:

Iterator[Arch101]

mutate(arch, rng, kind=None)[source]

Apply a simple one-edit mutation (edge toggle or op swap).

Return type:

Arch101

Parameters:
query(arch, dataset='cifar10', split='val', seed=None, budget=None, average=False, summary=False)[source]

Query performance metrics for an architecture from loaded data.

Parameters:

  • arch (Arch101) – Architecture to query.

  • dataset (str) – Dataset name (‘cifar10’).

  • split (str) – Split name (‘val’, ‘test’, ‘train’).

  • seed (Optional[int]) – Optional seed for reproducibility.

  • budget (Optional[Any]) – Optional budget specification.

  • average (bool) – If True, average metrics across the three training runs.

  • summary (bool) – If True, return the legacy summary dict (metric/metric_name/…).

Returns:

tuple (info_dict, metrics_by_budget) aligned with the original NASBench-101 API. When summary=True, the condensed dictionary from prior versions is returned for backwards compatibility.

Return type:

When summary=False (default)

is_valid(arch)[source]

Check if architecture is valid.

Return type:

bool

Parameters:

arch (Arch101)

train_time(arch, dataset='cifar10')[source]

Get training time for an architecture.

Return type:

Optional[float]

Parameters:
class nasbenchapi.nasbench201_api.NASBench201[source]

NASBench-201 API

OPS = ['none', 'skip_connect', 'nor_conv_1x1', 'nor_conv_3x3', 'avg_pool_3x3']
NUM_OPS = 5
NUM_EDGES = 6
__init__(pickle_path=None, verbose=True)[source]
Parameters:

  • pickle_path (str | None)

  • verbose (bool)

get_statistics()[source]
Return type:

Dict[str, Any]

random_sample(n=1, seed=None)[source]

Sample random architectures from NB201 search space.

Parameters:

  • n (int) – Number of samples to return.

  • seed (Optional[int]) – Optional random seed.

Returns:

List of sampled architecture strings in NB201 canonical format.

iter_all()[source]

Iterate over all architectures in the loaded data.

Returns:

Iterator over architecture strings.

get_index(arch)[source]

Convert an architecture string to its canonical index.

Parameters:

arch (str) – NB201 architecture string (e.g., ‘|none~0|+|skip_connect~0|nor_conv_1x1~1|+|…’)

Return type:

int

Returns:

Integer index (0..15624) corresponding to the architecture.

Raises:

ValueError – If the architecture string is invalid or cannot be parsed.

query(arch, dataset='cifar10', split='val', seed=None, budget=None)[source]

Query performance metrics for an architecture from loaded data.

Parameters:

  • arch (str) – NB201 architecture string.

  • dataset (str) – Dataset name (‘cifar10’, ‘cifar100’, ‘ImageNet16-120’).

  • split (str) – Split name (‘val’, ‘test’, or ‘train’).

  • seed (Optional[int]) – Optional seed (default: 777 for official NB201).

  • budget (Optional[Any]) – Optional epoch number (0-199, default: 199 for final epoch).

Returns:

metric, metric_name, cost, std, info.

Return type:

Dictionary with keys

class nasbenchapi.nasbench301_api.NASBench301[source]

NASBench-301 API.

Supports directory-converted JSON payloads or reserialized pickle.

OPS = ['max_pool_3x3', 'avg_pool_3x3', 'skip_connect', 'sep_conv_3x3', 'sep_conv_5x5', 'dil_conv_3x3', 'dil_conv_5x5', 'none']
NUM_OPS = 8
NUM_NODES = 4
NUM_EDGES_PER_NODE = 2
__init__(pickle_path=None, verbose=True)[source]
Parameters:

  • pickle_path (str | None)

  • verbose (bool)

get_statistics()[source]
Return type:

Dict[str, Any]

random_sample(n=1, seed=None)[source]

Sample random architectures from NB301 search space.

NB301 uses DARTS-style search space which is continuous, but we can sample discrete architectures from it.

Parameters:

  • n (int) – Number of samples to return.

  • seed (Optional[int]) – Optional random seed.

Returns:

List of sampled architecture representations.

iter_all()[source]

Iterate over all architectures in the loaded data.

Returns:

Iterator over architecture keys/indices.

get_index(arch)[source]

Get the index for an architecture in the loaded data.

Parameters:

arch (Any) – Architecture representation (dict, index, or path string).

Return type:

Optional[int]

Returns:

Integer index if architecture is found in loaded data, None otherwise.

query(arch, dataset='cifar10', split='val', seed=None, budget=None)[source]

Query performance metrics for an architecture from loaded data.

Note: NB301 directory payloads include recorded learning curves and final metrics. Validation queries read from the stored curves; test queries report the declared final accuracy for each entry.

Parameters:

  • arch (Any) – Architecture representation (dict or index).

  • dataset (str) – Dataset name (‘cifar10’, ‘cifar100’).

  • split (str) – Split name (‘val’, ‘test’).

  • seed (Optional[int]) – Optional seed for reproducibility (unused).

  • budget (Optional[Any]) – Optional budget specification (unused).

Returns:

metric, metric_name, cost, std, info.

Return type:

Dictionary with keys