config

Default configurations for sparsity modes.

ModeloptConfig ExportSparseConfig

Bases: ModeloptBaseConfig

Configuration (empty!) for the "export_sparse" mode.

Show default config as JSON

Default config (JSON):

{}

ModeloptConfig SparseGPTConfig

Bases: ModeloptBaseRuleConfig

Configuration for the "sparse_gpt" mode.

Show default config as JSON

Default config (JSON):

{
   "nn.Linear": {
      "*": {},
      "*lm_head*": null
   },
   "nn.Conv2d": {
      "*": {},
      "*lm_head*": null
   }
}

field nn.Conv2d: SparseConv2dConfig | None | Dict[str, SparseConv2dConfig | None]

Show details

Configuration for sparse nn.Conv2d module.

If the "nn.Conv2d" key is not specified, the default configuration (shown in JSON) will be used:

{
  "*": {},
  "*lm_head*": null
}

To deactivate any sparse nn.Conv2d module, use None instead of providing a dictionary {}.

To specify layer-specific configurations, you can specify a config for each submodule with the key specifying a glob pattern that matches the submodule name. For example, to convert to a sparse module for all nn.Conv2d layers except for those in the "lm_head" submodule use:

{
    "*": {...},
    "*lm_head*": None,
}

Note that glob expressions are processed sequentially in the order they are specified. Later keys in the config will overwrite earlier keys if they match the same submodule name.

If you want to specify the same configuration for all submodules, you can provide an unnested dictionary as well:

{...}

which is short for

{
    "*": {...},
}

field nn.Linear: SparseLinearConfig | None | Dict[str, SparseLinearConfig | None]

Show details

Configuration for sparse nn.Linear module.

If the "nn.Linear" key is not specified, the default configuration (shown in JSON) will be used:

{
  "*": {},
  "*lm_head*": null
}

To deactivate any sparse nn.Linear module, use None instead of providing a dictionary {}.

To specify layer-specific configurations, you can specify a config for each submodule with the key specifying a glob pattern that matches the submodule name. For example, to convert to a sparse module for all nn.Linear layers except for those in the "lm_head" submodule use:

{
    "*": {...},
    "*lm_head*": None,
}

Note that glob expressions are processed sequentially in the order they are specified. Later keys in the config will overwrite earlier keys if they match the same submodule name.

If you want to specify the same configuration for all submodules, you can provide an unnested dictionary as well:

{...}

which is short for

{
    "*": {...},
}

ModeloptConfig SparseMagnitudeConfig

Bases: ModeloptBaseRuleConfig

Configuration for the "sparse_magnitude" mode.

Show default config as JSON

Default config (JSON):

{
   "nn.Linear": {
      "*": {},
      "*lm_head*": null
   },
   "nn.Conv2d": {
      "*": {},
      "*lm_head*": null
   }
}

field nn.Conv2d: SparseConv2dConfig | None | Dict[str, SparseConv2dConfig | None]

Show details

Configuration for sparse nn.Conv2d module.

If the "nn.Conv2d" key is not specified, the default configuration (shown in JSON) will be used:

{
  "*": {},
  "*lm_head*": null
}

To deactivate any sparse nn.Conv2d module, use None instead of providing a dictionary {}.

To specify layer-specific configurations, you can specify a config for each submodule with the key specifying a glob pattern that matches the submodule name. For example, to convert to a sparse module for all nn.Conv2d layers except for those in the "lm_head" submodule use:

{
    "*": {...},
    "*lm_head*": None,
}

Note that glob expressions are processed sequentially in the order they are specified. Later keys in the config will overwrite earlier keys if they match the same submodule name.

If you want to specify the same configuration for all submodules, you can provide an unnested dictionary as well:

{...}

which is short for

{
    "*": {...},
}

field nn.Linear: SparseLinearConfig | None | Dict[str, SparseLinearConfig | None]

Show details

Configuration for sparse nn.Linear module.

If the "nn.Linear" key is not specified, the default configuration (shown in JSON) will be used:

{
  "*": {},
  "*lm_head*": null
}

To deactivate any sparse nn.Linear module, use None instead of providing a dictionary {}.

To specify layer-specific configurations, you can specify a config for each submodule with the key specifying a glob pattern that matches the submodule name. For example, to convert to a sparse module for all nn.Linear layers except for those in the "lm_head" submodule use:

{
    "*": {...},
    "*lm_head*": None,
}

Note that glob expressions are processed sequentially in the order they are specified. Later keys in the config will overwrite earlier keys if they match the same submodule name.

If you want to specify the same configuration for all submodules, you can provide an unnested dictionary as well:

{...}

which is short for

{
    "*": {...},
}