Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Commit 77a671b

Browse files
authored
Update configs.py
1 parent e6da317 commit 77a671b

File tree

1 file changed

+4
-1
lines changed
  • src/sparseml/exporters/transforms/kv_cache

1 file changed

+4
-1
lines changed

src/sparseml/exporters/transforms/kv_cache/configs.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,9 +125,12 @@ class Config:
125125
multiply_batch_by_num_att_heads=False,
126126
)
127127

128+
# reusing the transforms for codegen, because it happens to match what we need for gpt neo
129+
additional_transforms_gpt_neo = AdditionalTransformsCodeGen
130+
128131
GPT_NEO_CONFIG = KeyValueCacheConfig(
129132
model_name="gpt_neo",
130-
additional_transforms=AdditionalTransformsCodeGen,
133+
additional_transforms=additional_transforms_gpt_neo,
131134
key_num_attention_heads="num_heads",
132135
key_num_embedding_hidden_size="hidden_size",
133136
transpose_value_input=(0, 2, 1, 3),

0 commit comments

Comments
 (0)