aug_repeats #2575
-
|
How does num_aug_repeats and num_aug_splits actually work for data sampling? Is num_aug_repeats capped by the number of gpus? Is num_aug_splits the same as in Repeated Augmentation? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
'num_aug_splits' is where each batch is split into a non augmented plus num_splits-1 augmented versions. It's there to support AugMix https://arxiv.org/abs/1912.02781 ... an extra JSD loss term is added btw augmented and non. Other related args need to be activated for the full splits + augmix to be enabled. aug repeats is for 'Repeated Augmentation' (https://arxiv.org/abs/1901.09335, https://arxiv.org/abs/1902.05509) ... it's a distributed specific regularization/augmentation scheme where the global batch is made up of some repeated samples (but augmented differently) across different ranks. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the reply. Is there any particular reason why aug_repeats is not available for a single GPU or why aug_repeats > #GPUs - 1 seems to have no effect? It would be nice to be able to replicate high aug_repeat settings without needing lots of GPUs |
Beta Was this translation helpful? Give feedback.
'num_aug_splits' is where each batch is split into a non augmented plus num_splits-1 augmented versions. It's there to support AugMix https://arxiv.org/abs/1912.02781 ... an extra JSD loss term is added btw augmented and non. Other related args need to be activated for the full splits + augmix to be enabled.
aug repeats is for 'Repeated Augmentation' (https://arxiv.org/abs/1901.09335, https://arxiv.org/abs/1902.05509) ... it's a distributed specific regularization/augmentation scheme where the global batch is made up of some repeated samples (but augmented differently) across different ranks.