The MMDloss generates negative outputs

[lyfeng001]

I initialized the MMDLoss class and used it to process my data. Here is the code:

mmd_loss = `MMDLoss(mmd_type="quadratic")
……
loss_mmd = mmd_loss(source_embeddings.view(-1, 167*768), target_embeddings.view(-1, 167*768))

I expected the MMD loss to output a positive value, but some outputs are negative during training. Could you help me understand why this might be happening?

the blue one is the training loss, it became negative during the training.

[KevinMusgrave]

I can't remember why, but here's an explanation from the official implementation of MMD:

"The value of the mmd loss could be negative since we used the linear-time unbiased estimate of the mmd, which lends us an O(n) time cost but may cause negative mmd values for some mini-batch. This negative mmd value however will not influence the final classification performance."

Link: https://github.com/thuml/DAN/blob/master/README.md

[lyfeng001]

I get it! Thanks for your reply!