Currently the relative error when the reference value is zero is defined using an epsilon to avoid division by zero; see https://github.com/ROCmSoftwarePlatform/composable_kernel/blob/313bbea5886850acab286f45e9d9816cf0b0dca0/library/include/ck/library/host_tensor/host_tensor.hpp#L327
The problem is that this approve results in very large relative error even if the absolute error is small. For example, (v = 1e-8, v_ref = 0, eps = 1e-10), then ref_diff = 100. We can use more reasonable definition, for example:
rel_diff = (v - v_ref) / max(|v|, |v_ref|). See https://en.wikipedia.org/wiki/Relative_change_and_difference for other alternatives.
Currently the relative error when the reference value is zero is defined using an epsilon to avoid division by zero; see https://github.com/ROCmSoftwarePlatform/composable_kernel/blob/313bbea5886850acab286f45e9d9816cf0b0dca0/library/include/ck/library/host_tensor/host_tensor.hpp#L327
The problem is that this approve results in very large relative error even if the absolute error is small. For example,
(v = 1e-8, v_ref = 0, eps = 1e-10), thenref_diff = 100. We can use more reasonable definition, for example:rel_diff = (v - v_ref) / max(|v|, |v_ref|). See https://en.wikipedia.org/wiki/Relative_change_and_difference for other alternatives.