summaryrefslogtreecommitdiff
path: root/arch/frv
diff options
context:
space:
mode:
authorJames Bottomley <JBottomley@Odin.com>2016-01-20 14:58:29 -0800
committerLinus Torvalds <torvalds@linux-foundation.org>2016-01-20 17:09:18 -0800
commit564b026fbd0d28e9f70fb3831293d2922bb7855b (patch)
treeab7a5f6751cb7ef5764031e6f9bfee8a051aa2a7 /arch/frv
parenta4cc3c3c7356ded3eba5905f94279382a05d9c96 (diff)
string_helpers: fix precision loss for some inputs
It was noticed that we lose precision in the final calculation for some inputs. The most egregious example is size=3000 blk_size=1900 in units of 10 should yield 5.70 MB but in fact yields 3.00 MB (oops). This is because the current algorithm doesn't correctly account for all the remainders in the logarithms. Fix this by doing a correct calculation in the remainders based on napier's algorithm. Additionally, now we have the correct result, we have to account for arithmetic rounding because we're printing 3 digits of precision. This means that if the fourth digit is five or greater, we have to round up, so add a section to ensure correct rounding. Finally account for all possible inputs correctly, including zero for block size. Fixes: b9f28d863594c429e1df35a0474d2663ca28b307 Signed-off-by: James Bottomley <JBottomley@Odin.com> Reported-by: Vitaly Kuznetsov <vkuznets@redhat.com> Cc: <stable@vger.kernel.org> [delay until after 4.4 release] Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'arch/frv')
0 files changed, 0 insertions, 0 deletions