Miko/LSL-PyOptimizer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rewrite Mono float to string conversion; fix llDumpList2String

Browse Source 
llDumpList2String has changed its behaviour with respect to minus zero. Now it converts -0.0 to a string without the minus sign.

While testing this, we noticed several mismatches in the float to string conversions; the existing routine did not properly convert some values because as we discovered later, it is subject to double rounding; one of them is the built-in round-to-nearest-or-even while getting the first 7 significant digits, and the other is just an increment when the digit is a 5 or more, so round to nearest, ties away from zero, and is performed on the digit past the five or six visible digits that LSL shows.

The new code is a tad easier to understand and more robust.

A first variant of the new code is left commented out for history's sake, and will be removed in the next commit.
This commit is contained in:
Sei Lisa
2022-05-17 21:50:29 +02:00
parent 281ff4d96a
commit d2e64e25a6
3 changed files with 194 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
unit_tests/expr.suite/dumplist2string.out Normal file
View File

@@ -0,0 +1,37 @@
[ "String cast in various ways except DumpList2String"
, "<-0.00000, 0.00000, 0.00000>"
, "<-0.000000, 0.000000, 0.000000>"
, "-0.000000"
, "<0.000000, 0.000000, 0.000000>"
, "<0.00000, 0.00000, 0.00000>"
, "0.000000"
, "<0.000002, 0.000000, 0.000000>"
, "<0.00000, 0.00000, 0.00000>"
, "0.000002"
, "<0.000003, 0.000000, 0.000000>"
, "<0.00000, 0.00000, 0.00000>"
, "0.000003"
, "0.000003"
, "<100000000000.000000, 0.000000, 0.000000>"
, "<100000000000.00000, 0.00000, 0.00000>"
, "100000000000.000000"
, "100000000000.000000"
, "<-0.000000, 0.000000, 0.000000>"
, "<100000000000.000000, 0.000000, 0.000000>"
, "<-0.000000, 0.000000, 0.000000>"
, "<99999997952.000000, 0.000000, 0.000000>"
, "DumpList2String:"
, "<0.000000, 0.000000, 0.000000, 0.000000>"
, "<0.000000, 0.000000, 0.000000, 0.000000>"
, "0.000000"
, "<0.000001, 0.000000, 0.000000>"
, "<-0.000001, 0.000000, 0.000000>"
, "<0.000000, 0.000000, 0.000000>"
, "<0.000000, 0.000000, 0.000000>"
, "<0.000002, 0.000000, 0.000000>"
, "<-0.000002, 0.000000, 0.000000>"
, "<0.000003, 0.000000, 0.000000>"
, "<-0.000003, 0.000000, 0.000000>"
, "100000000000.000000"
, "------"
]