Javascript precision incrementdecrement number - Stack Overflow

admin管理员组

文章数量:1023838

I am working with values between and including 0 and 1 with 2 decimal points.

I am trying to increment from 0 to 1 in steps of 0.01 but am encountering issues:

The following is in a function that recursively calls itself.

if ($Y.drawChallengeTextAlpha.toFixed(1) < 1)
 $Y.drawChallengeTextAlpha += 0.01;

I never get past 0.95.

EDIT

I ended up with the following:

// $Y.drawChallengeTextAlpha is an integer from 0 to 100
if ($Y.drawChallengeTextAlpha < 100)
  $Y.drawChallengeTextAlpha += 1;  // May not always be 1

I then get my precise value with ($Y.drawChallengeTextAlpha / 100)

I am working with values between and including 0 and 1 with 2 decimal points.

I am trying to increment from 0 to 1 in steps of 0.01 but am encountering issues:

The following is in a function that recursively calls itself.

if ($Y.drawChallengeTextAlpha.toFixed(1) < 1)
 $Y.drawChallengeTextAlpha += 0.01;

I never get past 0.95.

EDIT

I ended up with the following:

// $Y.drawChallengeTextAlpha is an integer from 0 to 100
if ($Y.drawChallengeTextAlpha < 100)
  $Y.drawChallengeTextAlpha += 1;  // May not always be 1

I then get my precise value with ($Y.drawChallengeTextAlpha / 100)

• javascript

Share Improve this question Follow edited Nov 25, 2010 at 20:32 Russell asked Nov 25, 2010 at 20:03 RussellRussell 97866 gold badges1313 silver badges3030 bronze badges 2

• What browser? What version of JS? – Oded Commented Nov 25, 2010 at 20:06
• Chrome 7 on Ubuntu 10.04 – Russell Commented Nov 25, 2010 at 20:07

Add a ment  | 

5 Answers 5

Sorted by: Reset to default 4

toFixed rounds the number UP, that's why you start getting return values of 1 when you get in the vicinity of 0.95.

Floating points are messy. If you really need it to be 100% precise, use an integer variable, increment that by 1 in each iteration, test for when it reaches 100, and then, for the actual calculation, take your variable and divide it by 100 to obtain the decimal value you need.

(0.959).toFixed(1)

This is returning "1" on Firefox. Is there a reason you are using toFixed and not just testing if the number is less than 1?

(Incidentally, 0.01 cannot be represented as a non-terminating sequence of binary digits, so beware that it's not going to add up to 1 in 100 iterations; it will take at least 101.)

Maybe this is because typeof (3).toString() returns string, for example. Why don't you pare it to the actual number?

why dont you just use:

if ($Y.drawChallengeTextAlpha < 1)
 $Y.drawChallengeTextAlpha += 0.01;

Without the .toFixed(1) function? I think it will work just fine.

You can try using:

if (parseInt($Y.drawChallengeTextAlpha) < 1)
 $Y.drawChallengeTextAlpha += 0.01;

parseInt will round down so if you have a float/double smaller than 1, it will return 0. Higher than 1 and smaller than 2, will return 1 and so on.

toFixed() rounds ups. .96.toFixed(1) is 1. Change to toFixed(2).

I am working with values between and including 0 and 1 with 2 decimal points.

I am trying to increment from 0 to 1 in steps of 0.01 but am encountering issues:

The following is in a function that recursively calls itself.

if ($Y.drawChallengeTextAlpha.toFixed(1) < 1)
 $Y.drawChallengeTextAlpha += 0.01;

I never get past 0.95.

EDIT

I ended up with the following:

// $Y.drawChallengeTextAlpha is an integer from 0 to 100
if ($Y.drawChallengeTextAlpha < 100)
  $Y.drawChallengeTextAlpha += 1;  // May not always be 1

I then get my precise value with ($Y.drawChallengeTextAlpha / 100)

I am working with values between and including 0 and 1 with 2 decimal points.

I am trying to increment from 0 to 1 in steps of 0.01 but am encountering issues:

The following is in a function that recursively calls itself.

if ($Y.drawChallengeTextAlpha.toFixed(1) < 1)
 $Y.drawChallengeTextAlpha += 0.01;

I never get past 0.95.

EDIT

I ended up with the following:

// $Y.drawChallengeTextAlpha is an integer from 0 to 100
if ($Y.drawChallengeTextAlpha < 100)
  $Y.drawChallengeTextAlpha += 1;  // May not always be 1

I then get my precise value with ($Y.drawChallengeTextAlpha / 100)

• javascript

Share Improve this question Follow edited Nov 25, 2010 at 20:32 Russell asked Nov 25, 2010 at 20:03 RussellRussell 97866 gold badges1313 silver badges3030 bronze badges 2

• What browser? What version of JS? – Oded Commented Nov 25, 2010 at 20:06
• Chrome 7 on Ubuntu 10.04 – Russell Commented Nov 25, 2010 at 20:07

Add a ment  | 

5 Answers 5

Sorted by: Reset to default 4

toFixed rounds the number UP, that's why you start getting return values of 1 when you get in the vicinity of 0.95.

Floating points are messy. If you really need it to be 100% precise, use an integer variable, increment that by 1 in each iteration, test for when it reaches 100, and then, for the actual calculation, take your variable and divide it by 100 to obtain the decimal value you need.

(0.959).toFixed(1)

This is returning "1" on Firefox. Is there a reason you are using toFixed and not just testing if the number is less than 1?

(Incidentally, 0.01 cannot be represented as a non-terminating sequence of binary digits, so beware that it's not going to add up to 1 in 100 iterations; it will take at least 101.)

Maybe this is because typeof (3).toString() returns string, for example. Why don't you pare it to the actual number?

why dont you just use:

if ($Y.drawChallengeTextAlpha < 1)
 $Y.drawChallengeTextAlpha += 0.01;

Without the .toFixed(1) function? I think it will work just fine.

You can try using:

if (parseInt($Y.drawChallengeTextAlpha) < 1)
 $Y.drawChallengeTextAlpha += 0.01;

parseInt will round down so if you have a float/double smaller than 1, it will return 0. Higher than 1 and smaller than 2, will return 1 and so on.

toFixed() rounds ups. .96.toFixed(1) is 1. Change to toFixed(2).

本文标签： Javascript precision incrementdecrement numberStack Overflow