View Single Post
Old 09-04-09, 02:07 AM   #6
Frederf
Seasoned Skipper
 
Join Date: Apr 2005
Posts: 665
Downloads: 79
Uploads: 1
Default

Is it bad that I actually carried through the significant figures ("sig figs" as we call 'em) rules during my calculations? Unless you count the uncertainty in measuring distances using the ruler tool (which isn't involved in the theoretical magic time determination) 2.9624 minutes is 5 sig figs just like the (rounded) length of a nm. Can't unlearn the scientific habit.

Of course the error for using the 2000yd nm is tiny compared to the typical errors in the data. No one should feel bad or worry that they are missing out on any accuracy improvement worth mentioning. Even if the nomograph used the exact definition (which I doubt) it would only be off a pixel or two in certain cases more so than our typical level of precision using the mouse. However I'm pretty sure the nav map notes the difference as I've seen the 1000s of yards have a hiccup when switching to X.X nm consistent with the correct implementation.

I would correct the notion that "1.5 is potentially all values between 1.4 and 1.6." 1.5 would denote all values 1.45 up to but not including 1.55. There's no way to show the little bar over the 9, but I did just learn it's called a vinculum.
Frederf is offline   Reply With Quote