From octave-sources-request at bevo dot che dot wisc dot edu Tue Apr 3 16:50:09 2001 Subject: Missing values and NaN tools From: "Alois Schloegl" To: Date: Tue, 3 Apr 2001 23:50:09 +0200 Looking at the archives, the topic on missing values and NaN's arises periodically. I have often the same problem, so I thought about it in more detail and looked for the best solution, also considering the arguments in the archives. I came to the conclusion that as general rule that NaN's should be handled as missing values. I've implemented this idea in my latest "Time Series Analysis toolbox", which now handles data with missing values, too. You can download it from http://www-dpmi.tu-graz.ac.at/~schloegl/matlab/tsa/ Conseqently following this rule (NaN iid missing value) leads to surprisingly simple code and to a very clean and straightforward programs. I've enclosed examples for the basic functions like MEAN, VAR, STD and DETREND. You can find them also at http://www.dpmi.tu-graz.ac.at/~schloegl/matlab/NaN/. Furthermore, it leads to the consequence that SUM returns always a number; SUM should simply skip NaN's; in case of NaN's only, SUM should return 0. Because it is not possible to me to get rid of the built-in function SUM, I had to rename my version of SUM. I called it SUMSKIPNAN. For performence reasons, the developers of Matlab, Octave and other Math's software should implement skipping the NaN's in the built-in SUM. I hope having provided sufficient arguments that this is the soundest solution. However, please feel free to comment on this suggestion. Alois Schloegl e-mail: a dot schloegl at ieee dot org begin 666 NaN105.tgz M'XL(``(ERCH``^Q]CL'_2'JQ$]W?&!VWT]G1_%[!_YTMMT?\ MW][I=G;='3/$AB\7[Q84""T+RV]P^/[&26 MM];7-L31P?!83+QP, at ^]7&8BOY""FHGD7/A>[@D9RDC&>>8(:@YQ:F3"2Z68 M)'$6^#*5OO R$059%L13<>F%,D-[D@C@D@EA<,S#J>"Z]?)[* MK$\_VH 2"U]ZH6*!DJ9F77A:94LBP3R7%<+;E8'!W8)BC$0S##Y*\7*>YN)% MDL;!1TTRPNCU)/G;H].7AV^.AQ ^_-DDK-2WGX&/ M+L":,!BG7KH at +IVG@),EY_D5]&Y/+)(Y$P>:%V0@Q)@H%#!?-Y dot 4`$2)'YPO MZ-X<,I$ROW*91AFQB7X\/WXK7ND!GLM8IF#(F_DX9,'&DPFH(DFM9W0SNX", MCA?<\QGA3C-(A5*P%M!> M"!*K5%RJ?HZFWC(9RMGZQ!8"?)',6(5SFNM5$(80= %#?CX/;0*!QN+=X>C% MZ[_[+'X)7 at J+Z4"%42S, !DS"OUXGP!W)5Y//GI!;H, MGQZ^ dot AS]0C-X=C at Z/C@]%<]>GXBA>#,\&1W^]/;5\$2\>7ORYO7I@2/$J92& MSHJ at JTA=T/D<0-D8^3+W@C S!/@%C,Z 9>A#SB%54',97)*]A0#.%E_(S#"! MAK'6Y17"[HG at 7,1)#IL('6-Q!DSJ>!NG;7$83QQ;;/\@1A)$D^)-Z$TDM dot )T M3A!ZO0[3_FF2Y=3Z:"A$I^NZ;MOM=79M\?9TB F2$3-RXCJ=+?H=>K#2J;P, MU-U_0K%2T84K00]_PHS38'J1B^:D)=P??MAMTR dot 2RV&88$*G>GVGQK(= at 9!] MX3EFU?]7(*5TDG1JT= at T9VA\$/_8A<[I"S9Q\/[#GBCO0$0'L/U[ZVNPK=RK MJ;OU6F4KW/X49#',!XQ2$U!:MMAX,MC9WMOPQED3`'"C;%X%CL9[-Y_<1$2& MF5P-0:%7M(O]/4R0FHO_&L$\F at TPQ H$_QCU>A]"B>FW9R@?9!";?*$1J=,_ M"N+F>1#[S2SX73:O6T_<5FNO-I-Z;^KC[E4&H3$ at L#&6DV8C dot 3]OM,0&Y%OK M5Z!DJSU>M'^7:2)TRXSE,?M]P*,&K3V-J6C2>CD8""*1^+_RYY 616LQR.91 M]C&8Q<:K:CF;E5M$='K2XD=Z&M8&K:]H)?! ;(IT3_&+A,N GS8JXK41H/VG MI at ?CUH0G]41T%+S6GMA0ZW@LAJ]>D1(#[&R6I+DRUWHM(JLL86*T`%DTO+R> MB6:)*1; dot 9&KP%'\PAV(2 at 9I#T1^(_3@PF#WIF&8+$' dot !GOTZ^8\G_^5Y[F, M[$6I)=82KI\&?P:RMV+C;-9H[3J;US6)6L&,09T9GV/$"\.(O\[,2I-GG<_2 M(,Z!+MQ1\O/Z0CL/XM\;V;\9$_* IS&TUV_8]+"ES('F%PA ']]Z*_9-+MK_ M9[E_G]O_S^W_W=[V#N__=W>WW$ZO1_O_[I;[N/]_B*O8_R<#B$$S8/TB_PS[ MC>4]'2 at %)R_UX8A>!NSL.5^ZYR>H<9)&7@AES)2_V3QNNRU'O(53:<- MYHW])6T at 05O[SAO-)?][^Z;_W=UY$/_;KKC;@^RW-&]>>BE6JW(-L*KWR^49 MSAZ;]MJ?;ZUM?[V+[#_VE2GH=6]KP!_;?[>STW/9_F]O[W2V7*P%M!/L/MK_ MA[C*^ dot ^9/?HPT*+0S dot TS>\8V:?] at =')PO _%CY)+O1!P&W&>)A''@.T[+@$$ MK at J28E?S,%PLF;,9V9ADGJT(G)F>.DY!L9$P3*YH"&-RA>?[TM>&%[;9#W%/ MX0=;Y&5P**DY1X4` at 4(?0/LFHF7G>2Q_FWN$9S;S)I@8S5DUR>!7`KMDGL_F MN9AYJ1=)"E5- at SJI="BF['C'" dot %[QW%*IKS/[0_B3+RW9QPCMPJ6*'9@K;H- M58L,-^B:4:1>X86%1LP2 dot ,Z\/)[!I!(_Z!'30!!YJ$V dot )]1B)B*H" at 70/)'! M,?#2E3B>V9TZ;CI?<$OC6:WQ+ D7<1(%6"E!L22EX"1V:&8QG at W/F3FV!6K[KBQQ5K+?>NK>I] MT77%,SG6R_ICM/DQVOSWB#8O^;N=%?YN53'^9']W0XM8: MP-*NBOG2*&!%N'=EI+J*O*4]]^):A7DUOG[C^9/B<1D:P,9?$_%Q.95KB5]5# MV dot TG&%/8E$B0TL^4]8S at )J2+DB0C$)IR"IE1L=8_ dot -Y:"7"/!O#1(B\'V-@V MK?8X/[$ZEMIU dot HZ[4]6$F]\(34WY+J5+;&M#F7 at X2@'9RU)LLIN*22T_#MR^ MYG>7A6^Y4>!?#S at IHX dot W9\V^_;%52\PL7V dot 27WB6_C5:BN]%$L-7@Y*"!?^ M]K2$\[^BJ6Z&,I[F%]2T14+RO6AV1%_,&+Q5X#(RL : C39C\:LXT_=42R56 MIGW1L6Z:+ OD`NOB^ M1)."&!MG9-1?7^,-,-T<)]=BEB:7 at 2\I?J?"AQE;,XZ8 M$32N#%4YDGH2:'UM>PE',\;A8 M_&C[FV)3`06F0(\E+%5HJ*M#^ZLK$^VB+I%LP<_8K%O6)5IZ\83R&926LJR; MR38\ dot CDZM:PT at 3_(XV2_S3$\MI"6#HQ:VID%-6SC,IKH)[%V97R0L"_#5H*^ MDTGVQ' at >A'D[* G!,$!YUZPV,;+* M)K;HB:(HB8JC(D/XGR/? at !7G2O*Y-P=S? dot FIN!"Z5>.P7EURE9191\,S6QP= M'MM41V%SI(A '0U'KX9/^^(J23_2`J8B-;*2417OH dot at 5$,Z1,^ dot O/*@H"6YB=%`,>RIA]2E7(VDQ-RXHBJ3 LL3*>OCP^/G\/^O,U8<#G7P-*?%3%\';$&#T at H MO4LO`-NI:IE90VV),/0U at 1>KE-Q6P;;UM8A"?J6,*KFES 7%=B8>R8LV2(:7'H\-U) at JL7ED M-H<9K 87L*2DUK2W3],D=<1KBG 6^)(P%%D8&U+L4R(B at ISX5 dot :4TM %^[F: MN[/9L8W^U^AFPTPH`\BXV8JJ0A<(FILRGRB!/61U`.IEL)_PA)8LR;=-BA6\`H?W*0A]SF2AD!?E9 at B=@)X"I]R#/:""0,QV#+ dot A5I#Z9@=0@28V MYU0 at 5KB1<%%5U@=T'0![6UQ=! dot !SD5'(`DJ*@1'8'"BPQN;?Q(W#"[<@QSL1 M-J)= at R7YQ dot :FR]08!QZ)CX2L1HQT2ZN4DAH((Q@=&;%+P0E@1PS3D5\6'99T MXCH8G+T_[2DJ*ZHZR]((,>W6I!T/(_.JZE6KA__J:24T2OMHO MJ:RV'H%1RZVCAC..S!7I#?AD3*9>M+:=KHT_O6I2D.,]6PZK'.5=_"1N:)O! MM++Y;JDSAB$JG%QQGDR^80)$IZ2?G .!6.L`.SBE1BBL'T7J/,%+/6D<:1GM MZ"P5%^S `//,9EY^H5<#39WD$E84"S,G?""_E0A" MX>RP+ at ;\M<@W%XXC2%2>JR$B^PG36-G$9P?#T=N3`Q*M.Y?_W*6LAU+.QJNC M8LM"4IOU at R@@;2<O7K\Y.#'R[XV3 M&W&=0OTR6I-8!E+)*^6$W15R][1FLB'3JY%=)PYQJ>) at E@)JG$>3SX.P&)<3 M[1WQU,N"B5KUR2G5':D>5%%Q#UI_Y2VRPM H>U at 9@7S[?)["I dot Y5/;O2?[-U M4I<::3]B>=%9IMS;4R*:,"M?HS2$;(W(%X8P!"Q^A:$B$BD7'! at ]( -8]7D+J3]M$^A&4$W2=Z6SFM82XGE MVGF-+TF??29[9KU[]ZXO+O)\UM_%G\0_R%- at +!^T_KOK:VNCO\!Z9TMKO^F\_^/\;_[ORKU MWQ"#2OTWE6(OU7\O!Y[N7OQM:K=IB,?:[2^KW?[*DNT51R;OLV3[MJ)M?5C1 MY#8>XGQ?0E495!%>.5<4.-]#/O_3#%JWGL,+:F?8_BXUXV3_,7NWL^V$V3TM M`I^S_V[/V/^M3G?'Y?KOSF/]]X-FU86;XZ;[,)FG GF=?O-#%U+<%K*O!CTUE M"3=/)Q14 at @/S4BZNDM3/,'XM!+LIL7K7L(N!FKSC* M9)O*,%)Z[/?F\%G3E?9.-.OFL572Z< at + dot $H'FHP7=^R\OO8F#2*XA^TL(#)6 M]C+MN^YEU!95!S&D!K2^]AKF/8B]\";D.^^2"#UX`+1]!?D5?^QBJ:(E! QH MSQ)XP)CP\S>O2"K\!TQ at /U[_U44\AEI\2_^_5^;_>VZG^WC^\R&OPO]?#(H# M=,7+GR at UO;0',";T2TY^FE,2M .@08H=@+CQVJ B)0*GN?2GT5RU5ID4Y=[K M=Q3Q*2!V\>,D;M.PH135+<7J;49_U3ZC?W.CT;_'G4:-'G9#Y6S,^Y0JIV2I M)D)4\M1\8/8>=BO?>G]RGZ]T,?7$04PGMPI MO4ME'FL6[:W:WZB*:] dot YN at =:\8*5KWC?RYU>][(*Q0H:F&?3;3T9-+8;J\K[ MS54YCO!.O0*%CR* GWVR"12YMC'/+A\9\Z)Q,)TG\\S1!=)W at *K32U>>2O\6 MLLZB;G.,W(S3,'K1:'TU_"O)8L0Y,R[K-P-<.=^?<1;WJO4%T"GB^ dot + at U1NV MCT5 ,8 at Y#<+'XI=@+==#FNK2BKSTJ$QQ@T*D*05_ dot 8/^T718/BY0P>6 B@%* M[ICHOTZ$4 ;4B/JOL=!'("[9U"^_=:]>WKXADC%YLCJ+729'R^R(JMG\HAU] M\'4[^ at WA3?(YK)I9+90!CE>^.T3OX[EPOJ""R8&LKT7U3H&.!\24=E$'%:_% M(&CK4OS(UG at [F_PE8NR77EIRK8,+U^7(>!IA+31Q,K(`69Y.HAF5AI6FGK,V M1F42P6Y3"P9K=V>RP>#"A"*V%GDJA4I]Q MQ4ER. TLPV3JY-?Y/? at 8G]O_[[JN]O^ZG=XV[_]WM[<>_;^'N)R"^9Q$UQ4" M)#0ON01H/)]FM)5LEVX"O16(WH!(V4"3^83%2OVRR=6%)&M#5JQ2NW MFM5* M!U7.H(\-JOSZJL\HP]UA&IO_R!ZF]V=S8Y+%9B691:Z"-H&*TGU?(P'Z26: M;W6*YBN+, GQ<,YOV$3[G7[O![ at SFYT>=:@65U*YAZIO8%3<;G^;(+N[U)!K M2(M7B](;0U16`ZO.98E?+ M)?^U.$_/7'&[Z+A5DEJ_X TBQ^P at "? MV_]W.MOE^S^Z_/Z/G5[OT?X_Q%5]_].R5T2.UZDJR/S_=JZ>M6$8B,[^%UF$ M)1 A=LEB\!!*YD(I64KI$#(8 at P-)A_[\WDEW^K!3%]J&9+@W!7_@9^NDZ*1[ M+[H\X_KP3QD_V=2&<-J7EL'OU^%DA_U; M9B*ZVS[G:P^'S^[\H4L?T dot ]P&H dot H-" at <6]N%4BJ)]XPLTLM6+!C'ULN dot *WCX MG/ITV,>;)QK$(]SNU'#^55/]7$?Z.1+ C>]TC5'D:Q=,M_Z>+7RK:I;MZ<*K M1K8]2 at dot 9;6]SOKUMC+DLGV2V!5F+L5!<4YX=9W)-/@KQ/+C"<:!VJ71!*L#Q M,UA8"E^!4D)H9ETU#\:6ZV4-C1W:F?IHTM ARDR[2OA3#'+8*4]?C4['##D$ MJ _D6_\)"00" at 4 @$ @$`H% (! (!(*KX O9J44@`'@````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` M```````````````````````````````````````````````````````````` 9```````````````````````````````````` ` end ------------------------------------------------------------- Octave is freely available under the terms of the GNU GPL. Octave's home on the web: http://www.octave.org How to fund new projects: http://www.octave.org/funding.html Subscription information: http://www.octave.org/archive.html -------------------------------------------------------------