<!-- $Header: /cvsroot/nco/nco/doc/index.shtml,v 1.482 2008/05/11 16:03:27 zender Exp $ Purpose: NCO homepage Usage: http://dust.ess.uci.edu/nco/index.shtml file:///home/zender/nco/doc/index.shtml /usr/bin/scp ~/nco/doc/index.shtml ~/nco/doc/nco.png ~/nco/doc/nsf.png nco.sf.net:/home/groups/n/nc/nco/htdocs scp -p ~/nco/doc/index.shtml ~/nco/doc/nco.png ~/nco/doc/nsf.png dust.ess.uci.edu:/var/www/html/nco --> <!-- Anti-Spam measures taken from http://ddi.digital.net/~gandalf/spamfaq.html Do unto others: <a href="mailto:root@[127.0.0.1]"></a> This requires Server Side includes: <a href="mailto:abuse@!--#echo var="REMOTE_ADDR"-- ">Anti spambot</a> Send Spam to central authority for documentation: <a href="mailto:news.admin.net-abuse.email@myriad.alias.net"></a> Try to disable Spammers' machines: <a href="mailto:'halt'@localhost">Lynx Users: do not follow this link</a> <a href="mailto:;halt;@localhost">Lynx Users: do not follow this link</a> --> <!doctype html public "-//W3C//DTD HTML 3.2//EN"> <html> <head> <title>NCO Homepage</title> <meta name="Generator" content="Old fashioned typing"> <meta name="author" content="csz"> <meta name="Keywords" content="NCO Homepage, netCDF, netCDF operator, GCM, HDF, scientific data"> <link rel="icon" href="./nco.png" type="image/png"> </head> <body bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#008080" alink="#FF0000"> <table width="100%"><tr><td align="left" valign="top">/ <a href="http://lists.sf.net/mailman/listinfo/nco-announce">Announce</a> / <a href="http://sf.net/forum/forum.php?forum_id=9831">Developer</a> / <a href="http://sf.net/forum/forum.php?forum_id=9829">Discussion</a> / <a href="http://sf.net/forum/forum.php?forum_id=9830">Help</a> / <a href="#RTFM">Manual</a> / <a href="http://nco.sf.net">Homepage</a> / <a href="http://sf.net/projects/nco">Project</a> / <a href="http://nco.cvs.sf.net/nco/nco/src/nco/">Source</a> / </td> <td align="right" valign="top"> [<a href="http://www.antlr.org">ANTLR</a>] [<a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a>] [<a href="http://www.unidata.ucar.edu/packages/netcdf/netcdf-4">netCDF4</a>] [<a href="http://opendap.org">OPeNDAP</a>] [<a href="http://swamp.googlecode.com">SWAMP</a>] [<a href="http://www.unidata.ucar.edu/packages/udunits">UDUnits</a>] </td></tr></table> <p><hr width="100%"></p> <p><h1 align="center">Welcome to the netCDF Operator (NCO) Homepage</h1> <p><h2>Current NCO version is <!--#include virtual="VERSION"--> last updated on <!--#flastmod file="VERSION"--> </h2> <table border=0 width=100%> <td> <p><ul> <li><a href="#News">News & Announcements</a></li> <li><a href="#Definition">What is NCO?</a></li> <li><a href="#Contribute">How to Contribute</a></li> <li><a href="#prp_sei">NCO→SDO Project</a></li> <li><a href="#Publications">Publications</a></li> <li><a href="#Version">Version Summary</a></li> <li><a href="#Download">Download/CVS NCO Software</a></li> <li><a href="#RTFM">Documentation and User's Guide</a></li> <li><a href="#README/ChangeLog/TODO">README/ChangeLog/TODO</a></li> <li><a href="#FAQ">FAQ</a></li> <li><a href="#HPCC">Using NCO at UCI, NCAR, and other HPCCs</a></li> <li><a href="#BM">Benchmarks</a></li> <li><a href="#bld">Building from Source</a></li> <li><a href="#bug">Known Problems</a></li> <li><a href="#People">People</a></li> <li><a href="#Contact">Contacting the Project</a></li> </ul> </td> <td> <img src="nco.png" height=180 width=180 align=right> The temporary NCO logo…<br> Have a better one?<br> Send it in!<br> </td> </table> <hr> <!-- http://nco.sf.net#news --> <dt><a name="News"></a><a name="news"></a></dt> <h2>NCO News & Announcements</h2> <ul> <li>May 11, 2008: NCO 3.9.5 released</li> <li>March 14, 2007: NCO <a href="http://dust.ess.uci.edu/ppr/ppr_Zen08.pdf">overview paper</a> in Press in EMS</li> <li>March 4, 2008: NCO 3.9.4 released</li> <li>December 10, 2007: SWAMP <a href="http://dust.ess.uci.edu/ppr/pst_WZJ074.pdf">Poster</a> IN11B-0469 at Fall AGU</li> <li>December 7, 2007: NCO 3.9.3 released (and SWAMP!)</li> <li>December 4, 2007: NCO <a href="http://dust.ess.uci.edu/ppr/ppr_ZeM07_ijhpca.pdf">scaling paper</a> published in IJHPCA</li> <li>August 29, 2007: NCO 3.9.2 released (_FillValue support)</li> <li>August 27, 2007: <a href="http://swamp.googlecode.com">SWAMP site</a> spins-off from NCO</li> <li>June 29, 2007: NCO 3.9.1 released (netCDF4 deflation support)</li> <li>May 26, 2007: NCO 3.9.0 released (netCDF4 atomic type support)</li> <li>May 2, 2007: <a href="http://dust.ess.uci.edu/smn/smn_WZJ07_gpc_200705.pdf">Talk</a> on SWAMP at GPC, Paris, France</li> <li>April 20, 2007: NCO 3.2.0 released (with i386/x86_64 debs and RPMs)</li> <li>March 29, 2007: NCO 3.1.9 x86_64 RPMs released</li> <li>March 10, 2007: NCO 3.1.9 released (with i386/x86_64 debs and i386 RPMs)</li> <li>February 22, 2007: <a href="http://dust.ess.uci.edu/smn/smn_WZJ07_opendap_200702.pdf">Talk</a> on SWAMP at OPeNDAP Developer's Workshop, Boulder, Colorado</li> <li>January 15, 2007: <a href="http://dust.ess.uci.edu/smn/smn_nco_ams_200701.pdf">Talk</a> and <a href="http://dust.ess.uci.edu/ppr/abs_xtn_ZeW07.pdf">Extended Abstract</a> on NCO architecture and Scaling at AMS IIPS, San Antonio, Texas</li> <li>January 15, 2007: <a href="http://dust.ess.uci.edu/smn/smn_WZJ07_ams_200701.pdf">Talk</a> and <a href="http://dust.ess.uci.edu/ppr/abs_xtn_WZJ07.pdf">Extended Abstract</a> on SWAMP at AMS IIPS, San Antonio, Texas</li> <li>December 9, 2006: <a href="http://dust.ess.uci.edu/ppr/pst_ZMW06.pdf">Poster</a> on NCO Scaling at Fall AGU, San Francisco, California</li> <li>December 9, 2006: <a href="http://dust.ess.uci.edu/ppr/pst_WZJ06.pdf">Poster</a> on SWAMP at Fall AGU, San Francisco, California</li> <li>December 6, 2006: NCO 3.1.8 released</li> <li>November 11, 2006: NCO 3.1.7 released</li> <li>September 18, 2006: NCO 3.1.6 released</li> <li>August 29, 2006: NCO 3.1.5 released</li> <li>July 7, 2006: <a href="#Publications">Submitted manuscripts</a> that describe NCO and its scaling</li> <li>June 7, 2006: NCO 3.1.4 released</li> <li>May 30, 2006: NCO 3.1.3 released</li> <li>April 21, 2006: NCO 3.1.2 released</li> <li>January 30, 2006: NCO 3.1.1 released</li> <li>December 2, 2005: NCO 3.1.0 released—All arithmetic operators support MPI</li> <li>October 28, 2005: NCO 3.0.3 released</li> <li>October 17, 2005: <a href="#BM">NCO Parallel Benchmarks now available.</a></li> <li>August 22, 2005: NCO 3.0.2 released</li> <li>June 10, 2005: NCO 3.0.1 released</li> <li>March 24, 2005: NCO 3.0.0 released</li> <li>September 7, 2004: NCO 2.9.9 released—<tt>ncpdq</tt> now packs data quietly</li> <li>August 21, 2004: <a href="http://www.nsf.gov">National Science Foundation (NSF)</a> funds the <a href="#prp_sei">NCO→SDO Project</a>!</li> <li>August 7, 2004: NCO 2.9.8 debuts <a href="./nco.html#ncpdq"><tt>ncpdq</tt></a> operator—performs arbitrary dimension re-ordering and reversal</li> <li>March 4, 2004: Submitted to NSF the first <a href="http://dust.ess.uci.edu/prp/prp_sei/prp_sei.pdf">proposal</a> for Institutional funding for NCO.</li> <li><a href="./nco_news.shtml">Past NCO News & Announcements</a></li> </ul> <hr></p> <!-- http://nco.sf.net#Definition --> <dt><a name="Definition"></a><a name="dfn"></a></dt> <h2>What is NCO?</h2> <p>The <a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a> Operators, or NCO, are a suite of programs known as <b>operators</b>. Each operator is a standalone, command line program which is executed at the UNIX shell-level like, e.g., <tt>ls</tt> or <tt>mkdir</tt>. The operators take <a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a> files as input, then perform a set of operations (e.g., deriving new data, averaging, hyperslabbing, or metadata manipulation) and produce a <a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a> file as output. The operators are primarily designed to aid manipulation and analysis of gridded scientific data. The single command style of NCO allows users to manipulate and analyze files interactively and with simple scripts, avoiding the overhead (and some of the power) of a higher level programming environment. The <a href="./nco.html">NCO User's Guide</a> illustrates their use with examples from the field of climate modeling and analysis. Note that the <a href="./nco.html#averagers">“averagers”</a> are misnamed because they perform many non-linear operations as well, e.g., total, minimum, maximum, RMS: <ul> <li><a href="./nco.html#ncap2"><tt>ncap2</tt></a> netCDF Arithmetic Processor (<a href="./nco.html#xmp_ncap">examples</a>)</li> <li><a href="./nco.html#ncatted"><tt>ncatted</tt></a> netCDF Attribute Editor (<a href="./nco.html#xmp_ncatted">examples</a>)</li> <li><a href="./nco.html#ncbo"><tt>ncbo</tt></a> netCDF Binary Operator (includes <tt>ncadd</tt>, <tt>ncsubtract</tt>, <tt>ncmultiply</tt>, <tt>ncdivide</tt>) (<a href="./nco.html#xmp_ncbo">examples</a>)</li> <li><a href="./nco.html#ncea"><tt>ncea</tt></a> netCDF Ensemble Averager (<a href="./nco.html#xmp_ncea">examples</a>)</li> <li><a href="./nco.html#ncecat"><tt>ncecat</tt></a> netCDF Ensemble Concatenator (<a href="./nco.html#xmp_ncecat">examples</a>)</li> <li><a href="./nco.html#ncflint"><tt>ncflint</tt></a> netCDF File Interpolator (<a href="./nco.html#xmp_ncflint">examples</a>)</li> <li><a href="./nco.html#ncks"><tt>ncks</tt></a> netCDF Kitchen Sink (<a href="./nco.html#xmp_ncks">examples</a>)</li> <li><a href="./nco.html#ncpdq"><tt>ncpdq</tt></a> netCDF Permute Dimensions Quickly, Pack Data Quietly (<a href="./nco.html#xmp_ncpdq">examples</a>)</li> <li><a href="./nco.html#ncra"><tt>ncra</tt></a> netCDF Record Averager (<a href="./nco.html#xmp_ncra">examples</a>)</li> <li><a href="./nco.html#ncrcat"><tt>ncrcat</tt></a> netCDF Record Concatenator (<a href="./nco.html#xmp_ncrcat">examples</a>)</li> <li><a href="./nco.html#ncrename"><tt>ncrename</tt></a> netCDF Renamer (<a href="./nco.html#xmp_ncrename">examples</a>)</li> <li><a href="./nco.html#ncwa"><tt>ncwa</tt></a></td> netCDF Weighted Averager (<a href="./nco.html#xmp_ncwa">examples</a>)</li> </ul> <p>The operators are as general as <a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a> itself: there are no restrictions on the contents of the <a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a> file(s) used as input. NCO's internal routines are completely dynamic and impose no limit on the number or sizes of dimensions, variables, and files. NCO is designed to be used both interactively and with large batch jobs. The default operator behavior is often sufficient for everyday needs, and there are numerous command line (i.e., run-time) options, for special cases. NCO works well on most modern operating systems, including: Apple OS X, *BSD, Cray UNICOS, DEC Tru64, IBM AIX, HPUX, Linux, Microsoft Windows, NEC Super UX, SGI IRIX, and Sun Solaris.</p> <p><hr width="100%"></p> <!-- http://nco.sf.net#Contribute --> <dt><a name="Contribute"></a></dt> <h2>How to Contribute</h2> <p>In March, 2000, NCO became an Open Source project hosted by <a href="http://sf.net">SourceForge.net</a>. This facilitates collaboration, code contributions, and support. There is a <a href="./TODO">TODO list</a> about 200 items long! No matter what your programming background there is a task you can help with. From re-organizing the TODO list itself, to improving this cheesy webpage, to documentation, to designing and implementing new features and interfaces, we need your help! The <a href="http://sf.net/projects/nco">NCO project homepage</a> contains mailing lists, discussion forums, and instructions to make contributing to NCO easy. <p>As of June, 2003, you may donate money to NCO via Paypal. Your donation will be distributed to NCO developers as an incentive for them to tackle <a href="./TODO"><tt>TODO</tt></a> items on a pay-per-feature basis. <form action="https://www.paypal.com/cgi-bin/webscr" method="post"> <input type="hidden" name="cmd" value="_xclick"> <input type="hidden" name="business" value="zender@uci.edu"> <input type="hidden" name="item_name" value="NCO development"> <input type="hidden" name="item_number" value="nco_dnt_dvl"> <input type="hidden" name="no_note" value="1"> <input type="hidden" name="currency_code" value="USD"> <input type="hidden" name="tax" value="0"> <input type="image" src="https://www.paypal.com/images/x-click-but04.gif" border="0" name="submit" alt="Donate to NCO with PayPal - it's fast, free and secure!"> </form> These homepage donation counters track the influence of your monetary donations on NCO development: <dt>Donations received between 20030624 and 20070531: US$60.00.</dt> <dt>NCO features "incentivized" by these donations: None.</dt> <hr></p> <a name="nsf"></a> <!-- http://nco.sf.net#nsf --> <a name="sdo"></a> <!-- http://nco.sf.net#sdo --> <a name="sei"></a> <!-- http://nco.sf.net#sei --> <dt><a name="prp_sei"></a></dt> <!-- http://nco.sf.net#prp_sei --> <h2>NCO→SDO Project</h2> <img src="nsf.png" height=180 width=180 align=right> <p> The National Science Foundation <a href="http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0431203">NSF IIS-0431203</a> funds our <a href="http://dust.ess.uci.edu/prp/prp_sei/prp_sei.pdf">NCO→SDO Project</a>, “SEI(GEO): Scientific Data Operators Optimized for Efficient Distributed Interactive and Batch Analysis of Tera-Scale Geophysical Data” as part of the <a href="http://www.cise.nsf.gov/funding/pgm_display.cfm?pub_id=9473&div=iis">Science and Engineering Informatics (SEI)</a> program. We appreciate the proposal reviewers for and staff of the Divisions of <a href="http://www.cise.nsf.gov/div/index.cfm?div=iis">Information and Intelligent Systems (IIS)</a> and <a href="http://www.cise.nsf.gov/div/index.cfm?div=sci">Shared Cyberinfrastructure (SC)</a> in the <a href="http://www.nsf.gov/home/cise">Directorate for Computer and Information Science and Engineering (CISE)</a>. NCO development was completely voluntary and without institutional support from its inception until September 2004. Hence this NSF support has dramatically changed the scale and pace of NCO/SDO development. This URL, <a href="http://nco.sf.net#prp_sei"><tt>http://nco.sf.net#prp_sei</tt></a>, points to the most up-to-date information on the NCO/SDO proposal.</p> <p>The NSF project provides the human and technical resources to analyze, design, and implement novel advanced methods for distributed data analysis and reduction (DDRA) of geophysical data. These powerful methods will be “grid-enabled”, and we will demonstrate their efficacy with analyses of distributed climate prediction and observation datasets. On the human side, recruited full-time help for specific areas of SDO, alongside our current and future volunteers. We have a <a href="http://dust.ess.uci.edu/hire/prg_anl_03_adv.pdf">full-time programmer</a> to help design, implement, and release major code changes. We have a <a href="http://dust.ess.uci.edu/hire/sdn_grd_02_adv.pdf">graduate student researcher</a> pursuing innovative Ph.D. research in using <a href="http://www3.niu.edu/mpi">“grid-aware”</a> <a href="http://www.mpi-forum.org">distributed memory</a>, <a href="http://www.openmp.org">shared memory</a>, and <a href="http://opendap.org">client-server</a> software technologies to improve DDRA of climate datasets. Another graduate student helps us design, implement, and release major code changes. Contact us if you are seriously interested in tackling these problems on a full-time, part-time, or pay-per-feature basis.</p> <p>On the institutional and hardware side, this project connects the <a href="http://www.ess.uci.edu/esmf">Earth System Modeling Facility</a> (ESMF) to the <a href="http://www.calit2.net">California Institute for Telecommunications and Information Technology</a> (Cal-IT2) <a href="http://www.optiputer.net">OptIPuter</a> at the <a href="http://sdsc.ucsd.edu">San Diego Supercomputer Center</a>. These supercomputers will dedicate a TB of storage each to <a href="http://www.opendap.org">OPeNDAP</a>-served climate simulation datasets for DDRA. Once the proof-of-concept SDO DDRA is complete, we will attempt a DDRA intercomparison of the internationally distributed <a href="http://www.ccsm.ucar.edu/experiments/ccsm3.0">global climate change simulations</a> that reside on the <a href="http://www.earthsystemgrid.org">Earth System Grid</a>. We expect our analyses to yield otherwise undiscovered patterns among different climate prediction scenarios for the same model, and among different climate models.</p> <p>Finally we still encourage you to e-mail NCO/SDO endorsements to <a href="mailto:surname@uci.edu">Charlie “my surname is zender” Zender</a> with Subject: “NCO/SDO Proposal Endorsement”. An endorsement can be a sentence or a page, describing how NCO/SDO benefits your work or research. The information in your endorsement will be used to create supplementary documents to NSF and NASA for future NCO/SDO proposal advocacy. “What future proposals?” you ask, “Didn't you just get funded?” Yes, we are funded to optimize and “grid-enable” NCO. There are still many desirable and unexplored methods for distributed data analysis and reduction (DDRA) that would benefit geosciences. For example, have you tried analyzing multiple NASA satellite datasets in HDF5 format? Need I say more? If you send an endorsement, remember to include (at least) your Name, Title, and Institutional affiliation in the body.</p> </ul> <hr></p> <!-- http://nco.sf.net#pub --> <dt><a name="Publications"></a><a name="pub"></a><a name="pubs"></a></dt> <h2>Publications and Presentations</h2> <ul> <li><!-- Zen08 --> <a name="Zen08"></a> <!--http://www.ess.uci.edu/~zender#Zen08 --> Zender, C. S. (2008), netCDF Operators (NCO) for Analysis of Self-describing Gridded Geoscience Data, <i>In Press in Environ. Modell. Softw.</i>. <a href="http://dust.ess.uci.edu/ppr/ppr_Zen08.bib">BibTeX</a> <a href="http://dust.ess.uci.edu/ppr/ppr_Zen08.pdf">PDF (CSZ)</a></li> <li> <!-- WZJ074 --> <a name="WZJ074"></a> <!--http://www.ess.uci.edu/~zender#WZJ074 --> Wang, D. L., C. S. Zender and S. F. Jenks (2007): <i>A System for Scripted Data Analysis at Remote Data Centers</i>. Presented at the Fall Meeting of the American Geophysical Union, San Francisco, CA, December 10–14, 2007. <i>Eos Trans. AGU</i>, <b>88</b>(52), Fall Meet. Suppl., Abstract IN11B-0469. <a href="http://dust.ess.uci.edu/ppr/pst_WZJ074.bib">BibTeX</a> <a href="http://dust.ess.uci.edu/ppr/pst_WZJ074.pdf">PDF (DLW)</a> </li> <li> <!-- WZJ073 --> <a name="WZJ073"></a> <!--http://www.ess.uci.edu/~zender#WZJ073 --> Wang, D. L., C. S. Zender, and S. F. Jenks (2007), Server-side parallel data reduction and analysis, in <i>Advances in Grid and Pervasive Computing, Second International Conference, GPC 2007</i>, Paris, France, May 2–4, 2007, Proceedings. IEEE Lecture Notes in Computer Science, vol. 4459, edited by C. Cérin and K.-C. Li, pp. 744–750, Springer-Verlag, Berlin/Heidelberg. <a href="http://dust.ess.uci.edu/ppr/ppr_WZJ073.bib">BibTeX</a> <a href="http://dust.ess.uci.edu/ppr/ppr_WZJ073.pdf">PDF (DLW)</a> </li> <li><!-- WZJ07 --> <a name="WZJ07"></a> <!--http://www.ess.uci.edu/~zender#WZJ07 --> Wang, D. L., C. S. Zender, and S. F. Jenks (2007), <i>Server-side Parallel Data Reduction and Analysis</i>. Presented by D. Wang to the International Conference on Grid and Pervasive Computing (GPC), Paris, France, May 2–4, 2007. <a href="http://dust.ess.uci.edu/smn/smn_WZJ07_gpc_200705.pdf">PDF</a></li> <li><!-- ZeM07 --> <a name="ZeM07"></a> <!--http://www.ess.uci.edu/~zender#ZeM07 --> Zender, C. S., and H. J. Mangalam (2007), Scaling Properties of Common Statistical Operators for Gridded Datasets, <i>Int. J. High Perform. Comput. Appl, <b>21</b>(4), 485–498, doi:10.1177/1094342007083802</i>. <a href="http://dust.ess.uci.edu/ppr/ppr_ZeM07.bib">BibTeX</a> <a href="http://dust.ess.uci.edu/ppr/ppr_ZeM07.pdf">PDF (CSZ)</a> <a href="http://dust.ess.uci.edu/ppr/ppr_ZeM07_ijhpca.pdf">PDF (IJHPCA) (© 2007 by SAGE Publications)</a></li> <li><!-- WZJ07 --> <a name="WZJ07"></a> <!--http://www.ess.uci.edu/~zender#WZJ07 --> Wang, D. L., C. S. Zender, and S. F. Jenks (2007), <i>Server-side Data Reduction and Analysis with Script Workflow Analysis for Multi-Processing</i>. Presented by D. Wang to the OPeNDAP Developer's Workshop, Boulder, CO, February 21–23, 2007. <a href="http://dust.ess.uci.edu/smn/smn_WZJ07_opendap_200702.pdf">PDF</a></li> <li><!-- WZJ07 --> <a name="WZJ07"></a> <!--http://www.ess.uci.edu/~zender#WZJ07 --> Wang, D. L., C. S. Zender, and S. F. Jenks (2007), <i>Server-side Data Reduction and Analysis with Script Workflow Analysis for Multi-Processing</i>. Presented by D. Wang to the 23rd AMS Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, San Antonio, TX, January 14–18, 2007. <a href="http://dust.ess.uci.edu/smn/smn_WZJ07_ams_200701.pdf">PDF</a></li> <li><!-- WZJ07 --> <a name="WZJ07"></a> <!--http://www.ess.uci.edu/~zender#WZJ07 --> Wang, D. L., C. S. Zender, and S. F. Jenks (2007), DAP-enabled Server-side Data Reduction and Analysis, <i>Proceedings of the 23rd AMS Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology</i>, Paper 3B.2., January 14–18, San Antonio, TX. American Meteorological Society, AMS Press, Boston, MA. <a href="http://dust.ess.uci.edu/ppr/abs_xtn_WZJ07.bib">BibTeX</a> <a href="http://dust.ess.uci.edu/ppr/abs_xtn_WZJ07.pdf">PDF</a></li> <li><!-- ZeW07 --> <a name="ZeW07"></a> <!--http://www.ess.uci.edu/~zender#ZeW07 --> Zender, C. S., and D. L. Wang (2007), <i>High performance distributed data reduction and analysis with the netCDF Operators (NCO)</i>. Presented to the 23rd AMS Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, January 14–18, San Antonio, TX, January 14–18, 2007. <a href="http://dust.ess.uci.edu/smn/smn_nco_ams_200701.pdf">PDF</a></li> <li><!-- ZeW07 --> <a name="ZeW07"></a> <!--http://www.ess.uci.edu/~zender#ZeW07 --> Zender, C. S., and D. L. Wang (2007), High performance distributed data reduction and analysis with the netCDF Operators (NCO), <i>Proceedings of the 23rd AMS Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology</i>, Paper 3B.4., January 14–18, San Antonio, TX. American Meteorological Society, AMS Press, Boston, MA. <a href="http://dust.ess.uci.edu/ppr/abs_xtn_ZeW07.bib">BibTeX</a> <a href="http://dust.ess.uci.edu/ppr/abs_xtn_ZeW07.pdf">PDF</a></li> <li><!-- WZJ06 --> <a name="WZJ06"></a> <!-- http://dust.ess.uci.edu/ppr/pst_WZJ06.pdf --> Wang, D. , Zender, C. S., and S. F. Jenks (2006): <i>Server-side netCDF Data Reduction and Analysis</i>. Presented by D. Wang to the Fall Meeting of the American Geophysical Union, San Francisco, CA, December 5–9, 2006. <i>Eos Trans. AGU</i>, <b>87</b>(52), Fall Meet. Suppl., Abstract IN53B-0826. <a href="http://dust.ess.uci.edu/ppr/pst_WZJ06.pdf">PDF</a> <li><!-- ZMW06 --> <a name="ZMW06"></a> <!-- http://dust.ess.uci.edu/ppr/pst_ZMW06.pdf --> Zender, C. S., H. J. Mangalam, and D. L. Wang (2006): <i>Improving Scaling Properties of Common Statistical Operators for Gridded Geoscience Datasets</i>. Presented to the Fall Meeting of the American Geophysical Union, San Francisco, CA, December 5–9, 2006. <i>Eos Trans. AGU</i>, <b>87</b>(52), Fall Meet. Suppl., Abstract IN53B-0827. <a href="http://dust.ess.uci.edu/ppr/pst_ZMW06.pdf">PDF</a> </ul> <hr></p> <!-- http://nco.sf.net#vrs --> <dt><a name="Version"></a><a name="vrs"></a></dt> <h2>Version Summary</h2> New releases receive unique CVS tags and their tarballs are stored <a href="http://dust.ess.uci.edu/nco/src">here</a>. Source and binaries no longer debut on SourceForge (because SourceForge's upload mechanism is too time-consuming). Instead, retrieve all recently tagged versions <a name="#Download">with CVS</a> or from <a href="http://dust.ess.uci.edu/nco/src">this HTTP site</a>. <ul> <li><b>NCO/SDO 3.9.6</b>: (<i>In progress</i>) Memory management to improve threading; <li><b>NCO/SDO 3.9.5</b>: Multi-slabbing everywhere; <tt>ncecat -M</tt>; netCDF4-enabled builds on Cygwin; <tt>ncap2</tt> builds on Solaris; <li><b>NCO/SDO 3.9.4</b>: File consanguinity; <tt>ncks</tt>: CF auxiliary coords, LZ stats; <tt>ncap2</tt>: masks, index arrays, print attributes, OpenMP, <tt>atan2()</tt>, <tt>pow()</tt>; <li><b>NCO/SDO 3.9.3</b>: SWAMP release; <tt>ncecat -u</tt>; <tt>wget</tt> URLs; GCC 4.2; <tt>ncap2</tt> OpenMP; <li><b>NCO/SDO 3.9.2</b>: <tt>_FillValue</tt> supplants <tt>missing_value</tt>; <tt>UDUnits2 support</tt>; <tt>ncap2</tt> type-conversion support and printing; netCDF4 <tt>configure</tt> fixes; Change licenses to GPL3 and FDL1.2; <li><b>NCO/SDO 3.9.1</b>: <a href="http://nco.sf.net/nco.html#deflate">Support</a> lossless netCDF4/HDF deflation; Improve Intel compiler support; <tt>ncap2</tt> for/while loops; various and sundry build fixes; <li><b>NCO/SDO 3.9.0</b>: <a href="http://nco.sf.net/nco.html#nco4">Support netCDF4</a> types (<tt>ubyte</tt>, <tt>ushort</tt>, <tt>uint</tt>, <tt>int64</tt>, and <tt>uint64</tt>); Control <tt>ncks</tt> metadata <a href="http://nco.sf.net/nco.html#prn">copying</a>; Fix <tt>ncbo</tt> packed valued <a href="#bug_ncbo_pck">bug</a> (<i>upgrade recommended</i>); Improve IRIX 6.5 support; <li><b>NCO/SDO 3.2.0</b>: More forgiving exclusion list (<tt>-x -v <i>var_nm</i></tt>) <a href="http://nco.sf.net/nco.html#xcl">handling</a>; Fix <tt>rmssdn</tt> normalization; Support Mac OS X on Intel; Documentation bugfixes; <li><b>NCO/SDO 3.1.9</b>: <tt>ncap2</tt> in RPM packages; Quieter output; AIX <tt>configure</tt> supports <tt>ncap2</tt>; Update to DAP for netCDF 3.6.2; Fix <tt>nc[erw]a</tt> for coordinate min/max/ttl; <tt>ncecat</tt> allows files to differ in record dimension size; <li><b>NCO/SDO 3.1.8</b>: Support <tt>_FillValue</tt> with compile-time switch; Debian package synchronized, includes <tt>ncap2</tt> <li><b>NCO/SDO 3.1.7</b>: <tt>ncap2</tt> “double-parsing”, array initialization, supercedes <tt>ncap</tt> <li><b>NCO/SDO 3.1.6</b>: Support <a href="http://nco.sf.net/nco.html#srd">stride</a> in all hyperslabbing operators; change more WARNINGs to INFOs <li><b>NCO/SDO 3.1.5</b>: New <tt>ncap2</tt> array and hyperslab features; change some WARNINGs to INFOs, add Pathscale and update PGI and Intel compiler support <li><b>NCO/SDO 3.1.4</b>: Fix <tt>ncbo</tt> memory problem; report timer results <li><b>NCO/SDO 3.1.3</b>: <tt>ncap2</tt> automatically <a href="http://nco.sf.net/nco.html#att_prp">propagates</a> attributes to new/derived variables; NCO handles record variables with zero records; <a href="http://nco.sf.net/nco.html#cnv_CF_crd">support</a> CF <tt>coordinates</tt> convention; <tt>ncwa -b</tt> <a href="http://nco.sf.net/nco.html#ncwa">retains</a> degenerate dimensions; build fixes for Cray T3E, Solaris, Cygwin. <li><b>NCO/SDO 3.1.2</b>: <tt>ncap2</tt> control structures (conditionals, loops, etc.); keep non-processed data <a href="bug_pack_fix">packed</a> in output file; binary operators <tt>ncbo</tt> and <tt>ncflint</tt> work when only one variable has <tt>missing_value</tt> attribute; change <tt>ncra</tt> and <tt>ncea</tt> to always average coordinate variables <li><b>NCO/SDO 3.1.1</b>: NCO builds (again) with C++ compilers; ncap2 development code; server-side extensions <li><b>NCO/SDO 3.1.0</b>: MPI operators are bug-free; fix <t>./configure</t> for MPI and netCDF4; MRV algorithm speeds up <tt>ncwa</tt> 5—10 times; <tt>ncap</tt> random number generator <tt>rnd_nbr()</tt> <li><b>NCO 3.0.3</b>: Eliminate all critical read regions (accelerates all SMP operators); builds with netCDF4 alpha; read <t>NETCDF4_CLASSIC</t> (i.e., HDF) files automatically; write <t>NETCDF4</t> files <a href="http://nco.sf.net/nco.html#netcdf4">on request</a>; <a href="http://nco.sf.net#BM">benchmark results</a> quantify parallelism. <li><b>NCO 3.0.2</b>: Fewer critical read regions; all known memory leaks plugged; better Intel compiler support; <tt>SFTP</tt> protocol support; FTP protocol supports <tt>.netrc</tt>-based passwords; better model grid-property-awareness (e.g., <tt>lat_bnds</tt>); <tt>ncatted</tt> <tt>ncks</tt>, <tt>ncrename</tt> support metadata header padding with <tt>--hdr</tt>; <tt>ncbo</tt> supports heterogeneous variable ordering; <tt>ncatted</tt> bugfix <li><b>NCO 3.0.1</b>: benchmarks; <tt>ncks</tt> printing tweaks; memory cleanup; <tt>ncap</tt> double-parse speedup and <a href="bug_ncap_cmt">exponentiation bugfix</a>; thread operators by default <li><b>NCO 3.0.0</b>: New <tt>ncks -P</tt> print switch; <tt>ncap</tt> optimizations; <tt>-Z</tt> option supports 64-bit offsets (requires netCDF 3.6); NEC SX build improvements; support GCC 3.4.X; <tt>nco_put_vara()</tt> overloads in <tt>libnco_c++</tt> <li><b>NCO 2.9.9</b>: <tt>ncpdq</tt> packs/unpacks entire files; <tt>ncbo</tt> is threaded <li><b>NCO 2.9.8</b>: <a href="./nco.html#ncpdq"><tt>ncpdq</tt></a> released; <tt>ncwa --msk_sng</tt> consolidated mask switch; C99 compiler required <li><b>NCO 2.9.7</b>: Read input file lists from <tt>stdin</tt>; write <tt>nco_input_file_list</tt> attributes; add <tt>nco_put_att<int,short,long></tt> functions to libnco_c++; large file support (LFS); write <tt>nco_openmp_thread_number</tt> attribute <li><b>NCO 2.9.6</b>: Support <tt>-o fl_out</tt> syntax; 64-bit ABI for x86_64; <a href="http://www.debian.org">Debian</a> builds libnco_c++ <li><b>NCO 2.9.5</b>: <tt>ncflint</tt> handles <tt>missing_value</tt> data better, is commutative when weights are equal <li><b>NCO 2.9.4</b>: <tt>ncecat</tt> works on pure scalar files; change to <tt>cvs.sf.net</tt>; AIX GCC builds <li><b>NCO 2.9.3</b>: <tt>ncra</tt> and <tt>ncea</tt> now <a name="bug_mss_val_zro_ncra">work correctly</a> when <tt>missing_value</tt> equals zero or any intermediate sums <li><b>NCO 2.9.2</b>: <tt>ncap</tt> <tt>S/V</tt>, <tt>S%V</tt>, and <tt>S^V</tt> <a href="#bug_ncap_cmt">operations fixed</a> (<tt>S</tt> = scalar, <tt>V</tt> = variable) <li><b>NCO 2.9.1</b>: <tt>ncwa</tt> works again on packed <tt>NC_FLOAT</tt> data (<a href="#bug_pck_ncwa">broke when?</a>) <li><b>NCO 2.9.0</b>: Complete <tt>ncap</tt> modulo operator (<tt>V%S</tt>) and generic exponentiation (<tt>V^S</tt>), scalar divided by variable (<tt>S/V</tt>); workaround stack memory problem with GCC extensions; fix <tt>make check</tt> <li><b>NCO 2.8.8</b>: <tt>--enable-[optimize/debug]-custom</tt> options for GCC work again (broke in 2.8.6) <li><b>NCO 2.8.7</b>: <tt>ncap</tt> packing works again (<a href="#bug_pck">broke in 2.8.4</a>); Cygwin builds work again <li><b>NCO 2.8.6</b>: AIX and IRIX builds work again; custom GCC builds with <tt>-Werror</tt> <li><b>NCO 2.8.5</b>: Solaris <tt>make install</tt> works; <tt>ncap</tt> changes attribute values correctly; <a href="#bld_DODS">Support DODS 3.4+, deprecate 3.3-</a>; add <tt>--enable-[optimize/debug]-custom</tt> options to <tt>./configure</tt> <li><b>NCO 2.8.4</b>: Index-based hyperslabs work again (<a href="#bug_idx_hyp">broke in 2.7.3</a>, <i>upgrade highly recommended</i>); <tt>ncap</tt> peak memory usage reduced still more; support for AMD Opteron x86_64 architecture <li><b>NCO 2.8.3</b>: Fix big <tt>ncap</tt> memory leak; fix regressions; remove scary packing warnings</li> <li><b>NCO 2.8.2</b>: <tt>ncbo</tt> handles distinct <tt>missing_value</tt>s correctly; improve error diagnostics</li> <li><b>NCO 2.8.1</b>: <a href="./nco.html#rx">“Variable wildcarding”</a>—extended regular expressions in <tt>-v var,...</tt> lists—support in all operators</li> <li><b>NCO 2.8.0</b>: New “binary” operator <a href="./nco.html#ncbo"><tt>ncbo</tt></a>, obsoletes <tt>ncdiff</tt>; <a href="http://www.debian.org">Debian</a> integration</li> <li><b>NCO 2.7.4</b>: Autotools build support for AIX; fix UDUnits confusion with floating point hyperslab specifications</li> <li><b>NCO 2.7.3</b>: Support <a href="http://www.unidata.ucar.edu/packages/udunits">UDUnits</a> (and some <a href="http://www.cgd.ucar.edu/cms/eaton/cf-metadata/CF-working.html">CF</a>) conventions, e.g., <tt>ncks -d wvl "1 picometer","1 furlong" -d time,"2001-03-19 06:00:0.0","2001-03-19 18:00:0.0"</tt></li> <li><b>NCO 2.7.2</b>: Printing (<tt>ncks -H</tt>) plays well with multi-slabbing</li> <li><b>NCO 2.7.1</b>: (Packaged) Fallback to B. Sittler's <tt>getopt()</tt> for long option support; store fixed, packed variables correctly with <tt>ncap</tt>.</li> <li><b>NCO 2.7.0</b>: Support for GNU/POSIX long options; <tt>ncks</tt> introduces <i>multi-slabbing</i> (= multiple hyperslabs) capability</li> <li><b>NCO 2.6.6</b>: Fix and document <tt>ncrename</tt> attribute renaming; add type conversion functions to <tt>ncap</tt></li> <li><b>NCO 2.6.5</b>: Cray, Sun build improvements</li> <li><b>NCO 2.6.4</b>: Mac OS X, NEC, SGI build improvements</li> <li><b>NCO 2.6.3</b>: Mac OS X builds work seamlessly; documentation fixed</li> <li><b>NCO 2.6.2</b>: Major improvements in autotools support; <tt>ncap</tt> supports user-defined dimensions</li> <li><b>NCO 2.6.1</b>: Minor fix to unpacking support</li> <li><b>NCO 2.6.0</b>: All arithmetic operators (<tt>ncap</tt>, <tt>ncdiff</tt>, <tt>ncea</tt>, <tt>ncflint</tt>, <tt>ncra</tt>) support unpacking</li> <li><b>NCO 2.5.6</b>: Handle <tt>missing_value</tt> correctly for integer variables in <tt>ncra</tt>, merge <tt>autoconf</tt> build</li> <li><b>NCO 2.5.5</b>: Handle files without existing global “<tt>History</tt>” attribute correctly</li> <li><b>NCO 2.5.4</b>: Add support NEC-SX support; update Cray support; small build changes</li> <li><b>NCO 2.5.3</b>: Fix native builds under MS Windows with Cygwin <tt>ncap</tt>; hint to <tt>ncwa</tt></li> <li><b>NCO 2.5.2</b>: Add math functions to <tt>ncap</tt>; hint to <tt>ncwa</tt></li> <li><b>NCO 2.5.1</b>: <tt>ncatted</tt> deletes all attributes of <tt>att_var</tt> when <tt>att_nm</tt> not specified</li> <li><b>NCO 2.5.0</b>: man(1) pages from <a href="http://www.debian.org">Debian</a> distribution</li> <li><b>NCO 2.4.6</b>: Free (speech) sort routines</li> <li><b>NCO 2.4.5</b>: Arithmetic speedups</li> <li><b>NCO 2.4.4</b>: 64-bit AIX builds</li> <li><b>NCO 2.4.3</b>: Handle non-standard “History” attributes</li> <li><b>NCO 2.4.2</b>: <tt>ncap</tt> supports nesting files with <tt>#include</tt></li> <li><b>NCO 2.4.0</b>: <tt>ncks</tt> supports writing native binary output with <tt>-b</tt> switch</li> <li><b>NCO 2.3.1</b>: <tt>ncap</tt> builds on all Unices</li> <li><b>NCO 2.3.0</b>: Library redesign</li> <li><b>NCO 2.2.4</b>: <tt>ncap</tt> fully supports packing and unpacking variables</li> <li><b>NCO 2.2.3</b>: <tt>ncap</tt> supports manually unpacking variables stored with <tt>scale_factor</tt> and <tt>add_offset</tt></li> <li><b>NCO 2.2.2</b>: fix rare <tt>ncwa</tt> crash with weights</li> <li><b>NCO 2.2.1</b>: fix <tt>ncap</tt> divide and <tt>fmodf</tt> bugs</li> <li><b>NCO 2.2.0</b>: (RPM available) <a href="./nco_news.shtml#20020203">20020203</a>: <tt>ncap</tt> public debut (has division error, please upgrade)</li> <li><b>NCO 2.1.3</b>: <tt>ncap</tt> implements LHS casting</li> <li><b>NCO 2.1.2</b>: C++ API to netcdf; libnco_c++ open for business</li> <li><b>NCO 2.1.0</b>: <tt>ncap</tt> full lexer/parser operational</li> <li><b>NCO 2.0.3</b>: small fixes to 2.0.0 (e.g., <tt>ncks</tt> printing)</li> <li><b>NCO 2.0.2</b>: Add packing/unpacking capabilities based on <tt>scale_factor</tt> and <tt>add_offset</tt> convention</li> <li><b>NCO 2.0.1</b>: Add user-configurable multi-threaded capability to arithmetic-intensive operators <tt>ncwa</tt>, <tt>ncea</tt>, and <tt>ncra</tt></li> <li><b>NCO 2.0.0</b>: (RPM available) <a href="./nco_news.shtml#20010507">20010507</a>: Switched all internal calls to netCDF 3.x API</li> <li><b>NCO 1.2.2</b>: <a href="./nco_news.shtml#20010211">20010211</a>: Allow weights with <tt>ncwa</tt> in min/max/ttl operations; easier building with HDF4. <tt>ncrename</tt> can now work on specific variables rather than whole file. <tt>ncks</tt> has <tt>-q</tt> option for cleaner printing</li> <li><b>NCO 1.2.1</b>: <a href="./nco_news.shtml#20000828">20000828</a>: Added type-conversion of integers to floating point before arithmetic; fixed broken <tt>-v</tt> option</li> <li><b>NCO 1.2</b>: <a href="./nco_news.shtml#20000730">20000730</a>: Added non-linear operations to <tt>ncra/ncea</tt> and <tt>ncwa</tt>: min,max,total,rms,rmssdn,sqrt. Added <a href="http://opendap.org">DODS</a> compliance.</li> <li><b>NCO 1.1.48</b>: <a href="./nco_news.shtml#20000515">20000515</a>: Potentially serious bugfix to <tt>ncdiff</tt>, recommend upgrade</li> <li><b>NCO 1.1.47</b>: <a href="./nco_news.shtml#20000510">20000510</a>: Improved <tt>ncra</tt> and <tt>ncrcat</tt> to allow lazier specifications of input files</li> </ul> <hr></p> <!-- http://nco.sf.net#dwn --> <dt><a name="Download"></a><a name="dwn"></a></dt> <h2>Download/CVS NCO Software</h2> <a name="src"></a> <h3>NCO Source Code</h3> The best way to acquire NCO sources is with <a href="http://www.cvshome.org">CVS</a>. The second best way is to download the source and documentation as a compressed tarfile: <ul> <li> <a href="nco.tar.gz">nco.tar.gz</a> (<!--#fsize file="nco.tar.gz"--> compressed tar-file)<br> <!--#exec cmd="openssl dgst -md5 nco.tar.gz"--><br> <!--#exec cmd="openssl dgst -sha1 nco.tar.gz"--> </li> </ul> <p>NCO's browsable <a href="http://nco.cvs.sf.net/nco/nco">CVS Repository</a> contains up-to-the-minute sources and is the easiest way to stay synchronized with NCO features. Retrieving NCO requires some familiarity with GNU development tools, especially <a href="http://www.cvshome.org">CVS</a> and <a href="http://www.gnu.org/software/make/make.html">Make</a>. Sourceforge provides <a href="http://sf.net/cvs/?group_id=3331">generic instructions</a> for accessing their CVS servers. First log into the NCO CVS server: <p><tt>cvs -d:pserver:anonymous@nco.cvs.sf.net:/cvsroot/nco login</tt></p> Hit <tt>return</tt> when asked for the CVS password. This login procedure is only required the first time you access the CVS server. Then you may retrieve any NCO distribution you wish. Usually you wish to retrieve a recent tagged (i.e., released) version. This command retrieves and places NCO version <tt>3.9.5</tt> (which is tagged as <tt>nco-3_9_5</tt> due to CVS rules) into local directory <tt>nco-3.9.5</tt>: <p><tt>cvs -z3 -d:pserver:anonymous@nco.cvs.sf.net:/cvsroot/nco co -r nco-3_9_5 -d nco-3.9.5 nco</tt></p> This command retrieves the current (“bleeding edge”) development version of NCO into a local directory named <tt>nco</tt>: <p><tt>cvs -z3 -d:pserver:anonymous@nco.cvs.sf.net:/cvsroot/nco co -kk nco</tt></p> Track changes to the development version using <p><tt>cd nco;cvs update -kk</tt></p> One difference between running a "tagged" release (e.g., <tt>nco-3_9_5</tt>) and the development version is that the tagged release operators will print a valid version number (e.g., <tt>3.9.5</tt>) when asked to do so with the <tt>-r</tt> flag (e.g., <tt>ncks -r</tt>). The development version simply places today's date in place of the version. Once the autotools builds are working more robustly, the confusion over versions should largely disappear. <a name="doxy"></a> <h3>Developer NCO Source Documentation</h3> <p>Automated source documentation, created by the <a href="http://www.stack.nl/~dimitri/doxygen/">Doxygen</a> tool is available. Some developers find this documentation helpful, as it can clarify code and data relationships in the code.</p> <ul> <li><a href="http://sand.ess.uci.edu/nco/doxygen/main/files.html"> Source documentation for NCO and NetCDF4(alpha13)</a></li> </ul> <p>Please be aware that the documentation may be slightly inaccurate and infrequently updated. Comments and suggestions are certainly welcomed on the <a href="http://sf.net/forum/forum.php?forum_id=9829">forums</a>. </p> <a name="bnr"></a> <h3>NCO Binaries and OS-specific Packages</h3> <p>NCO developers are too short-handed to provide pre-built binary executables for all platforms. We do attempt to provide (theoretically) platform-independent sources in the most common UNIX packaging formats (Debian and RPM). Moreover, the source tarballs are always up-to-date, and work on our development systems (<a href="http://www.ubuntu.com">Ubuntu</a>-flavored <a href="http://www.debian.org">Debian</a> GNU/Linux for x86 and x86_64) and in our high-end scientific computing environments (<a href="http://www-1.ibm.com/servers/aix">AIX 5</a> for Power4/5). Below there are also links to binaries and packages created by volunteers and organizations who port NCO to other platforms. We welcome volunteers willing to perform regular regression testing and porting of NCO to other platforms. <a name="debian"></a><a name="dbn"></a> <!-- http://nco.sf.net#debian --> <a name="ubuntu"></a><a name="ubn"></a> <!-- http://nco.sf.net#ubuntu --> <h3><a href="http://www.debian.org">Debian</a> and <a href="http://www.ubuntu.com">Ubuntu</a> GNU/Linux</a></h3> <ul> <!-- <li><a href="http://www.debian.org">Debian</a> and <a href="http://www.ubuntu.com">Ubuntu</a> GNU/Linux-compatible Intel systems, prebuilt binary executable <a href="http://www.debian.org">deb</a>: <a href="http://packages.debian.org/testing/math/nco.html">nco-3.9.5</a></li> --> <a href="http://packages.debian.org/unstable/math/nco.html">Debian</a> and <a href="http://packages.ubuntulinux.org/gutsy/source/nco">Ubuntu</a> GNU/Linux homepages for NCO. ‘<tt>apt-get install nco</tt>’ installs the standard NCO for your Debian-compatible OS. <dt>Debian package for most recent NCO release (install with, e.g., ‘<tt>dpkg --install nco_3.9.5-1_i386.deb</tt>’):</dt> <li><a href="src/nco_3.9.5-1_amd64.deb">nco_3.9.5-1_amd64.deb</a> (<!--#fsize file="src/nco_3.9.5-1_amd64.deb"-->): Executables AMD64-compatible (last updated <!--#flastmod file="src/nco_3.9.5-1_amd64.deb"-->)</li> <li><a href="src/nco_3.9.5-1_i386.deb">nco_3.9.5-1_i386.deb</a> (<!--#fsize file="src/nco_3.9.5-1_i386.deb"-->): Executables i386-compatible (last updated <!--#flastmod file="src/nco_3.9.5-1_i386.deb"-->)</li> <li><a href="src/nco_3.9.5-1.dsc">nco_3.9.5-1.dsc</a> (<!--#fsize file="src/nco_3.9.5-1.dsc"-->): Description (last updated <!--#flastmod file="src/nco_3.9.5-1.dsc"-->)</li> <li><a href="src/nco_3.9.5-1_i386.changes">nco_3.9.5-1_i386.changes</a> (<!--#fsize file="src/nco_3.9.5-1_i386.changes"-->): Changes since last deb package (last updated <!--#flastmod file="src/nco_3.9.5-1_i386.changes"-->)</li> <li><a href="src/nco_3.9.5.orig.tar.gz">nco_3.9.5.orig.tar.gz</a> (<!--#fsize file="src/nco_3.9.5.orig.tar.gz"-->): Upstream Source (last updated <!--#flastmod file="src/nco_3.9.5.orig.tar.gz"-->)</li> <li><a href="src/nco_3.9.5-1.diff.gz">nco_3.9.5-1.diff.gz</a> (<!--#fsize file="src/nco_3.9.5-1_i386.changes"-->): Debian patch to upstream source (last updated <!--#flastmod file="src/nco_3.9.5-1_i386.changes"-->)</li> Thanks to Daniel Baumann, Francesco Lovergine, Brian Mays, Rorik Peterson, and Matej Vela for packaging NCO for Debian over the years. </ul> <a name="redhat"></a><a name="rhl"></a><a name="rpm"></a> <!-- http://nco.sf.net#redhat --> <!-- http://nco.sf.net#rhl --> <!-- http://nco.sf.net#rpm --> <a name="fedora"></a><a name="fdr"></a><a name="fc"></a> <!-- http://nco.sf.net#fedora --> <!-- http://nco.sf.net#fdr --> <!-- http://nco.sf.net#fc --> <a name="centos"></a><a name="cnt"></a><a name="rhel"></a> <!-- http://nco.sf.net#centos --> <!-- http://nco.sf.net#cnt --> <!-- http://nco.sf.net#rhel --> <h3><a href="http://fedora.redhat.com">Fedora</a> and <a href="http://www.centos.org">CentOS</a> GNU/Linux</h3> <ul> <dt>Our most recent NCO RPMs (install with, e.g., ‘<tt>yum install nco-3.9.5-1.fc7.i386.rpm</tt>’):</dt> <li><a href="src/nco-3.9.5-1.fc7.i386.rpm">nco-3.9.5-1.fc7.i386.rpm</a> (<!--#fsize file="src/nco-3.9.5-1.fc7.i386.rpm"-->): Executables for i386/Fedora Core 7-compatible environments (last updated <!--#flastmod file="src/nco-3.9.5-1.fc7.i386.rpm"-->)</li> <li><a href="src/nco-3.9.5-1.fc7.src.rpm">nco-3.9.5-1.fc7.src.rpm</a> (<!--#fsize file="src/nco-3.9.5-1.fc7.src.rpm"-->): Source (last updated <!--#flastmod file="src/nco-3.9.5-1.fc7.src.rpm"-->)</li> <li><a href="src/nco-3.9.5-1.x86_64.rpm">nco-3.9.5-1.x86_64.rpm</a> (<!--#fsize file="src/nco-3.9.5-1.x86_64.rpm"-->): Executables for x86_64/CentOS 4-compatible environments (last updated <!--#flastmod file="src/nco-3.9.5-1.x86_64.rpm"-->)</li> <li>More information on x86_64 RPMs available <a href="http://dust.ess.uci.edu/~wangd/s9y">here</a></li> NCO is part of the Fedora Extras distribution (thanks to Ed Hill). As such, the NCO <a href="http://www.rpm.org">RPMs</a> in the current Fedora Core release lag our upstream RPMs by no more than 6–8 months. Searching <a href="http://rpmfind.net">rpmfind.net</a> for NCO yields these <a href="http://rpmfind.net/linux/rpm2html/search.php?query=nco">NCO RPMs</a>: <li><a href="http://download.fedora.redhat.com/pub/fedora/linux/extras/6/i386/repodata/repoview/nco-0-3.1.5-3.fc6.html">nco-0-3.1.5-3.fc6.html</a>: Official Fedora Core 6 NCO binary package (contains all executables) homepage</li> <li><a href="http://download.fedora.redhat.com/pub/fedora/linux/extras/6/i386/repodata/repoview/nco-devel-0-3.1.5-3.fc6.html">nco-devel-0-3.1.5-3.fc6.html</a>: Official Fedora Core 6 NCO development package (contains libraries and headers) homepage</li> </ul> <a name="gentoo"></a> <a name="gnt"></a> <h3><a href="http://www.gentoo.org">Gentoo</a> GNU/Linux</h3> <ul> <a href="http://packages.gentoo.org/packages/?category=sci-misc;name=nco">Gentoo GNU/Linux homepage for NCO</a>. <a href="http://www.gentoo.org/doc/en/portage-manual.xml">Portage packages</a> by George Shapavalov and Patrick Kursawe: <li><a href="http://packages.gentoo.org/ebuilds/?nco-3.2.0">nco-3.2.0</a>: Latest(?) Gentoo package</li> </ul> <a name="mac"></a> <!-- http://nco.sf.net#mac --> <a name="darwin"></a> <!-- http://nco.sf.net#darwin --> <a name="drw"></a> <!-- http://nco.sf.net#drw --> <a name="macosx"></a> <!-- http://nco.sf.net#macosx --> <a name="osx"></a> <!-- http://nco.sf.net#osx --> <a name="macports"></a> <!-- http://nco.sf.net#macports --> <a name="fink"></a> <!-- http://nco.sf.net#fink --> <h3><a href="http://www.apple.com/macosx/">Mac OS X/Darwin</a></h3> <ul> <li><a href="http://fink.sf.net">Fink</a> packages for <a href="http://fink.sf.net/pdb/package.php/nco">NCO</a>: Currently NCO 3.9.2. Maintained by Alexander Hansen.</li> <li><a href="http://trac.macosforge.org/projects/macports/wiki">MacPorts</a> infrastructure for <a href="http://svn.macports.org/repository/macports/trunk/dports/science/nco">NCO</a>: <a href="http://svn.macports.org/repository/macports/trunk/dports/science/nco">Portfile</a> for NCO 3.9.3. Maintained by Takeshi Enomoto.</li> </ul> <hr></p> <!-- http://nco.sf.net#RTFM --> <dt><a name="RTFM"></a></dt> <h2>Documentation and User's Guide</h2> <p>View the NCO User's Guide in any of these formats: <ul> <li><a href="./nco.dvi">DVI</a> Device Independent (<tt>kdvi</tt>, <tt>xdvi</tt>)</li> <li><a href="./nco.html">HTML</a> Hypertext (any browser)</li> <li><a href="./nco.info">Info</a> GNU Info (<tt>M-x Info</tt>, <tt>emacs</tt>)</li> <li><a href="./nco.pdf">PDF</a> Portable Document Format (<tt>acroread</tt>, <tt>evince</tt>, <tt>kpdf</tt>, <tt>xpdf</tt>)</li> <li><a href="./nco.ps">Postscript</a> Printing (<tt>ghostview</tt>, <tt>kghostview</tt>)</li> <li><a href="./nco.texi">TeXInfo</a> Documentation Source code (<tt>emacs</tt>)</h2></li> <li><a href="./nco.txt">Text</a> Plain text (<tt>more</tt>)</h2></li> <li><a href="./nco.xml">XML</a> Extensible Markup Language (<tt>firefox</tt>)</h2></li> </ul> <tt>nco.texi</tt> is the most up-to-date. Files <tt>nco.dvi</tt>, <tt>nco.ps</tt>, and <tt>nco.pdf</tt> are the most complete and contain all the mathematical formulae (typeset with TeX) missing from the screen-oriented formats. The screen-oriented formats—<tt>nco.html</tt>, <tt>nco.info</tt>, <tt>nco.txt</tt>, and <tt>nco.xml</tt>—contain all the documentation except the highly mathematical sections. <h3>Other documentation:</h3> <ul> <li>This <a href="http://dust.ess.uci.edu/doc/abb/abb.pdf">abbreviation key</a> unlocks the mysteries of the source code abbreviations and acronyms.</li> <li>Unidata netCDF for Developers Workshop <a href="http://www.unidata.ucar.edu/software/netcdf/docs/workshop/utilities/Nco.html">NCO examples</a></li> </ul> <!-- This documentation went MIA sometime in 2004: <a href="http://www.cgd.ucar.edu/csm/support/Document/pdf/nco.pdf">pamphlet</a> and <a href="http://www.cgd.ucar.edu/csm/support/Document/shows/nco/nco_files/v3_document.htm">slide presentation</a> introduce and summarize the NCO operators. They were created by Sylvia Murphy of the NCAR <a href="http://www.cgd.ucar.edu/csm/cmps/cmps_home.html">Community Climate System Model Support Network</a>. --> <hr></p> <!-- http://nco.sf.net#README --> <dt><a name="README/ChangeLog/TODO"></a></dt> <h2>README/ChangeLog/TODO</h2> <p>Files containing useful information about the current NCO distribution: <ul> <li><a href="./README"><tt>README</tt></a> Platforms and software requirements for a successful NCO installation</li> <li><a href="./ChangeLog"><tt>ChangeLog</tt></a> Up-to-date history of NCO changes</li> <li><a href="./TODO"><tt>TODO</tt></a> An unordered list describing the features and fixes we currently have in mind for NCO.</li> </ul> <hr></p> <!-- http://nco.sf.net#FAQ --> <dt><a name="FAQ"></a></dt> <h2>FAQ: Frequently Asked Questions</h2> <p>These questions show up almost as frequently as my mother. But they are more predictable: <ul> <li><i>I still have questions, how do I contact the NCO project?</i> The NCO project has various Q&A and discussion forums described <a href="#Contact">below</a>. </li> <li><i>Where can I find prebuilt NCO executables?</i> Pre-built executables of some versions of NCO for the operating systems <a href="#Download">described above</a> (Debian-compatible GNU/Linux, Fedora/RedHat GNU/Linux, Gentoo GNU/Linux, and Mac OS X). If you have an account at UCI or SDSC, try the locations in my personal home directories listed below (AIX, IRIX). At NCAR, first try <tt>/usr/local/bin</tt> (AIX, IRIX). Otherwise, you are on your own.</li> <a name="NCAR"></a> <!-- http://nco.sf.net#NCAR --> <a name="ncar"></a> <!-- http://nco.sf.net#ncar --> <li><i>Does NCAR support NCO?</i> The NCAR CISL Technical Consulting Group (TCG) supports NCO like other community software packages such as <tt>lapack</tt>. No other NCAR division has coordinated NCO support. The NCAR CISL-suported executables are in <tt>/usr/local/bin</tt>. If you notice problems with the NCO installation on SCD machines, or if you would benefit from a more recent release or patch, then inform the relevant system administrators, e.g., <a href="mailto:consult1@ncar.ucar.edu">NCAR User Support</a>. If you have a comment, suggestion, or bug report, then contact the developers as described below. </li> <li><i>Is there an easy way to keep up with new NCO releases?</i> Subscribe to the <a href="http://lists.sf.net/mailman/listinfo/nco-announce"><tt>nco-announce</tt></a> mailing list. This list is for NCO-related announcements, not for questions. <tt>nco-announce</tt> is very low volume, averaging one message every few months.</li> </ul> <hr></p> <!-- http://nco.sf.net#HPCC --> <dt><a name="NCAR"></a></dt> <h2>Using NCO at UCI, NCAR, and other High Performance Computing Centers (HPCCs)</h2> <p>Users at UC Irvine, NCAR, and SDSC <i>may</i> find pre-built, <i>almost</i> up-to-date NCO executables in the following locations. These executables are <i>unsupported</i> and are almost <i>guaranteed</i> to hail from different NCO versions and thus to behave slightly differently. (They are built from the main trunk, not a tagged version, and currently it is impossible to tell exactly what version they are). Moreover they may be compiled with debugging enabled and thus may be <i>slower</i> than executables optimized for production use. Users at NCAR should try the <i>supported</i> executables in <tt>/usr/local/bin</tt> first. <ul> <li>ESS <tt>soot.ess.uci.edu</tt> (Linux 2.6.19): <tt>/home/zender/bin/LINUX</tt></li> <li>ESS <tt>esmf.ess.uci.edu</tt> (AIX 5.1.0.0): <tt>/home/zender/bin/AIX</tt></li> <li>ESS <tt>sand.ess.uci.edu</tt> (Linux 2.6.19): <tt>/home/zender/bin/LINUXAMD64</tt></li> <li>CGD network (Solaris 2.7): <tt>/home/zender/bin/SUNMP</tt></li> <li>SCD <tt>blueice.ucar.edu</tt> (AIX 5.3): <tt>/usr/local/bin</tt></li> <li>SCD <tt>bluevista.ucar.edu</tt> (AIX 5.3): <tt>/usr/local/bin</tt></li> <li>SCD <tt>tempest.ucar.edu</tt> (IRIX 6.5): <tt>/usr/local/bin</tt></li> </ul> <hr></p> <!-- http://nco.sf.net#BM --> <dt><a name="BM"></dt> <h2>Benchmarks</h2> <p> We are benchmarking NCO in SMP (e.g., AIX) and MPI (e.g., Linux) environments. The full NCO Benchmarks (in PDF form) are available <a href="http://dust.ess.uci.edu/nco/bm/index.html">here</a>. Please send us your suggestions for improved configurations. The NCO distribution includes all the custom benchmarking software, and is designed to allow you to contribute your own benchmarks. <a href="http://nco.sf.net#Contact">Contact us</a> if you're interested.</p> Current tests include data stored locally and across OPeNDAP servers. We will benchmark NCO with new features (e.g., netCDF4) and hardware (e.g., parallel filesystems) as time permits.</p> <p> There are two parallel versions of NCO—one uses <a href="http://www.openmp.org/drupal/">OpenMP</a> on SMP (including hyperthreaded) machines and the other uses the <a href="http://www-unix.mcs.anl.gov/mpi/">Message Passing Interface (MPI)</a> on distributed architectures. OpenMP requires a compiler that supports it. Commercial compilers which we have used include those from <a href="http://www.ibm.com/developerWorks/toolbox/download.html/">IBM</a> and the <a href="http://www.pgroup.com/">Portland Group</a>. <a href="http://www.intel.com/cd/software/products/asmo-na/eng/compilers/index.htm">Intel's Linux compiler</a> is free (beer). The MPI operators will run with most MPI implementations, including LAM, MPICH, and MPICH2. LAM installs easily with our own testing environment (latest Ubuntu). </p> <hr> <dt><a name="Rqr"></a><a name="rqr"></a></dt> <!-- http://nco.sf.net#rqr --> <dt><a name="bld"></a></dt> <!-- http://nco.sf.net#bld --> <h2>Build-from-Source Requirements</h2> <ul><b>Best Practices:</b> <a name="bld_rqr"></a> <!-- http://nco.sf.net#bld_rqr --> <li>Install <a href="http://www.antlr.org">ANTLR 2.7.x</a> (required for <tt>ncap2</tt>), <a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a> (absolutely required), <a href="http://opendap.org">OPeNDAP</a> (enables network transparency), and <a href="http://www.unidata.ucar.edu/packages/udunits">UDUnits</a> (allows dimensional unit transformations) before building NCO. If possible build these all with the same compiler (and switches). NCO is mostly written in C99, and although you <i>can</i> mix and match compilers, this is often difficult in practice and is not recommended. The exception is <tt>ncap2</tt> which is written in C++. ANTLR, OPeNDAP, and NCO must be built with the same C++ compiler to properly resolve the C++ name-mangling. NCO does not yet support newer ANTLR versions because the ANTLR 3.x C++ interface is incomplete. </li> </ul> <h2>Hints to Build-from-Source:</h2> <ul><b>Generic Build Hints:</b> <a name="bld_cnf"></a> <!-- http://nco.sf.net#bld_cnf --> <li><i>The two build NCO build mechanisms:</i> NCO has two build mechanisms: (1) GNU configure and (2) a custom Makefile (<tt>nco/bld/Makefile</tt>) designed for hand-editing. Try the traditional GNU <tt>configure</tt> method first, i.e., <tt>./configure;make;make check;make install</tt>. This method involves no hand-editing of makefiles, but trouble-free builds with it are less common than we would like. The file <tt>nco/configure.eg</tt> contains a table which describes the success you can expect building NCO with the <tt>configure</tt> mechanism. Please contribute patches required to get the <tt>configure</tt> mechanism working properly for NCO on your platform. The <a href="http://dust.ess.uci.edu/nco/rgr">regressions archive</a> contains gory details of successful (and failed) builds for many platforms. <br> All platforms continue to work fine with the <tt>nco/bld/Makefile</tt> method: <tt>cd ~/nco/bld;make all;make test</tt>. Disadvantages are that <tt>nco/bld/Makefile</tt> (1) Usually requires editing to fix your platforms paths; (2) Does not support shared libraries; and (3) Requires a degree from Hogwarts to understand. </li> <!-- http://nco.sf.net#bld_chn --> <a name="bld_chn"></a> <li><i>Build Chain Version Incompatibilities:</i> NCO requires GNU <tt>make</tt> version ≥ 3.78.1 to support target-specific variable values in <tt>bld/Makefile</tt>. NCO most often fails to build because <tt>ncap</tt> fails to build. Two of the most common reasons that <tt>ncap</tt> fails to build are (1) GNU <tt>bison</tt> is out-of-date (upgrade <tt>bison</tt>) and (2) <tt>--enable-optimize-*</tt> is specified to <tt>configure</tt> (do not specify it).<br> </li> <!-- http://nco.sf.net#bld_c99 --> <a name="bld_c99"></a> <li><i>C99-compliance & old compilers:</i> NCO versions ≥ 2.9.8 requires compilers which support the 1999 ISO C-Standard called C99. Currently, the only C99 feature NCO takes advantage of is restricted pointers. To build NCO versions ≥ 2.9.8 without a C99-compliant compiler, simply undefine the <tt>restrict</tt> type qualifier before compiling NCO. Do this, e.g., with <tt>CPPFLAGS='-Drestrict='</tt>. More details on C99 compatibility are in the header file <a href="http://nco.cvs.sf.net/nco/nco/src/nco/nco.h"><tt>nco/src/nco/nco.h</tt></a>.<br> </li> <!-- http://nco.sf.net#bld_netcdf --> <a name="bld_netcdf"></a> <li><i>Old netCDF libraries:</i> NCO versions ≥ 3.0.0 require netCDF versions ≥ 3.6 to take advantage of the new 64-bit offset file type. The library <i>and</i> include directories (e.g., <tt>/usr/lib</tt> and <tt>/usr/include</tt>, respectively) for netCDF version ≥ 3.6 must be on the path searched by the compiler, and must precede the locations of any older netCDF installations. To build NCO versions ≥ 3.0.0 with older netCDF versions < 3.6, simply define the pre-processor token <tt>NC_64BIT_OFFSET</tt> to <tt>0</tt> before compiling NCO. Do this, e.g., with <tt>CPPFLAGS='-DNC_64BIT_OFFSET=0'</tt> or <tt>USR_TKN='-DNC_64BIT_OFFSET=0'</tt> if you use <tt>configure</tt> or <tt>bld/Makefile</tt>, respectively. Users attempting to build NCO with OPeNDAP versions ≤ 3.6.x may need this workaround because OPeNDAP did not fully support netCDF version 3.6.x functionality until OPeNDAP versions ≥ 3.7.0. (The indicator for this problem is a build error like <tt>"nco_fl_utl.c", line 30.20: 1506-045 (S) Undeclared identifier NC_64BIT_OFFSET</tt>.) All modern NCO versions require netCDF versions ≥ 3.5.0 to support <tt>nc_type NC_NAT</tt> in NCO source code. NCO versions 3.9.0 and greater (May, 2007) require netCDF versions ≥ 3.6.0 to resolve the <tt>nc_inq_format()</tt> function. </li> <!-- http://nco.sf.net#bld_ncap --> <a name="bld_ncap"></a> <li><i><tt>ncap</tt> build hints:</i> <tt>ncap</tt> is a very sophisticated and somewhat fragile program. <tt>ncap</tt> requires <tt>flex</tt> to build correctly. AT&T <tt>lex</tt> will not work because it does not understand <tt>EOF</tt> rules that <tt>ncap_lex.l</tt> uses. <tt>bison</tt> version 1.29 (the default version on early Mac OS X) is buggy and does not parse <tt>ncap_yacc.y</tt> correctly; previous and later versions of <tt>bison</tt> work correctly. <tt>bison</tt> version 1.25 will not build <tt>ncap</tt> on Solaris. Please upgrade <tt>bison</tt> to the latest version (e.g., 1.875+) before reporting <tt>ncap</tt> build problems. If <tt>ncap</tt> keeps you from building NCO, and you do not need <tt>ncap</tt>, configure with <tt>--disable-ncap</tt> (and know that <tt>ncwa</tt> will fail to build as a result). </li> <!-- http://nco.sf.net#bld_DAP --> </a><a name="bld_dap"></a></a><a name="bld_DAP"></a><a name="bld_DODS"></a><a name="bld_OPeNDAP"> <li><i>Build Hints with OPeNDAP/DODS:</i><br> NCO versions 3.1.3— support automatic building and linking for <a href="http://opendap.org">OPeNDAP</a> versions 3.6.x and GCC versions 4.0+.<br> NCO versions 3.0.1—3.1.2 support automatic building and linking for <a href="http://opendap.org">OPeNDAP</a> versions 3.5.x and GCC versions 3.4+.<br> NCO versions 2.8.5—3.0.0 support automatic building and linking for <a href="http://opendap.org">OPeNDAP</a> versions 3.4.x and GCC versions 3.3+.<br> NCO versions ?.?.?—2.8.4 support automatic building and linking for <a href="http://opendap.org">DODS</a> versions 3.2.x and GCC versions 3.2+.<br> No other version combinations are supported. </li> <!-- http://nco.sf.net#bld_nco_c++ --> <a name="bld_nco_c++"></a> <li><i><tt>libnco_c++</tt> build hints:</i> <tt>libnco_c++</tt> requires a good C++ compiler to build correctly. Few people need <tt>libnco_c++</tt>, so the workaround is not to build it. This is done by configuring with <tt>--disable-nco_cplusplus</tt>. </li> </ul> <ul><b>Platform-Specific Build Hints:</b> <!-- http://nco.sf.net#bld_aix --> <a name="bld_aix"></a> <li><i>AIX build hints:</i> NCO versions 2.8.7+ do not build with autotools under AIX when <tt>--enable-optimize-*</tt> is selected. The workaround is to not use <tt>--enable-optimize-*</tt>. NCO versions 2.9.7+ do not build shared libraries with AIX xlc compilers because of silly AIX library dependencies activated by OpenMP directives. The workaround is to configure with <tt>--disable-shared</tt>. </li> <!-- http://nco.sf.net#bld_irix --> <a name="bld_irix"></a> <li><i>IRIX build hints:</i> The most common problem with building on SGI IRIX is and out-of-date toolchain. </li> <!-- http://nco.sf.net#bld_osx --> <a name="bld_osx"></a> <li><i>Mac OS X build hints:</i> Mac OS X users report success building NCO without patches so long as they (1) Install the latest version of <tt>bison</tt>, (2) Disable shared libraries and (3) Disable extended regular expressions. This is done with <tt>./configure --disable-regex --disable-shared</tt> Building NCO with shared libraries on Mac OS X on may be possible. To try, first re-build and re-install the netCDF library using <tt>CFLAGS=-fno-common</tt>. Apparently netCDF, unlike most codes, does not set this automatically. Then build NCO normally. Mac OS X ships with a broken version of the POSIX <tt>cut</tt> utility which causes many NCO regression tests to appear to fail. In fact the NCO commands succeed, but the <tt>cut</tt> output fails. The workaround is not to heed the regression tests, or to install a working <tt>cut</tt>, e.g., from the GNU <tt>coreutils</tt> package. </li> <!-- http://nco.sf.net#bld_slr --> <a name="bld_slr"></a> <li><i>Solaris build hints:</i> All recent NCO versions build with autotools on Solaris 2.7—2.9. The most common problem with building on Solaris with <tt>cc</tt>/<tt>CC</tt> is that <tt>bison</tt> may be out-of-date. Install an up-to-date <tt>bison</tt> (e.g., 1.875+). Solaris 2.10, i.e., Solaris10, had the audacity to change the prototype of the <tt>ctime_r()</tt> function and so does not build NCO 3.0.1. The fix is to upgrade to the current NCO version. </li> <!-- http://nco.sf.net#bld_sx --> <a name="bld_sx"></a> <li><i>SX build hints:</i> NEC SX does not understand the <tt>S_ISLNK</tt> macro, preferring instead the (non-POSIX) <tt>S_IFLNK</tt> macro. This macro appears in the file <a href="http://nco.cvs.sf.net/nco/nco/src/nco/nco_fl_utl.c"><tt>nco_fl_utl.c</tt></a>. The patch I wrote for this may be broken. Will someone with NEC SX please send us a working patch? </li> <!-- http://nco.sf.net#bld_wnd --> <a name="bld_wnd"></a> <li><i>Windows build hints:</i> Building on Microsoft Windows is possible with Cygwin. The resulting executables will behave normally on local files, but will not have most of NCO's network functionality (e.g., <tt>scp</tt>, <a href="http://opendap.org">OPeNDAP</a>). See the <a href="http://nco.sf.net/nco.html#wnd">User's Guide</a> for more details. </li> </ul> <hr></p> <dt><a name="bug"></a></dt> <!-- http://nco.sf.net#bug --> <dt><a name="prb"></a></dt> <!-- http://nco.sf.net#prb --> <h2>Known Problems</h2> Problems with current release: <ul><b>Platform-specific Run-time Problems:</b> <a name="bug_ncap_flt_fnc_aix"></a> <li><i>Float-valued intrinsic arithmetic functions in <tt>ncap</tt> on AIX:</i> <tt>ncap</tt> versions through 3.9.5 have a bug that causes all float-valued intrinsic math functions to fail under AIX. Float-valued math functions are the ISO C99 functions, e.g., <tt>cosf()</tt>, <tt>fabsf()</tt>, <tt>logf()</tt>. The user does not invoke these functions directly— the user always specifies the <i>generic function name</i>, e.g., <tt>cos()</tt>, <tt>abs()</tt>, <tt>log()</tt>. NCO automatically calls the native single precision (i.e., float-valued) math functions when the generic function argument is a native float (e.g., naked constants like <tt>1.0f</tt> or variables stored as <tt>NC_FLOAT</tt>). Double precision arguments cause NCO to invoke the standard (double-valued) form of the generic function, e.g., <tt>cos()</tt>, <tt>fabs()</tt>, <tt>log()</tt>. <!-- Whevnever this bug is fixed, here is some text Version info: <tt>ncap</tt> float functions work in AIX (<a href="#bug_ncap_flt_fnc_aix">long-time bug</a>, upgrade will be required for AIX ncap users) ANNOUNCE info: First, AIX NCO installations should upgrade to this version to get the AIX-specific ncap bug-fix. Bug-fixes: 1. fxm: AIX uses NCO-supplied float-valued functions (e.g., cosf()) --> </li> </ul> Problems with older releases: <ul><b>Generic Run-time Problems:</b> <!-- http://nco.sf.net#bug_idx_hyp --> <a name="bug_idx_hyp"></a> <li><i>Index-based hyperslab problems:</i> NCO versions 2.7.3—2.8.3 have a nasty bug that causes index-based hyperslabs, e.g., <tt>-d lat,1,</tt> to behave like value-based hyperslabs, e.g., <tt>-d lat,1.0</tt> under some conditions. Unfortunately, the incorrect answers generated may be hard to notice! This problem was most often enountered by users trying to assemble monthly averages using the stride feature of <tt>ncrcat</tt>. One common symptom is that the time-offset of the output file is incorrect. <i>Versions 2.7.3—2.8.3 are buggy and should be upgraded ASAP</i>. Re-do any data-processing that used index-based hyperslabbing with these versions of NCO. </li> </ul> <ul><b>Operator-specific Run-time Problems:</b> <!-- http://nco.sf.net#bug_ncap_cmt --> <a name="bug_ncap_cmt"></a> <li><i>Arithmetic problems with <tt>ncap</tt> division, modulo, and exponentiation:</i> <tt>ncap</tt> versions < 3.0.1 incorrectly exponentiate variables to variable powers (<tt>V^V</tt>). We recommend that all <tt>ncap</tt> users upgrade.<br> <tt>ncap</tt> versions up to 2.9.1 incorrectly handle division, modulo, and exponentiation operations of the form <tt>S/V</tt>, <tt>S%V</tt>, and <tt>S^V</tt> where first operand (<tt>S</tt>) is scalar (i.e., either typed directly in the <tt>ncap</tt> script or converted from an attribute) and the second operand (<tt>V</tt>) is a full variable (i.e., stored in a file or computed by <tt>ncap</tt>). Instead of the requested quantity, <tt>ncap</tt> returned <tt>V/S</tt>, <tt>V%S</tt>, and <tt>V^S</tt>. In other words <tt>ncap</tt> treated some non-commutative operations as commutative. This is now fixed. The <tt>V/V</tt>, <tt>V%V</tt>, <tt>V^V</tt>, <tt>V/S</tt>, <tt>V%S</tt>, <tt>V^S</tt>, <tt>S/S</tt>, <tt>S%S</tt>, and <tt>S^S</tt> operations were never affected. We recommend that all <tt>ncap</tt> users upgrade. </li> <!-- http://nco.sf.net#bug_ncbo_pck --> <a name="bug_ncbo_pck"></a> <li><i>Incorrect <tt>ncbo</tt> output for packed input:</i> <tt>ncbo</tt> versions ???—3.2.0 incorrectly write differences of packed input. This only affects packed variables. <!-- http://nco.sf.net#bug_ncflint_mss_val --> <a name="bug_ncflint_mss_val"></a> <li><i>Problems with <tt>ncflint</tt> and <tt>missing_value</tt>s:</i> The algorithm <tt>ncflint</tt> used to perform interpolation in versions up to 2.9.4 was not commutative. It returned the weighted valid datum when the other datum was <tt>missing_value</tt>, or it returned <tt>missing_value</tt>, depending on the order the input files were specified. As of version 2.9.5, <tt>ncflint</tt> always returns <tt>missing_value</tt> when either input datum is <tt>missing_value</tt>. Possible future implementations are discussed <a href="http://nco.sf.net/nco.html#ncflint">here</a>. <!-- http://nco.sf.net#bug_mss_val_zro_ncra --> <a name="bug_mss_val_zro_ncra"></a> <li><i>Problems with <tt>ncra</tt> and <tt>ncea</tt> when <tt>missing_value</tt> = 0.0:</i> The algorithm <tt>ncra</tt> and <tt>ncea</tt> used to perform arithmetic in versions up to 2.9.2 breaks if <tt>missing_value</tt> is 0.0. Why, you ask? Running average (or total, etc.) algorithms must initialize the answer to 0.0. This is done since the sum accumulates in place as <tt>ncra</tt> and <tt>ncea</tt> proceeds across records and files. (Normalizing this accumulation by the total number of records is the last step). The old algorithm compared both the current running average and the new record to the <tt>missing_value</tt>. If either comparison matched, then nothing accumulated for that record. This zero-initialization led to a state where it was impossible to ever recognize valid data. As a result nothing accumulated and the answer was always zero. The record and ensemble averages would also fail (in a non-obvious) way whenever an intermediate sum equalled <tt>missing_value</tt>. The chances of the latter event ever happening are exceedingly remote. The new algorithm compares only the new record to the <tt>missing_value</tt>. This fixes both problems and is faster, too. <!-- http://nco.sf.net#bug_pck_ncwa --> <a name="bug_pck_ncwa"></a> <li><i>Packing problems with <tt>ncwa</tt>:</i> NCO versions ???—2.9.0 have a bug that causes <tt>ncwa</tt> to fail (produce garbage answers) when processing packed <tt>NC_FLOAT</tt> data. Version 2.9.1 fixes this problem. This problem may have been noticed most by <a href="http://opendap.org">OPeNDAP</a> users since many netCDF climate datasets served by <a href="http://opendap.org">OPeNDAP</a> are packed <tt>NC_FLOAT</tt>s. Upgrade to 2.9.1 if you use <tt>ncwa</tt> on packed data. </li> <!-- http://nco.sf.net#bug_pck --> <a name="bug_pck"></a> <li><i>Packing problems with <tt>ncap</tt>:</i> NCO versions 2.8.4—2.8.6 have a bug that causes the <tt>ncap</tt> intrinsic packing function <tt>pack()</tt> to fail. Version 2.8.7 fixes this problem. </li> </ul> </ul> <p><hr></p> <!-- End Bugs --> <!-- Begin People --> <a name="ppl"></a> <!-- http://nco.sf.net#ppl --> <a name="people"></a> <!-- http://nco.sf.net#people --> <dt><a name="People"></a></dt> <!-- http://nco.sf.net#People --> <h2>People:</h2> <p>(Please <a href="#Contact">communicate</a> with us using the project forums rather than contacting us individually) <ul> <li> <a name="zender"></a><a name="Zender"></a> <!-- http://nco.sf.net#zender --> <a href="http://www.ess.uci.edu/~zender">Charlie Zender</a>, Associate Professor of <a href="http://www.ess.uci.edu">Earth System Science</a> and Director, <a href="http://www.ess.uci.edu/esmf">UCI Earth System Modeling Facility</a>. <i>Role</i>: Project PI. <i>Contributions</i>: NCO core library, porting, release manager <i>Current Research</i>: 1. Extend empirically verified analytic model for terascale data reduction of gridded datasets to account for cluster- and network- effects. 2. Enable and optimize NCO for intra-file-level parallelism using netCDF4/HDF5 parallel filesystem features. <i>Other Interests</i>: Atmospheric Physics, Climate Change. </li> <li> <a name="jenks"></a><a name="Jenks"></a> <!-- http://nco.sf.net#jenks --> <a href="http://spds.ece.uci.edu/~sjenks">Steve Jenks</a>, Assistant Professor of <a href="http://www.eecs.uci.edu">Electrical Engineering and Computer Science </a> and Director, <a href="http://spds.ece.uci.edu">Scalable Parallel and Distributed Systems Lab</a>. <i>Roles</i>: Collaborator. Thesis advisor. <i>Current Research</i>: 1. Consulting on SWAMP design and implementation. <i>Other Interests</i>: Cache memory usage </li> <li> <a name="papadopoulos"></a><a name="Papadopoulos"></a> <!-- http://nco.sf.net#papadopoulos --> <a href="http://users.sdsc.edu/~phil/homepage.html">Phil Papadopoulos</a>, Program Director, Grid and Cluster Computing, <a href="http://sdsc.ucsd.edu">SDSC</a>, and Adjunct Associate Professor of <a href="http://www.cse.ucsd.edu">Computer Science</a>, UCSD. <i>Role</i>: Co-PI. 1. Coordinates UCSD-end of NCO/SDO benchmark machines. <i>Other Interests</i>: NPACI Rocks </li> <li> <a name="wang"></a><a name="Wang"></a> <!-- http://nco.sf.net#wang --> <a href="http://dust.ess.uci.edu/~wangd">Daniel Wang</a>: Ph.D. Candidate, Electrical Engineering and Computer Science <i>Role</i>: Architect distributed data reduction and analysis solutions for geoscience workflows <i>Current Research</i>: 1. Use of compiler techniques and whole program analysis to discover and exploit parallelism in scripted workflows. 2. Resource management in combined computation-data service environments. 3. Design and implement <a href="http://swamp.googlecode.com">SWAMP</a> to provide automated server-side data analysis capabilities to the geoscience community. <i>Other Interests</i>: Large data, data management, programming languages and practice. </li> <li><dt> <a name="butowsky"></a><a name="Butowsky"></a> <!-- http://nco.sf.net#butowsky --> <a href="http://dust.ess.uci.edu/hbutowsk">Henry Butowsky</a>, Henry Butowsky, software engineer. <i>Roles</i>: Scientific programmer <i>Current Research</i>: 1. Efficient complex data analysis with storage-layer constraints. 2. Develop and thread the <tt>ncap2</tt> interpreter. <i>Other Interests</i>: Compilers and interpreters. </li> <li> <a name="capps"></a><a name="Capps"></a> <!-- http://nco.sf.net#capps --> <a href="http://dust.ess.uci.edu/scapps/web">Scott Capps</a>, Ph.D. Candidate, Earth System Science. <i>Current Research</i>: 1. Test NCO on real tera-scale geoscience analysis problems and identify necessary improvements in <tt>ncap2</tt> and in SWAMP. 2. Role of satellite-observed sub-gridscale wind variability on air-surface exchange. <i>Other Interests</i>: Weather, climate </li> <p>Thanks also to <a href="http://nco.sf.net/nco.html#ppl">past NCO/SDO contributors</a>. </ul> <hr></p> <!-- End People --> <!-- Begin Contacts --> <!-- http://nco.sf.net#Help --> <a name="hlp"></a> <!-- http://nco.sf.net#hlp --> <a name="help"></a> <!-- http://nco.sf.net#help --> <a name="Help"></a> <!-- http://nco.sf.net#Help --> <a name="contact"></a> <!-- http://nco.sf.net#contact --> <dt><a name="Contact"></a></dt> <!-- http://nco.sf.net#Contact --> <h2>Contacting the Project:</h2> <p>Contact us through the Project Forums (rather than personal e-mail) so other users can benefit from and contribute to our exchange. Let us know how NCO is working for you—we'd like to hear. Have you read the <a href="#RTFM">documentation</a> and browsed the forums to see if your question/comment has been reported before? Please read the Guide's suggestions for productive <a href="./nco.html#help">Help Requests and Bug Reports</a>. <ul> <li><i>Where should I ask my questions on how to use NCO?</i> On the <a href="http://sf.net/forum/forum.php?forum_id=9830">Help</a> site. </li> <li><i>Where should I post suggestions/comments on NCO features and usage?</i> On the <a href="http://sf.net/forum/forum.php?forum_id=9829">Discussion</a> site. </li> <li><i>Where are NCO development and bug-squashing discussed?</i> At the <a href="http://sf.net/forum/forum.php?forum_id=9831">Developer</a> site.</li> </ul> <hr></p> <!-- End Contacts --> <a href="http://sf.net"><img src="http://sf.net/sflogo.php?group_id=3331&type=1" width="88" height="31" border="0" alt="SourceForge.net Logo"></a> <table width="100%"><tr><td align="left" valign="top">/ <a href="http://lists.sf.net/mailman/listinfo/nco-announce">Announce</a> / <a href="http://sf.net/forum/forum.php?forum_id=9831">Developer</a> / <a href="http://sf.net/forum/forum.php?forum_id=9829">Discussion</a> / <a href="http://sf.net/forum/forum.php?forum_id=9830">Help</a> / <a href="http://nco.sf.net">Homepage</a> / <a href="#RTFM">Manual</a> / <a href="http://sf.net/projects/nco">Project</a> / <a href="http://nco.cvs.sf.net/nco/nco/src/nco/">Source</a> / </td> <td align="right" valign="top"> [<a href="http://www.antlr.org">ANTLR</a>] [<a href="http://www.unidata.ucar.edu/packages/netcdf">netCDF</a>] [<a href="http://www.unidata.ucar.edu/packages/netcdf/netcdf-4">netCDF4</a>] [<a href="http://opendap.org">OPeNDAP</a>] [<a href="http://swamp.googlecode.com">SWAMP</a>] [<a href="http://www.unidata.ucar.edu/packages/udunits">UDUnits</a>] </td></tr></table> </body> </html>
distrib
>
Mandriva
>
2010.0
>
i586
>
media
>
contrib-release
>
by-pkgid
>
c19e19dccb2f9a49f95629cbbd11307c
>
files
>
61
