ARSC HPC Users' Newsletter 371, October 5, 2007

A Fortran Programmers' Debugging Module


[ by Lee Higbie ]

I know none of us writes code that needs debugging, but when we do I have a debugging module that may help. Localizing and understanding the cause of errors in legacy codes is probably most easily accomplished with standard debuggers; this module is intended for new code.

For my own programs I prefer to build debugging capability into the code rather than use a "debugger". During development, annotated information can be printed with no program disruption or modification. After delivery, if a problem arises, I can tell a user to change a file name, say, and send me the generated debug output so I can see what happens in their environment. The DebugClass module grew out of one I wrote for event driven programs, where it is often difficult to duplicate problems.

The basic idea of the DebugClass module is that a program's author inserts debugging statements in code at development time at points where there may be problems. These can be enabled or disabled as needed during execution to print out information the original author thinks most likely to shed light on problems. For example, I often have trouble with I/O and interpolation schemes so I insert debug code around them.

Version 1 of the Fortran Debug Module, a test/demo program and sample namelist files can be copied from http://people.arsc.edu/~higbie/ . The module, DebugClass.F95, includes detailed usage instructions. It defines six logical and one integer variables, intended for coarse control, and can read a file for fine-grain control options.

Module DebugClass usage syntax:

  1. At the beginning of your program, insert use DebugClass
  2. Create a DebugClass object by call debugInit(see documentation in module for args)
  3. Wherever debugging output may be required, insert one of: call debWr(__LINE__, __FILE__, 'message', <data>, flag = 'flgs') or if(debugOn) call debWr(__LINE__, __FILE__, 'message', & <data>, flag = 'flgs')

    The second of these two choices has the advantage that it avoids the subroutine call when no debugging is used. In fact, if debugOn is changed to a false PARAMETER (insert parameter in declaration near line 75), any good compiler will produce code that does not even execute the if statement. (If your compiler's preprocessor doesn't expand __LINE__ and __FILE__ use whatever it wants. We have compilers that expand __LINE__ to a string or an integer. The debWr subroutines accept either.)

  4. Create namelist.debFlags to set any fine-control flags and override default values in DebugClass. For example, I might use the following line for namelist.debFlags &debugOptions flagString = 'interp interpolate input output' /

    When you run your program, every call debWr( ... , flag = 'this') will check the string "this" against the flags and print the message when "this" matches one in the namelist input.

    Using the debugOptions above, call debWr( ... , flag = 'iterp') call debWr( ... , flag = 'input') will produce output but call debWr( ... , flag = 'iterpo') call debWr( ... , flag = 'io') will not. The output will be in the form: fileName @ lineNo: message, data or, if you are running in an MPI environment, fileName [mpiTask #] @ lineNo: message, data
  5. At the end of your program, just before exiting, call finalizeDebug ()

    This will print out some statistics about the debug options including a list of the fine-control flags encountered. This can help find misspelled or forgotten fine-control flags.

DebugClass version 1 includes output routines for integer, floating point, alternating integer-floating, and alternating integer-logical (the <data> in the calls above). It is trivial to add additional possibilities to subroutine debWr; each of the various overloadings are about 50 lines long.

If you find DebugClass.F95 useful and add any additional routines, please forward the enhancements to me. If you find bugs, please let me know so I can fix them. For example, routines for complex data and arrays of data might be useful, as would more thorough testing.

Copy or Move, Which is the Best File Transfer Method?


[ by Oralee Nudson ]

With the upcoming retirement of iceflyer, you may be participating in the tedious task of tarring up your files and moving them to one of the ARSC storage servers. During this process, please remember that it is best to first copy ("cp") the files over, confirm the transfer is complete, then delete the original file. This three step process is much safer than simply moving ("mv") the files. This is because a failed "mv" command may result in an unexpected loss of data.

Should a "mv" command fail before the entire tar file is transferred, the original file may be lost or corrupted. This is because the "mv" command only writes to cache when space is available, then subsequently flushes the information to disk. This can be observed by performing an "ls -l" listing on the destination file during a large transfer. A file size of zero bytes will be displayed and will not be updated until the storage server updates the listing during file closure. Should a problem occur during the cache writing or flushing, both the original and the new destination files may be lost. Therefore, to prevent unrecoverable mishaps during this busy time of year, remember to use cp instead of mv!

SC07 Registration is Open

SC07 will be held November 10-16 in Reno, Nevada. Early bird registration is available until 10/15/2007. For more information, check out the SC07 website:

http://sc07.supercomputing.org/

You can catch up with ARSC staff at the HPCMP booth (#528). The University of Alaska Fairbanks and International Polar Year (IPY) will also have a booth this year (#2247). You can learn more about IPY, here:

http://www.ipy.org/

We hope to see you at SC again this year!

ARSC Chief Scientist participates in CERT podcast

Greg Newby, ARSC, participated in the Carnegie Mellon University Software Engineering Institute's CERT Program podcast project on computer security issues. The podcast, part of a series on building strong information and cyber security systems, is available at http://www.cert.org/podcast/show/newby.html

Quick-Tip Q & A


A:[[ I'm grepping for "Bachelor Bitter" (both words).  However, grep
  [[ doesn't find it when there's a newline separating the two words.
  [[ Can grep ignore newlines, somehow? Do you have any other ideas?

  #
  # We recieved a few solutions using GNU grep, but none using standard
  # Unix grep.  Greg Newby's perl solution should help those lacking GNU
  # grep.  Thanks to all!
  #


  #
  # Andrew Roberts
  #

  This:
    grep -P "Bachelor\nBitter" <file>
  will find the words when they are separated by a line.

  While this:
    grep -P "Bachelor\WBitter" <file>
  will also catch Bachelor Bitter on the same line, or separated by a
  non-word character.

  However, the -P option is not supported on some Linux distributions (but
  it is on Midnight).


  #
  # Nathan Prewitt
  #

  Try this:
    grep -A1 Bachelor filename 
 grep -B1 Bitter


  #
  # Ryan Czerwiec
  #

  There are essentially two parts, to find lines containing "Bachelor
  Bitter" and lines ending in Bachelor followed by lines beginning with
  Bitter.  The first part is easy (sub your own filename):

    grep Bachelor filename 
 grep Bitter

  If they don't have to be consecutive words,

    grep "Bachelor Bitter" filename

  If they do, and

    grep -w "Bachelor Bitter" filename

  If only whitespace is allowed on either end of the expression, (ex.
  Bachelor Bitters would be excluded).

  The second part is not easy.  The problem with grep, awk, sed, etc. is
  that they treat input line-by-line and so do not skip endlines easily.
  Use the above examples to modify this expression as needed, but having
  assumed the last case of consecutive words surrounded by whitespace, here
  is my brute force solution (there must be a more elegant way):

  ( grep -wn "Bachelor[[:space:]]\+Bitter" filename ;
    grep -A 1 -wn "Bachelor[[:space:]]*$" filename 

    grep -B 1 -w "^[0-9]\+-[[:space:]]*Bitter") 

      sed s/[:-]/" "/ 
 sort -n 
 uniq 
 cut -f2- -d' '

  All that goes on one line.  Unfortunately, the very handy -A and -B grep
  options only exist on GNU systems and maybe a few others.


  #
  # Greg Newby
  #

  I don't think grep can ignore newlines, and didn't see a way to do this
  with sed or similar programs.  You could transform the file to map all
  newlines to spaces, then feed it to grep.

  As an alternative, here's a simple Perl program that uses the $\ variable
  to read an entire file at once.  Then, you can search for embedded
  newlines (\n):

    #!/usr/bin/perl -w
    $/=undef; # slurp mode.  'man perlfunc' for more info
    while (<>) {
      print "Match: $_\n" if /Bachelor\nBitter/;
    }

  Adding features to such a program (like only printing the matching lines,
  based on \n delimiters) would not be hard, depending on what you want it
  for.  Note that the newline character is not a space...if you want spaces
  handled the same as newlines, look at 'man perlre' for some options.



Q: I have a python script that I run on a few different machines.  
   On one machine python is in /usr/local/bin, on a different machine
   it's located in /usr/bin.  So I end up with two versions of the 
   script.  One that starts with: 

   #!/usr/local/bin/python

   and another that starts with 

   #!/usr/bin/python

   Every time I switch between machines I mess up at least one run
   because I forget to the change the "#!" path.  Is there a way I can
   avoid having to make this change?

[[ Answers, Questions, and Tips Graciously Accepted ]]


Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top