ARSC T3D Users' Newsletter 91, June 14, 1996

pxfgetarg() - Handling Command Line Arguments in T3D Programs

[Don Morton, visiting ARSC from Cameron University, contributes this article. See his contact information, provided below.]

Many implementations of Fortran, particularly those in a Unix environment, allow for command-line argument passing when loading the executable. These functions, IARGC() and GETARG() are documented on Denali's man pages. In the following example, we demonstrate the use of these functions in a program which simply echos the arguments provided on the command line:

argtest1.f:


  ccc   argtest1.f
  
        program argtest1
        implicit none
  
        integer numargs,            ! Number of arguments on command line
       &        i                   ! Loop counter
  
        character*80 argument       ! Character string for storing an argument
  
        external iargc
        integer iargc
  
  ccc   Find out how many arguments there are
        numargs = iargc()
  
  ccc   Print each of the arguments - Note that the 0th argument, following
  ccc   Unix tradition, is the name of the binary.
        do i=0,numargs
           call getarg(i, argument)
           print *, 'Argument ', i, ':  ', argument
        enddo
  
        end
The above works fine on most Unix platforms, including the Cray Y-MP, Linux PC's, SGI's, etc. However, if you try this on a T3D program, you'll run into a link error stating that getarg() is an unresolved function call. Denali's man page for getarg() implies that the function is implemented on all Cray Research systems, but this is in error. Tucked in the pages of a Cray Research Service Bulletin is the real scoop.

Cray states that the increased address space on the T90 and MPP systems requires a two-word Fortran Character Descriptor in some of the Fortran system interface routines. Thus, they can no longer reliably detect when Hollerith arguments are used in place of Fortran character arguments and vice versa, particularly in routines which have optional arguments (such as getarg(), which can be called with an optional third argument).

Cray has provided alternative interface routines that conform to the Posix Fortran 77 Language Interfaces standard. The alternative for getarg() is pxfgetarg(), and ipxfargc() is the Posix-compliant alternative to iargc(). So, the above program, written for portability would look like:

argtest2.F:


  ccc   argtest2.F
  
        program argtest2
        implicit none
  
        integer numargs,            ! Number of arguments on command line
       &        i                   ! Loop counter
  
        character*80 argument       ! Character string for storing an argument
  
  #ifdef _CRAY-MPP
        external ipxfargc
        integer  ipxfargc
  ccc   extra variables needed for calling pxfgetarg()
        integer arglength, ierror
  #else
        external iargc
        integer  iargc
  #endif
  
  ccc   Find out how many arguments there are
  #ifdef _CRAY-MPP
        numargs = ipxfargc()
  #else
        numargs = iargc()
  #endif
  
  ccc   Print each of the arguments
        do i=0,numargs
  #ifdef _CRAY-MPP
           call pxfgetarg(i, argument, arglength, ierror)
           if (ierror .ne. 0) then
              write(0,*) 'Problem with input argument ', i
              stop
           endif
  #else
           call getarg(i, argument)
  #endif
           print *, 'Argument ', i, ':  ', argument
        enddo
  
        end
Several points concerning the above program:
  1. Note the .F extension which, on many systems (including denali) will cause the preprocessor to be invoked before compilation. On systems where this doesn't happen automatically, you'll need to run it through the C preprocessor first, with appropriate flags.
  2. The _CRAY-MPP macro is defined automatically when using the CRAY MPP compilers.
  3. pxfgetarg() is more functional than getarg(), allowing for more robust execution. Full details are available in the man pages.
From:

  =======================================================================
  
    Don Morton                              Email:  morton@arsc.edu  

  
    Visiting Scientist  (summer)            Voice: (907) 474-5507    

  
    Arctic Region Supercomputing Center     Fax  : (907) 450-8601    

  
    University of Alaska                                             

  
    Fairbanks, AK 99775                                              

  
    http://grizzly.cameron.edu/morton.html                           

  =======================================================================

List of Posix-Compliant Routines

A list of system-interface routines for which Cray has created alternative, Posix-compliant versions follows. Man pages are available, and documentation may be found in the "Application Programmer's Library Reference Manual, publication SR-2165:

  Old Routine    New Routine
  -----------    -----------
  access()       pxfaccess()
  chdir()        pxfchdir()
  chmod()        pxfchmod() 
  chown()        pxfchown() 
  chroot()       pxfchroot() 
  creat()        pxfcreat()
  getarg()       pxfgetarg()
  getenv()       pxfgetenv()
  iargc()        ipxfargc()
  link()         pxflink()
  open()         pxfopen()
  rename()       pxfrename()
  stat()         pxfstat()
  uname()        pxfuname()
  unlink()       pxfunlink()
  utime()        pxfutime()

Porting Heterogeneous Distributed Codes to the T3D - Part II

Last week (Newsletter #90), Don Morton discussed the master/slave paradigm for distributed programming, and contributed programs for a pair of cooperating processes. With these, he demonstrated the recommended method of porting PVM master/slave codes to the Y-MP/T3D.

The method he used follows CRI's advice (see next article) of opening only one circuit between the master (on the Y-MP) and the entire group of slaves (on the T3D). The master communicates with the slave on PE0, and no other. The slave on PE0, in turn, is designated as special, and relays data and instructions to all of the other slaves. In a sense, this amounts to having two masters, one on the Y-MP and a second on PE0. Another view: the process on the Y-MP acts as an interface, or front-end, to a master/slave group on the T3D.

Another simpler approach is to treat the Y-MP process as the sole master, and open a pvm circuit from it to each and every slave. This follows the traditional master/slave paradigm. It may require less recoding than the "dual" master approach, and could be an easy first step in porting distributed codes to the Y-MP/T3D. Once a program following this method is working, the programmer could move some, or even all, of the master's duties to the slave on PE0.

The following codes show a Y-MP master coordinating a group of equivalent T3D slaves (compare with last week's examples). To run these, you must set the PVM environment variable, PVM_PE_LIST, to "all" before launching the PVM daemon. (See next article.)

master:


        include 'fpvm3.h'
  
        parameter (NUMPROCS=5, ISIZE=1000, NUMSLAVES=NUMPROCS-1)
        dimension vec(ISIZE)
        dimension list(0:NUMSLAVES)   ! For storing TID's
  
  ccc   Enroll in PVM
        call pvmfmytid(mytid)
  
        print *, 'Hello World, I am the master: task ', mytid
  
  ccc   Spawn slaves
        call pvmfspawn("slave", 0, '*', NUMSLAVES, list, info)
        print *, 'Task ', mytid, ' spawned ', info, ' slaves...'
        print *, 'TID List of Slaves:'
  
        do i=0,info-1
           print *, list(i)
        enddo
  
  ccc   Initialize a vector for transmission to slaves      
        do i=1,ISIZE
           vec(ISIZE) = real(i)
        enddo            
  
  ccc   Send it to all slave processes
        msgtag = 1
        call pvmfinitsend(PVMDATADEFAULT, info)
        call pvmfpack(REAL8, vec, ISIZE, 1, info)
  
        do i=0,NUMSLAVES-1
          call pvmfsend(list(i), msgtag, info)
          print *, 'Task ', mytid, ' sent data to ', list(i)
        enddo
  
  ccc   Wait for ACK from all slaves
        msgtag = 2
  
        do i=0,NUMSLAVES-1
          call pvmftrecv(-1, msgtag, 30, 0, info)
          if (info .NE. 0) then 
            call pvmfbufinfo (info, nbytes, imsgtag, isender_tid, ierr)
            print *, 'Master received ACK from slave: ', isender_tid
          else
            print *, 'Master timed out waiting for ACK from slave.'
            goto 999
          endif 
        enddo
        
  ccc   exit PVM
  999   continue
        print *, 'Task ', mytid, ' exiting...'
        call pvmfexit(info)
  
        end
slave:

        include 'fpvm3.h'
  
        parameter (ISIZE=1000)
        dimension vec(ISIZE)
  
  ccc   Enroll in PVM
        call pvmfmytid(mytid)
        call pvmfgetpe(mytid, mype)
        call pvmfparent(myparent)
  
        print *, 'Slave, task ', mytid, ' PE ', mype, 
       + ' my parent ', myparent, ' group instance ', inum
  
  ccc   Waiting for "data" from master
        msgtag = 1
        call pvmftrecv(myparent, msgtag, 30, 0, info)
        if (info .NE. 0) then
          call pvmfunpack(REAL8, vec, ISIZE, 1, info)
          print *, 'Slave ', mytid, ' received data ...'
        else
          print *, 'Slave ', mytid, ' timed out waiting for data.'
          goto 999
        endif 
  
  ccc   Send ACK to master
        msgtag = 2
        call pvmfinitsend(PVMDATADEFAULT, info)
        call pvmfpack(INTEGER8, 1, 1, 1, info)
        call pvmfsend(myparent, msgtag, info)
  
  ccc   Exits PVM
  999   continue
        print *, 'Task ', mytid, ' exiting...'
        call pvmfexit(info)
  
        end
Makefile:

  all: master slave

  slave: slave.f  
          TARGET=cray-t3d /mpp/bin/cf77 -I/usr/include/mpp -o slave slave.f  
          cp slave ${HOME}/pvm3/bin/CRAY/

  master: master.f  
          TARGET=target cf77 -I/usr/include/pvm3 -o master master.f -lpvm3

Environment Variable 'PVM_PE_LIST'

PVM_PE_LIST tells the pvm daemon with which T3D PEs it must prepare to communicate. For example:

  'unset PVM_PE_LIST'   ## circuit to task spawned on PE0 
                        ## PVM daemon ALWAYS opens circuit to PE0.
  'PVM_PE_LIST=0'       ## circuit to PE0
  'PVM_PE_LIST=2'       ## circuit to PE0 and PE2
  'PVM_PE_LIST=1,2,4,8' ## circuit to PEs 0,1,2,4, and 8
  'PVM_PE_LIST=all'     ## circuit to every spawned task
PVM_PE_LIST is ignored by programs run directly on the T3D (using mppexec). Only if your program uses the pvm daemon and calls "pvm_spawn" or "pvmfspawn" should you worry about it.

The following runs of the master/slave program given in today's Newsletter show the effect of different settings for PVM_PE_LIST. If a PE is not on the list, pvmfspawn doesn't return a valid task id number (TID) for that PE, and thus, the master can't communicate with it.


  ## Note: run under Korn shell.
    
  denali$ PVM_PE_LIST=all
  denali$ pvm
  pvm> quit
    
  denali$ master
   Hello World, I am the master: task 262146
   Task 262146 spawned 4 slaves...
   TID List of Slaves:
   393216
   393217
   393218
   393219
   Task 262146 sent data to 393216
   Task 262146 sent data to 393217
   Task 262146 sent data to 393218
   Task 262146 sent data to 393219
   Master received ACK from slave: 393219
   Master received ACK from slave: 393218
   Master received ACK from slave: 393216
   Master received ACK from slave: 393217
   Task 262146 exiting...
    
    
  denali$ unset PVM_PE_LIST 
  denali$ pvm
  pvm> quit
    
  denali$ master            
   Hello World, I am the master: task 262146
   Task 262146 spawned 4 slaves...
   TID List of Slaves:
   393216
   1
   1
   1
   Task 262146 sent data to 393216
   Task 262146 sent data to 1
   Task 262146 sent data to 1
   Task 262146 sent data to 1
   Master received ACK from slave: 393216
   Master timed out waiting for ACK from slave.
   Task 262146 exiting...
    
    
    
  denali$ export PVM_PE_LIST=0,2
  denali$ pvm
  pvm> quit
    
  pvmd still running.
  denali$ master
   Hello World, I am the master: task 262146
   Task 262146 spawned 4 slaves...
   TID List of Slaves:
   393216
   1
   393218
   1
   Task 262146 sent data to 393216
   Task 262146 sent data to 1
   Task 262146 sent data to 393218
   Task 262146 sent data to 1
   Master received ACK from slave: 393218
   Master received ACK from slave: 393216
   Master timed out waiting for ACK from slave.
   Task 262146 exiting...
We've noted several times that CRI discourages programmers from opening more than one circuit from daemon to T3D. Okay... here is the source, taken from 'docview.'




  pvm.hence.50

  PE communication

  2.6.1.2

  

  PEs on the CRAY T3D system communicate with the daemon and other PVM 

  tasks by using sockets.  In theory, any PE can do so.  But UNICOS

  limits the number of open files per application and the number of open

  sockets in the system.  So, if a CRAY T3D application running on a

  large number of PEs were to set up communications for each PE, it may

  hit either or both of these limits.

  

  Socket communications are very slow, especially compared to the cost 

  of communications between PEs.  Because much of socket communication 

  is single-threaded in the agent process, the PVM daemon, or UNICOS 

  itself, the cost goes up as more PEs try to communicate at the same 

  time.

  

  For these reasons, by default only PE0 establishes communications with 

  the daemon, and Cray Research suggests you consider using PVM in this 

  manner.  However, you can specify additional PEs by setting the

  PVM_PE_LIST environment variable, as follows:

  

       setenv PVM_PE_LIST 0,4,8,12

       setenv PVM_PE_LIST all

  

  This environment variable must be set for both the PVM daemon pvmd3 

  and the application to read, and both must read the same value.

  

  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  

                                  Note

  

  At present, PE0 always establishes communications with the daemon, 

  even if PE0 is not specified in PVM_PE_LIST.  It is suggested that 

  PVM_PE_LIST specify PE0 if it is being used to ensure future

  compatibility. It is possible that future releases may introduce other

  mechanisms for controlling access to the daemon.

  

  

  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

List of Differences Between T3D and Y-MP

The current list of differences between the T3D and the Y-MP is:
  1. Data type sizes are not the same ( Newsletter #5 )
  2. Uninitialized variables are different ( Newsletter #6 )
  3. The effect of the -a static compiler switch ( Newsletter #7 )
  4. There is no GETENV on the T3D ( Newsletter #8 )
  5. Missing routine SMACH on T3D ( Newsletter #9 )
  6. Different Arithmetics ( Newsletter #9 )
  7. Different clock granularities for gettimeofday ( Newsletter #11 )
  8. Restrictions on record length for direct I/O files ( Newsletter #19 )
  9. Implied DO loop is not "vectorized" on the T3D ( Newsletter #20 )
  10. Missing Linpack and Eispack routines in libsci ( Newsletter #25 )
  11. F90 manual for Y-MP, no manual for T3D ( Newsletter #31 )
  12. RANF() and its manpage differ between machines ( Newsletter #37 )
  13. CRAY2IEG is available only on the Y-MP ( Newsletter #40 )
  14. Missing sort routines on the T3D ( Newsletter #41 )
  15. Missing compiler allocation flags ( Newsletter #52 )
  16. Missing compiler listing flags ( Newsletter #53 )
  17. Missing MXMA routine on the T3D ( Newsletter #75 )
  18. Missing getarg() routine on the T3D ( Newsletter #91 )

A Call for Material

If you have discovered a good technique or information concerning the T3D and you think it might benefit others, please send it to the e-mail address above and it will be passed on through this newsletter.
Current Editors:
Ed Kornkven ARSC HPC Specialist ph: 907-450-8669
Kate Hedstrom ARSC Oceanographic Specialist ph: 907-450-8678
Arctic Region Supercomputing Center
University of Alaska Fairbanks
PO Box 756020
Fairbanks AK 99775-6020
E-mail Subscriptions: Archives:
    Back issues of the ASCII e-mail edition of the ARSC T3D/T3E/HPC Users' Newsletter are available by request. Please contact the editors.
Back to Top