Frequently Asked Questions

Contents

1. Accessing ICHEC

1. Where should a new user begin?
2. How do I join the ICHEC mailing lists?
3. What is the overall application process?
4. How do I connect to an ICHEC system?
5. Can I connect from home or when traveling?
6. What is SSH? How does it work?
7. How do I sftp files between my desktop and ICHEC if I have to use a local ssh proxy?
8. I have received a request for my login details, what should I do?
9. I can login with SSH but cannot connect with SFTP?

2. Compilers and software applications

1. What languages are available?
2. What software applications are available?
3. Is help with Makefiles available?
4. How do I run Gaussian calculations on Stokes?

3. Submitting batch jobs

1. What is a batch request?
2. How do I submit a job to Stokes or Stoney?
3. How can I execute a number of serial jobs (task farming)?
4. How can I find out how much resources (CPU core hours) are available to my project?
5. How can I view the stdout/stderr spool files for a running job?
6. How is the order that jobs run in decided?

4. ICHEC support

1. How do I contact the ICHEC helpdesk?
2. I have forgotten my password - can you help?
3. How do I acknowledge ICHEC in my work?

5. Common problems

1. "Disk quota exceeded"
2. How do I remove ^M characters at the end of input lines?
3. How do I run remote X applications?
4. Can I share files with members of my project?

1. Accessing ICHEC

New ICHEC users should refer to the Stokes and Stoney support page as a starting point. Tutorials on specific topics can be found here. New users should also refer to the Education & Training section for training courses and upcoming workshop schedules.

When an account is created, your email address is automatically subscribed to a mailing list:

• ichec-announce@lists.ichec.ie: low-traffic list for announcements regarding the Centre

If you wish to receive updates about the Centre, please e-mail us at support@ichec.ie, asking to be subscribed to the ichec-announce mailing list.

An overview of the application procedure from start to submitting jobs:

1. A PI submits an application via
2. http://www.ichec.ie/services/full_national_service.
3. If the project is accepted, an account is automatically created for the PI with a generated username and password (if the PI is not already an ICHEC user).
4. New users to ICHEC (apart from the PI) who wish to take part in the project should individually register with ICHEC (see Services) and apply to join the project.
5. The PI approves users who wish to join the project.
6. PI and users log in and run jobs.

Once you have been assigned a username and initial password you can connect to the systems using SSH:

• In general: ssh username@system_name.ichec.ie
• E.g. Stokes: ssh username@stokes.ichec.ie

You will need SSH access outbound (port 22/tcp) to login to these machines. We have noticed in some institutions you will have to login to a network server with this SSH access first (e.g. DIAS). If you get a "Connection timed out" error, please contact your local network administrator first to ensure you have the appropriate ports open. Failing this, please submit an issue to the ICHEC Helpdesk.

To connect to ICHEC systems you need to be logging in from IP space assigned to a partner institution. This means that under normal circumstances you cannnot connect directly from home or when travelling. However your host institution may well provide a system to which you can connect from home etc. and from which you can subsiquently connect to ICHEC systems. Whether or not such a system is mechanism is provided is a matter for local policy at your institution.

Secure Shell (SSH) is a program to log into another computer over a network, to execute commands in a remote machine, and to move files from one machine to another. It provides strong authentication and secure communications over insecure channels. SSH provides secure X connections and secure forwarding of arbitrary TCP connections.

SSH works by the exchange and verification of information, using public and private keys, to identify hosts and users. The ssh-keygen command creates a directory ~/.ssh and files that contain your authentication information. The public key is stored in ~/.ssh/identity.pub and the private key is stored in ~/.ssh/identity. Share only your public key. Never share your private key! To further protect your private key you should enter a passphrase to encrypt the key when it is stored in the filesystem. This will prevent people from using it even if they gain access to your files.

If your desktop is on a private network and/or you need to ssh to ICHEC machines via a local ssh proxy then the best way to transfer files directly between your desktop and ICHEC is to use sftp over an ssh tunnel. To create the tunnel use something like:

ssh -f -N -L 2222:stokes.ichec.ie:22 username@sshproxy.localdomain

Then all connections to port 2222 on your desktop will be forwarded via sshproxy to stokes.ichec.ie. So to ftp between ICHEC and your desktop you can simply use:

sftp -oPort=2222 ichec_username@localhost

If you receive a request for your login details either by email or telephone you should NOT disclose them regardless of whether the person contacting you purports to be from ICHEC or otherwise as it may be an attempt to steal your login credentials as can be the case with online banking for example. Please ignore the request and report it to us directly.

This problem can occur if you have commands in your .bashrc file which produce output, perhaps an error message which can cause the problem to appear to be intermittent. Such commands should be placed in your .bash_profile file instead where they will not interfere with the SFTP connection process.

2. Compilers and software applications

Fortran, C, and C++ are available on the ICHEC systems. The commands used to invoke the compilers and/or loaders vary from system to system. For more information, see the system specific pages; Stokes and Stoney. A number of scripting and interpreted languages are also available.

Please refer to the Software section for an up-to-date overview. If there is a particular package you would like to see installed, please contact the ICHEC Helpdesk with your request.

For some examples and guidelines regarding Makefiles see our tutorial on Building and Porting. If you have specific questions feel free to contact us via the Helpdesk.

We have included instructions on the Gaussian page of our software section.

3. Submitting batch jobs

On the Stokes cluster, batch processing is managed by the Portable Batch System (PBS) and Moab Scheduler. PBS batch requests (jobs) are shell scripts that provide timing, memory, and processor information. For example scripts, see the Stokes and Stoney support page. PBS uses qstat -a to check the status, and qdel to delete a batch request. For more information also see our tutorial on batch processing and queueing.

All jobs should be submitted to Stokes via the PBS. PBS uses qsub my_script_name to submit a job. A sample script can be found on the Stokes and Stoney support page.

A taskfarm utility is in place on Stokes you can find details here.

Use the mybalance command as follows:

This command will return the number of core hours available to all your projects (in the above example icphy001c). So for instance, if you wish to run a 48 CPU core job for 24 hours, you will need to ensure that you have a minimum of 24*48=1152 core hours on your project's account.

On Stokes and Stoney you can monitor the output to stdout/stderr while a job is running by examining the appropriate file in:

You can use the tail -f command to continuously monitor the final lines of the file however this may slow the program's I/O operations depending on the volume of output produced. When the job completes these spool files are automatically copied to the jobs working directory.

The order that jobs run in is decided by the Moab scheduler based on a priority it calculates for each job. The calculation of queue priority is relatively complex in order to deal with long queues, large numbers of jobs and various classes of project. Below is a list of the factors that affect queue priority:

• Jobs gain priority based on the time they wait as an idle job in the queue.
• Fair share targets are applied that balance resource usage over time across both projects and project types.
• An expansion factor is used to modify job priority based on requested walltime. Shorter jobs are given extra priority.
• Only idle jobs gain queue priority. Jobs blocked due to soft or hard limits (see explaination below) can't run and don't gain queue priority over time.

In addition to priority, a backfill mechanism can allow jobs to skip the queue fill space that would otherwise have been unused. This allows smaller jobs with shorter runtimes to maximise our utilisation while larger jobs wait for processors to free up.

Stokes has hard and soft limits on the number of jobs and processors a single user can use. The soft limits apply when there is a queue and the considerably higher hard limits apply when there are no jobs waiting to run. When a user exceeds whatever limit is in force all their queued jobs are moved from the queue to a blocked state and are only unblocked when a job completes or the limit is no longer active. Blocked jobs are only checked periodically and not on the regular job scheduling interval.

In addition there is a limit on the number of jobs a user may have queued even when they have not exceeded the hard or soft limits. Additional jobs will be moved to a blocked state until a queued job runs or is cancelled. For further information see the scheduling policies page.

4. ICHEC support

The ICHEC Helpdesk and the email address are the two main entry points to ICHEC's support teams for registered users. Here you can get help in using the service, find out more about ICHEC or send us your comments.

Please send an e-mail to support@ichec.ie asking for a new password.

We would appreciate a formal acknowledgement of ICHEC by inclusion in any resulting publications of the following sentence or some variation thereof: "The authors wish to acknowledge the SFI/HEA Irish Centre for High-End Computing (ICHEC) for the provision of computational facilities and support."

It is also important that you notify us of the publication at info@ichec.ie

5. Common problems

When projects disk quota is exceeded the PI will recieved an email notification. You can check the disk usage using the

command. Details of how to do so on Stokes and Stoney can be found here.

UNIX treats the end of lines differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character gets added to the end of each line as ^M in vi. To remove the ^M characters use the following command:

If you are connecting from a MS Windows machine you will need to have Xming, Hummingbird Exceed or similar installed and running on your workstation. You also need to ensure that X11 forwarding is enabled.

On a typical Unix system X11 support will already be in place and you need connect using: ssh -X username@systemname.ichec.ie. Now you can execute any of your X applications and it should forward the X application via SSH to your local screen. For example:

You should now see the Xclock is running on your local screen.

In addition to individual user's home directories e.g. /ichec/home/users/username each project has an associated project directory /ichec/work/project_code which is accessible by all members of that group. By default a user's home directory and its contents cannot be read by other users though you can change the directory permissions to allow this if you wish.

This command will make the contents of the named directory readable by the other members of you project.