# Cluster integration: Problem connecting to Sun Grid Engine

I'm trying to launch some kernels in a cluster engine running Sun Grid Engine using Cluster integration. But I'm getting the error "Cannot find components required for SGE".

Needs["ClusterIntegration"]
kernels = LaunchKernels[SGE["server", 16]]


SGE::load: Cannot find components required for SGE. >>

I doubt this is an error in the cluster side, as I get the same error regardless if I point to the correct server host name or not, so I think it fails locally (Mathematica 9.0.1.0)

Also the documented option "Timeout" doesn't seems to exist, and the option "NetworkInterface" does not explain how to specify the interface..

Options[SGE]


{"EnginePath" -> "/usr/local/sge/sge_root", "KernelOptions" -> "-subkernel -mathlink -LinkMode Connect -LinkProtocol TCPIP \ -LinkName linkname", "KernelProgram" -> "/usr/local/Wolfram/Mathematica/7.0/Executables/math", "NativeSpecification" -> "", "NetworkInterface" -> "", "ToQueue" -> False}

Any help would be appreciated.

-
I have never had too much luck with the cluster integration package, nor do I think it's a reasonable approach for running on hpc clusters. The cluster integration package works like this: it assumes that you run the main kernel on a machine that is not part of the cluster, and then launches each subkernel as separate jobs. Just launching the kernels might take a lot of time if there's a long wait queue, and performance will be impaired if the main kernel is not run on the cluster. I recommend that you simply do not use this package. – Szabolcs Jun 4 '14 at 15:09
This is a small package I wrote to run on our cluster (also SGE, but probably still sifgnificantly different from your cluster). Very likely it won't run on your cluster as-is, but it shows you how to handle the problem. You can modify it to work on your own cluster. One key part is figuring out how to launch a Mathematica kernel on node Y when your main kernel is running on node X. I need to use rsh on our cluster for that, moreover I needed to use a specific version from a specific location. This will be different for your cluster. – Szabolcs Jun 4 '14 at 15:12
It's also possible that you will need to use ssh instead of rsh. On all other clusters I used I had to work with ssh`. – Szabolcs Jun 4 '14 at 15:12
I meant that you can use Mathematica on a cluster, but do not use the CluterIntegration package, which tries to do this in the wrong way. Take a look at how the package I linked to works, and if you have questions about what specific bits do, just ping me in the chatroom. – Szabolcs Jun 4 '14 at 15:15
Sorry, I think I was not 100% clear. If you need to do interactive work, or make use of the GUI, the solution should be the ClusterIntegration package. You're on the right track. I don't know how to use this because this package is not really suitable for non-interactive work, and launching parallel kernels takes as long as the wait time on your cluster (can be days here if requesting several nodes). If you're looking to run non-interactive jobs, then then don't use ClusterIntegration. See my comments above instead. – Szabolcs Jun 4 '14 at 15:54