Boolean value indicating that the application can (True) or can't (False) use the PlayChapter method.
Create the file objects to be used in the compression process. This is the meat of the class. Here, the decisions are made regarding how the compression will be done as this needs to be determined prior to creating the file objects.
The philosophy of the gzip.GzipFile and bz2.BZ2File classes is that they open a GZip or BZip2 file for reading or writing as if it were a normal file. Any (de)compression is done transparently. This allows for a simpler programming interface as the classes need no extra methods called except read() and write().
This is a composite task. It iterates over its “task_list” and feeds the output of each step as input to the next task.
This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.
In a composite entry, i.e. Reduce>Coadd, for each step the same options will be used. However they can be filtered to exclude all options that are not understood by (any of) the next processing steps. I.e. the overscan correction option that can be used for Reduce can better be removed for the subsequent steps.
Return a list of option dictionaries, suitable for each step of the task_sequence. Options not recognised by a step are discarded for that step
This method is used to retrieve the result from executing the task. This result should be a list of instances of one of the many sub-types of DBObject. The method is abstract
Returns a list of objects which are the results of the processing done by this task. Note that objects must have been committed for this method to work.
A sequence identifier is the first argument given to the dpu.run() method. It defines which task(s) is (are) executed.
Returns all recognized options for the first task in the sequence of tasks defined by the sequence identifier.
Print all recognized options for the first task in the sequence of tasks defined by the sequence identifier.
This class is used as a frontend to “run” processes (tasks) on the local machine in the same way jobs are run using the dpu.
This class is used as a frontend to “run” processes (tasks) in a distributed fashion, for example on the HPC cluster in Groningen. This class uses the HTTP frame work (“Distributed Processing Unit”) coded for this purpose (see common/net/dpu*).
The Processor makes a connection to the specified DPU, schedules tasks for processing and then starts these processes. Output messages from the DPU may be printed on standard out, and written to a log file in the local directory.
Processes can then be started by calling the “run” method and supplying a task identifier argument and options for that task:
Get logs for finished jobs or the job identifier(s) specified. Logs are printed on stdout and written to file.
Q = QUEUED : # of jobs in the queue R = RUNNING : # of jobs currently being processed F = FINISHED : # of jobs finished processing N = NORMAL : # of jobs finished normally A = ABORTED : # of jobs aborted by the cluster E = ERROR : # of jobs ending with an error S = SKIPPED : # of jobs skipped U = USERABORT : # of jobs aborted by the user
Wait for either the specified jobs, or all your known jobs to finish. This method is useful when using the dpu / Processor class in a script.
Count the total number of jobs inside the joblist and refuse to process them if the total is more than nmax (default 1000). Goal is not to overload the queueing system.
Group jobs by type: ‘IND’, ‘SEQ’, ‘SYN’. Then take max DPU_TIME for all IND jobs (they run independently of anything) and add all SEQ and SYN jobs.
Xml file (or string) is parsed and ouput is a Python object Element, which can have Element children, etc … The xml attributes and character data is stored in Element.
If follow_symlinks is not set and src is a symbolic link, a new symlink will be created instead of copying the file it points to.
The destination directory must not already exist. If exception(s) occur, an Error is raised with a list of reasons.
If the optional symlinks flag is true, symbolic links in the source tree result in symbolic links in the destination tree; if it is false, the contents of the files pointed to by symbolic links are copied. If the file pointed by the symlink doesn’t exist, an exception will be added in the list of errors raised in an Error exception at the end of the copy process.
The optional ignore argument is a callable. If given, it is called with the src parameter, which is the directory being visited by copytree(), and names which is the list of src contents, as returned by os.listdir():
Since copytree() is called recursively, the callable will be called once for each directory that is copied. It returns a list of names relative to the src directory that should not be copied.
The optional copy_function argument is a callable that will be used to copy each file. It will be called with the source path and the destination path as arguments. By default, copy2() is used, but any function that supports the same signature (like copy()) can be used.
This is commonly everything needed to clean up a filename read from a configuration file, for example.
The elements of the list are path objects. This does not walk into subdirectories (see path.walkfiles).
the file. The default is None, meaning the content of the file is read as 8-bit characters and returned as a list of (non-Unicode) str objects.
Recursively move a file or directory to another location. This is similar to the Unix “mv” command. Return the file or directory’s destination.
If the destination is a directory or a symlink to a directory, the source is moved inside the directory. The destination path must not already exist.
A lot more could be done here… A look at a mv.c shows a lot of the issues this implementation glosses over.
If ignore_errors is set, errors are ignored; otherwise, if onerror is set, it is called to handle the error with arguments (func, path, exc_info) where func is platform and implementation dependent; path is the argument to that function that caused it to fail; and exc_info is a tuple returned by sys.exc_info(). If ignore_errors is false and onerror is None, an exception is raised.
the file. If present, the content of the file is decoded and returned as a unicode object; otherwise it is returned as an 8-bit str.
Set the access/modified times of this file to the current time. Create the file if it does not exist.
This performs a depth-first traversal of the directory tree. Each directory is returned just before all its children.
Return a datetime object for the date described by the given absdate (the number of days since January 1, 0000, an int or float), and optionally the abstime (number of seconds).
Return a sorted list of objects based on a given set of property names. If multiple property names are given, sort first on the first property name, and where multiple objects exist with the same first property name, sort by the second propertyname, and so on.
objects is an iterable of objects with persistent properties that can be sorted using rich comparison operators (>, <, >=, etc.)
Compress a list of strings retaining the first string followed by the unique portion of the remaining strings separated by separator.
the type will be determined as the minimum type required to hold the objects in the sequence. This argument can only be used to ‘upcast’ the array. For downcasting, use the .astype(t) method.
if __array__ returns a copy, obj is a nested sequence, or a copy is needed to satisfy any of the other requirements
This method returns the fractional number of seconds since that date. I.e. this is minus the integral number of days since that date.
Calculates the fractional lunar illumination (FLI) based on the Moon phase. The fli is about 10% off compared to Ephem. This is due to factors that are not taken into account here, such as the distance to the Moon etc.
Loosely follows tempfile.mktemp syntax, but constructs a name based on the current time (seconds since 00:00:00 1970-01-01 UTC) with microsecond precision followed by a 6 character random string. This should virtually guarantee temp-name uniqueness. The returned string has the following form:
If dir is not None, it will be supplied with a trailing ‘/’ if one is not given. If suffix is not ‘’, it will be preceeded by a ‘.’ if one is not given.
Returns a list of lists, which together contain all the items in “alist” where all but the last list have length equal to “group_size”.
Changes a list of numbers into a list of ranges with an increment of 1. A range is a tuple of two values, the first the starting value, the second the final value of a range.
Return a function to sort a list of objects based on a given set of properties. If multiple properties are given, sort first on the first property, and where multiple objects exist with the same first property, sort by the second property, and so on. For example:
objects is a list of objects with persistent properties that can be sorted using rich comparison operators (>, <, >=, etc.)
Give the difference between two sets of coordinates in units of arcsec. This is used to calculate the arcsec to move from coordinates (ra0, dec0) to coordinates (ra1, dec1), where the positive directions are North and East.
string, or cast -0 as a float (i.e., ‘-00:01:00’ or (-0.0, 1, 0), but not -0 or -00). If you don’t, the parser will not see it as negative and your Dec offset will likely have the wrong sign.
Open a file from a (un)compressed file and return the file object. Currently supports GZip and BZip2 archives and regular files.
Convert one (RA, Dec) coordinate into the corresponding (alt, az) coordinate with the help of HA (LST - RA) and latitude. HA can be supplied directly (in which case RA is redundant) or calculated. LST can be calculated with the help of MJD (or dt) and longitude if necessary. (alt, az) is returned in degrees above the horizon and degrees East of North, respectively.
Sorts the matched lists in the input dictionary by the value in the first key, and where multiple entires exist with the same first key value, sort by the second key value, and so on. Return a copy of the input dictionary sorted as specified.