BACKGROUND
contemporary informatics and genomic research require efficient, flexible and robust management of large heterogeneous data  <cit> , advanced computational tools  <cit> , powerful visualization  <cit> , reliable hardware infrastructure  <cit> , interoperability of computational resources  <cit> , and provenance of data and protocols  <cit> .
there are several alternative approaches for high-throughput analysis of large amounts of data such as using various types of shell-scripts  <cit>  and employing tool-specific graphical interfaces  <cit> .
the large-scale parallelization, increased network bandwidth, need for reproducibility, and wide proliferation of efficient and robust computational and communication resources are the driving forces behind this need for automation and high-throughput analysis.
there is a significant push to increase and improve resource development, enable the expansion of integrated databases and vibrant human/machine communications, and increase the distributed grid and network computing bandwidth  <cit> .
to meet these needs we present a new visual language programming framework for genomics and bioinformatics research based on heterogeneous graphical workflows.
we demonstrate the construction, validation and dissemination of several analysis protocols using a number of independent informatics and genomics software suites - miblast  <cit> , emboss  <cit> , mrfast  <cit> , gwass  <cit> , maq  <cit> , samtools  <cit> , bowtie  <cit> , etc.
these types of genomics and informatics tools were chosen as they span a significant component of the informatics research and, at the same time, they have symbiotic relations which enable their interoperability and integration.
the continual evolution and considerable variability of informatics and genomics data, software tools and web-service present challenges in the design, management, validation and reproducibility of advanced biomedical computing protocols.
there are a number of graphical environments for visual design of computational protocols , tool integration, interoperability and meta-analysis  <cit> .
most of them aim to enable new types of analyses, facilitate new applications, promote interdisciplinary collaborations, and simplify tool interoperability  <cit> .
compared to other environments, the pipeline offers several advantages, including a distributed grid-enabled, client-server, and fail-over-safe infrastructure, quick embedding of new tools within the pipeline computational library, and efficient dissemination of new protocols to the community.
table  <dig> provides a summary of the synergies and differences between the loni pipeline and several alternative graphical workflow environments.
additional comparisons between the loni pipeline and various alternative environments for software tool integration and interoperability are presented here  <cit> .
y = yes, n = no; taverna mir plug-in, mir = mygrid information repository; drmaa = distributed resource management application api.
previously, we have demonstrated a number of imaging  <cit> , brain mapping  <cit>  and neuroscientific  <cit>  applications using the pipeline environment.
one shape morphometry analysis protocol, using brainparser  <cit>  several shape manifold models of regional boundary  <cit>  implemented via the pipeline environment is illustrated on figure  <dig>  this is a complete neuroimaging solution  which automatically extracts, models and statistically analyzes local and regional shape differences between several cohorts based on raw magnetic resonance imaging data.
the results section below includes hands-on examples demonstrating specific informatics and genomics computing protocols for sequence data analysis and interoperability of heterogeneous bioinformatics resources.
ii.
implementation
as an external inter-resource mediating layer, the pipeline environment  <cit>  utilizes a distributed infrastructure model for mediating disparate data resources, software tools and web-services.
no software redesign or rebuilding modifications of the existing resources are necessary for their integration with other computational components.
the pipeline extensible markup language  schema enables the inter-resource communication and mediation layer.
each xml resource description contains important information about the tools location, the proper invocation protocol , run-time controls and data-types.
the pipeline xml schema http://pipeline.loni.ucla.edu/support/xml-overview/ also includes auxiliary metadata about the resource state, specifications, history, authorship, licensing, and bibliography.
using this resource metadata, the pipeline infrastructure facilitates the integration of disparate resources and provides a complete and comprehensive protocol provenance  <cit>  for the data, tools, hardware and results.
individual module descriptions and entire protocol xml objects are managed as .pipe files, facilitate the broad dissemination of resource metadata descriptions via web services, and promote constructive utilization of multidisciplinary tools and expertise by professionals, novice users and trainees.
in this paper we demonstrate the concrete details for utilizing several informatics and genomics suites of tools and show how such disparate resources may be integrated within the pipeline environment.
many additional resources  are also available within the core pipeline computational library  and more can easily be added to  pipeline server libraries following the protocols described below.
in addition to presenting a number of bioinformatics and genomics applications using the pipeline environment, this manuscript described some improvements of pipeline version  <dig>  over previous versions  <cit> , e.g., enhanced user-management, client-server pipeline web-start  interface, directory access control, and improved java authentication and authorization service interface.
pipeline architecture
the pipeline software architecture design is domain and hardware independent which makes the environment useful in different computational disciplines and on diverse hardware infrastructures.
the pipeline environment may be utilized in three synergistic mechanisms  <cit> .
the first one http://pipeline.loni.ucla.edu/downloads involves the local use of the pipeline client via connection to a remote server running natively on a hardware system which includes all appropriate plug-ins for system-specific grid managers, file systems, network, and communication protocols.
the second type of pipeline server distribution relies on virtualization technology.
the virtualized pipeline infrastructure provides end-users with the latest stable pre-compiled environment including all pre-installed open-source informatics tools.
the resulting pipeline virtual environment , contains the complete self-contained execution environment that can be run locally or on remote grid computing environment.
because the pipeline virtualization environment tightly mirrors that of the loni grid http://www.loni.ucla.edu/twiki/bin/view/infrastructure/gridcomputing, users gain access to all loni image processing, brain mapping and informatics tools and services.
version  <dig>  of the virtual pipeline environment is based on ubuntu http://www.ubuntu.com and vmware http://www.vmware.com technologies.
the third pipeline distribution mechanism is called distributed pipeline server .
this distribution includes a user-friendly graphical user interface  for automated native system configuration, installation and deployment of the pipeline server, the available xml computational library, and back-end software tools.
the pipeline environment uses a client-server architecture, but each pipeline client may also act as a local server and manage job submission and execution.
following proper authentication, the process of a client submitting a workflow for execution to a specified server prompts the server to translate  the workflow into parallel jobs and send them to the grid resource manager which in turn farms these to the back-end grid .
when a job is complete, the server retrieves the results from the grid resource manager and sends out subsequent jobs from the active workflow.
the client receives status updates from the server at regular intervals, figure  <dig>  currently, the pipeline server supports distributed resource management application api  interface and java gridengine database interface  to communicate to the grid resource manager.
these include many of the popular grid resource managers, including sun/oracle grid engine http://en.wikipedia.org/wiki/oracle_grid_engine, gridway http://www.gridway.org, pbs/torque http://www.clusterresources.com.
software implementation
the entire pipeline source code is in java, including the client, server, network, execution and database components.
the pipeline server has a grid plug-in component, which provides communication between the pipeline server and grid resource manager.
in addition to jgdi and drmaa support, it supports custom plug-in of grid resource managers.
the api for grid plug-in is available on the pipeline website http://pipeline.loni.ucla.edu/support/server-guide/pipeline-grid-plugin-api-developers-guide/.
ii.
<dig> software development
user-management
as the hardware resources  are generally limited and the available pipeline servers are finite, a handful of heavy users may disproportionately monopolize the pipeline services and the underlying hardware infrastructure.
to prevent this from occurring, a fair-usage policy is implemented to manage the sharing of these limited resources.
this policy, referred to as "user management," ensures that no users may utilize more than a predefined percent  of the available resources at a fixed time.
different pipeline servers may have different usage percentage values.
the number submissions allowed is dynamic and changes after each new job is submitted, as it depends on the number of available nodes/slots and the number of user-specific jobs already scheduled.
for instance, a running pipeline server which has only  <dig> slots available where the limit percent value is set to  <dig> would allow the first user to utilize no more than  <dig> slots.
the pipeline user-manager calculates the number of slots the user can use with the following formula:
where: t is the total number of available slots, u is the number of currently used slots by all the users, uc is the number of used slots by current user only, and p is the limit percent value specified by the pipeline server administrator.
this user management protocol significantly improves the server usability and allows each user to submit at least part of their jobs in real time, although it slightly reduces the optimal usage of the server.
directory access control
the pipeline server allows administrator control over the user access to binary executables, workflows/modules and server files .
server administrators may also specify a list of users who can, or cannot, access a list of directories.
this is a convenient feature when a pipeline server supports several different categories of groups of users and some server files should not be visible or executable by various users or groups.
user authentication using jaas
the pipeline authenticates users using the java authentication and authorization service http://java.sun.com/javaee/security/, which allows the server operator to authenticate usernames and passwords against any type of system.
when a user connects to a pipeline server, the pipeline tries to create a new jaas object called logincontext and if the creation is successful, attempts to call the object's login method.
if the method returns "true", then the pipeline allows the user to continue.
otherwise the user is disconnected from the server with an "authentication rejected" message.
file permission on shared temporary directories
each pipeline server has one directory where all the temporary files are stored.
files located in this directory need to be accessible only by the pipeline server administrator and by the user who started the workflow.
pipeline server has a special flag in its preferences file which enables this feature and creates special permissions for each file in the temporary directory, which enables safe and secure management of files in the temporary directory.
stability
the performance of the pipeline server  has significantly improved over v. <dig>  <cit>  in terms of client-server communication, server reliability and stability.
this was accomplished by introducing architectural changes, bug-fixes , and additional new features like failover, grid plug-ins and array job-submission.
failover: the server failover feature improves robustness and minimizes service disruptions in the case of a single pipeline server failure.
it is available on servers running linux or other unix operating systems.
the core of failover is running two actual pipeline servers in parallel, a primary and a secondary, a virtual pipeline server name, and de-coupling and running the persistence database on a separate system.
each of the two servers monitors the state of its counterpart.
in the event that the primary server with the virtual pipeline server name has a catastrophic failure, the secondary server will assume the virtual name, establish a connection to the persistence database, take ownership of all current pipeline jobs dynamically, and restart the formerly primary server as secondary.
grid plug-ins: this feature allows pipeline to run a new grid plug-in process instead of attaching the plug-in to the actual pipeline process.
this makes grid plug-ins and resource management libraries isolated from the pipeline server and prevents server crashes if any of the lower level libraries crash.
after having this feature, the server up time has been dramatically increased.
array jobs: the pipeline server now supports array job submission which improves the total processing time of a workflow by combining repeated jobs into one array job.
each job in the array has its own output stream and error stream.
this feature increases the speed of job submission especially for modules with multiple instances .
depending of the module's number of instances, there is 10-65% speed improvement when using array jobs versus sequential individual job submissions.
this feature is configurable and server administrator may set preferences about how array jobs will be submitted on each pipeline server.
ii.
<dig> implementation of genomics and informatics pipeline protocols
the following steps are necessary to develop a complete pipeline biomedical solution to a well-defined computational challenge - protocol design, tool installation, module definition, workflow implementation, workflow validation, and dissemination of the resulting workflow for broader community-based testing and utilization.
each of these steps is described in detail below and screencasts and videos demonstrating these steps are available online http://pipeline.loni.ucla.edu/support/.
protocol design
this is the most important step in the design of a new pipeline graphical workflow to solve a specific informatics or genetics problem and typically involves multidisciplinary expert users with sufficient scientific and computational expertise.
in practice, most genomics challenges may be approached in several different ways and the final pipeline xml workflow will greatly depend on this initial step.
in this step, it may be most appropriate to utilize a top-down approach for outlining the general classes of sequence data analysis, then the appropriate sub-classes of analyses, specific tools, test-data, invocation of concrete tools, and a detailed example of executable syntax for each step in the protocol.
below is a hierarchical example of a discriminative design for a sequence alignment and assembly protocol demonstrated in the results section.
these steps are not different from other approaches for developing and validating informatics and genomics protocols, however the explicit hierarchical formulation of these steps is only done once by the expert user.
all subsequent protocol redesigns, modifications and extensions may be accomplished  directly on the graphical representation of the protocol with the pipeline graphical user interface .
table  <dig> contains an example of the specification of an alignment and assembly protocol, which is also demonstrated as a complete pipeline genomics solution on figure  <dig>
this protocol is implemented as a pipeline graphical workflow and demonstrated in the results section.
figure  <dig> shows the corresponding pipeline graphical workflow implementing this genomics analysis protocol.
tool installation
once the protocol is finalized, the workflow designer and the administrator of the pipeline server need to ensure that all tools  are available at the specified server locations.
note that tools are installed in specific locations which may be varying for different hardware platforms and sites.
the pipeline library manager http://pipeline.loni.ucla.edu/support/server-guide/configuration/ facilitates the portability of the xml-based pipeline graphical workflows by defining a hash-map between different software suites, computational tools, versions and executable locations.
thus a well-defined pipeline graphical workflow only references the software suite, its version and the specific tool necessary for the specific computational task.
the pipeline server then interprets, maps and constructs the specific executable commands which are relayed to the grid manager as concrete jobs and scheduled for execution.
module definition
the module definition is accomplished via the pipeline gui.
each of the executable processes needs to be described individually and independently as a node  in the workflow graph http://pipeline.loni.ucla.edu/support/user-guide/creating-modules/.
this step also includes the independent testing and validation of the execution of each of the individual nodes  using appropriate data.
the result of this module definition step is an xml file  representing the tool invocation syntax, which can be broadly used, shared, modified, extended and integrated with other module descriptions to form complex graphical workflow protocols.
workflow implementation
this protocol skeletonization process is important as it lays out the flow of the data and indicates the complete data analysis provenance http://pipeline.loni.ucla.edu/support/user-guide/building-a-workflow/.
after all necessary modules are independently defined and validated, we need to integrate them into a coherent and scientifically-valid pipeline workflow.
frequently the developer makes use of module groupings to abstract computational complexity, conditional and looping modules to direct the processing flow, and specify data sources  and data sinks .
in addition, workflow-wide and module-specific meta-data documentation is also provided in this step.
these include appropriate workflow preferences, variables, references/citations , urls, licenses, required vs. optional parameters, inputs, outputs and run-time controls, etc.
workflow validation
the workflow validation step involves the testing and fine-tuning of the implemented workflow to ensure the results of each intermediate step as well as the final results are reasonable, appropriately captured and saved, and tested against alternative protocols .
this automated workflow validation step ensures that input datasets and the output results are well-defined, verifies the provenance information about the pipeline workflow, as well as provides user documentation, limitations, assumptions, potential problems and solutions, usability and support annotation.
dissemination
validated pipeline workflows may be disseminated as xml documents via email, web-sites and pipeline server libraries, as well as biositemaps objects  <cit>  in xml or rdf format http://www.biositemaps.org.
any valid pipeline workflow may be loaded by any remote pipeline client.
however, execution of a particular workflow may require access to a pipeline server where all tools referenced in the workflow are available for execution, the user has the appropriate credentials to access the remote pipeline servers, data, software tools and services.
in addition, some minor workflow modifications may be necessary prior to the execution of the pipeline workflow  although, each pipeline client is itself a server, typical users would not run workflows on the same  machine but rather remotely login to a pipeline server to outsource the computing-intensive tasks.
pipeline clients can disconnect and reconnect frequently to multiple pipeline servers to submit workflows and monitor the status of running workflows in real time.
iii.
RESULTS
we now demonstrate the complete process of installing, xml-wrapping , employing and integrating tools from several informatics software suites - miblast   <cit> , emboss   <cit> , mrfast   <cit> , gwass   <cit> , maq   <cit> , samtools   <cit> , and bowtie  <cit> .
each of these packages includes a large number of tools and significant capabilities.
the pipeline xml module definitions for some of the tools within these packages may not yet be implemented, or may be incompletely defined within the pipeline library.
however, following this step-by-step guideline, the entire community may extend and improve the existing, as well as describe and distribute additional, pipeline xml module wrappers for other tools within these suites.
additional genomics and informatics suites and resources may also be similarly described in the pipeline xml syntax and made available to the community, as needed.
iii.
<dig> miblast
◦ url: http://www.eecs.umich.edu/~jignesh/miblast/
◦ description: miblast is a tool for efficiently blasting a batch of nucleotide sequence queries.
such batch workloads contain a large number of query sequences .
these batch workloads can be evaluated by blasting each individual query one at time, but this method is very slow for large batch sizes.
◦ installation: the downloading, installation and configuration of the miblast suite on linux os kernel takes only  <dig> minutes following these instructions: http://www.eecs.umich.edu/~jignesh/miblast/installation.html.
◦ pipeline workflow
▪ xml metadata description: the module descriptions for each of the nodes in the pipeline workflow took about  <dig> minutes each.
the design of the complete workflow took  <dig> hours because this suite generates many implicit outputs and is intended to be run each time from the core build directory.
this presents a challenge for multiple users using the same routines and generating the same implicit filenames .
to circumvent this problem, we added several auxiliary modules in the beginning  and at the end .
notice that wrapper shell-scripts had to also be developed to address the problems with implicit output filenames.
finally, flow-of-control connections, in addition to standard data-passing connections, were utilized to direct the execution of the entire protocol.
▪ name: miblast_workflow.pipe
▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfoblast
▪ screenshots:
• input: figure  <dig> shows a snapshot of the input parameters  for the corresponding pipeline workflow.
• pipeline execution: figure  <dig> shows the completed miblast pipeline workflow and a fragment of the output alignment result.
• output: table  <dig> contains the beginning of the output result from the miblast workflow.
▪ approximate time to complete: 20- <dig> minutes.
iii.
<dig> emboss
◦ url: http://emboss.sourceforge.net/
◦ tool: emboss matcher
◦ description: finds the best local alignments between two sequences.
it can be used to compare two sequences looking for local sequence similarities using a rigorous algorithm.
◦ installation: the downloading, installation and configuration of the entire emboss suite on linux os kernel takes  <dig> minutes following these instructions: http://emboss.sourceforge.net/docs/faq.html.
◦ pipeline workflow
▪ xml metadata description: the pipeline xml for a number of emboss tools are available.
each of these metadata module descriptions was complete via the pipeline module gui and took about 30- <dig> minutes.
as a demonstration, only the emboss matcher module is presented here.
▪ name: emboss_matcher.pipe
▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfoemboss
▪ screenshots:
• input: figure  <dig> shows the input parameters  for the corresponding pipeline workflow.
• pipeline execution: figure  <dig> demonstrates the completed execution of this emboss module.
• output: table  <dig> contains the beginning of the output result from the emboss matcher alignment.
▪ approximate time to complete:  <dig> minutes.
iii.
<dig> mrfast
◦ url: http://mrfast.sourceforge.net
◦ description: mrfast is designed to map short  reads generated with the illumina platform to reference genome assemblies in a fast and memory-efficient manner.
◦ installation: the downloading, installation and configuration of the mrfast suite on linux os kernel takes only  <dig> minutes following these instructions: http://mrfast.sourceforge.net/manual.html.
◦ pipeline workflow
▪ xml metadata description: the example includes pipeline module descriptions of the mrfast fasta-indexing  and fasta-mapping  tools.
each of these metadata module descriptions was complete via the pipeline module gui and took about 10- <dig> minutes.
▪ name: mrfast_indexing_mapping.pipe
▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfomrfast
▪ screenshots:
• input: figure  <dig> shows the input parameters  for the corresponding pipeline workflow.
• pipeline execution: figure  <dig> demonstrates the completed execution of this mrfast workflow.
• output: table  <dig> contains the beginning of the output result from the mrfast workflow.
▪ approximate time to complete:  <dig> minutes.
iii.
<dig> gwass
◦ url: http://www.stats.ox.ac.uk/~marchini/software/gwas/gwas.html
◦ description: the genome-wide association study software  is a suite of tools facilitating the analysis of genome-wide association studies.
these tools were used in the design and analysis of the  <dig> genome-wide association studies carried out by the wellcome trust case-control consortium .
one specific example of a gwass tool is impute v. <dig>  which is used for genotype imputation and phasing https://mathgen.stats.ox.ac.uk/impute/impute_v <dig> html.
◦ installation: the downloading, installation and configuration of the gwass suite on linux os kernel takes about  <dig> minutes.
▪ xml metadata description: here we demonstrate the pipeline module description of impute, which is a program for genotype imputation in genome-wide association studies and provides fine-mapping based on a dense set of marker data .
▪ name: gwass_impute.pipe
▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfgwass
▪ screenshots:
• input: figure  <dig> shows the input parameters  for the corresponding pipeline workflow.
• pipeline execution: figure  <dig> demonstrates the completed execution of this gwass impute module.
• output: table  <dig> contains the beginning of the output result from the gwass impute module.
▪ approximate time to complete: 2- <dig> minutes.
iii.
<dig> interoperabilities between independently developed informatics resources
there are 100's of examples of heterogeneous pipeline graphical workflows that illustrate the interoperability between imaging tools independently developed for different purposes by different investigators at remote institutions.
these can be found under the workflows section of the pipeline library, figure  <dig>  as well as on the web at: http://www.loni.ucla.edu/twiki/bin/view/loni/pipeline_genomicsinformatics.
similarly, bioinformatics and genomics investigators can construct novel computational sequence analysis protocols using the available suites of informatics resources .
below is one example of integrating the informatics tools described above.
iii.
<dig>  indexing and mapping
in this example, we illustrate how to integrate mrfast and emboss water informatics tools.
the left branch of the pipeline workflow uses emboss to align  <dig> sequences  using water.
the right branch of the workflow does fast alignment using  <dig> independent datasets  and  <dig> separate mrfast references .
the interesting demonstration here is that the emboss water output  is later directly used as derived data sequence which is re-aligned to the  <dig> mrfast reference sequences.
◦ pipeline workflow
▪ design: this tool-interoperability example illustrates feeding the output of the emboss water module  as in input in mrfast indexing module and subsequent mapping using mrfast.
▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfomrfast
▪ name: bioinfo_integratedworkflow_emboss_water_mrfast.pipe
▪ screenshots:
• inputs: figure  <dig> shows the input parameters  for the corresponding pipeline workflow.
• pipeline execution: figure  <dig> demonstrates the completed execution of this emboss/mrfast workflow.
▪ output: table  <dig> contains the beginning of the output result from this heterogeneous workflow.
▪ approximate time to complete:  <dig> minutes.
iii.
<dig>  alignment and assembly
another interesting example illustrating the power of tool interoperability using the pipeline environment involved the alignment and assembly computational genomics protocol we presented in the implementation section.
in this example, we demonstrate the interoperability between maq, samtools, and bowtie tools.
◦ pipeline workflow
▪ design: this genomics pipeline workflow begins with an optional preprocessing step extracting a sub-sequence of the genomic sequence .
then, the illumina sequencing data  are converted first to fastq and later  to binary fastq file  format.
the data is then aligned to a reference genome, the map file is converted to bam file and the reference genome is indexed.
the alignment map file is converted to samtools  format first and then to a binary bam file.
next, duplicated reads are removed, the bam file is sorted, md tagged and indexed.
in addition the maq-based alignment, bowtie was used  to align the data to the reference sequence.
such alternative paths in the data processing protocol are easily constructed and modified in the pipeline environment.
▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfomaq
▪ name: maq_samtools_bowtie_integrated_cranium.pipe
▪ screenshots:
• inputs: figure  <dig> shows the input parameters  for the corresponding pipeline workflow.
• pipeline execution: figure  <dig> demonstrates the completed execution of this heterogeneous pipeline workflow.
▪ output: table  <dig> contains the beginning of the output result from this heterogeneous workflow.
▪ approximate time to complete: 4- <dig> minutes.
iv.
CONCLUSIONS
this paper reports on the applications of the pipeline environment  <cit>  to address two types of computational genomics and bioinformatics challenges - graphical management of diverse suites of tools, and the interoperability of heterogeneous software.
specifically, this manuscript presents the concrete details of deploying general informatics suites and individual software tools to new hardware infrastructures, the design, validation and execution of new visual analysis protocols via the pipeline graphical interface, and integration of diverse informatics tools via the pipeline xml syntax.
we demonstrate each of these three protocols using several established informatics packages - miblast , emboss , mrfast , gwass , maq , samtools , and bowtie.
these examples demonstrate informatics and genomics applications using the pipeline graphical workflow environment.
the pipeline is a platform-independent middleware infrastructure enabling the description of diverse informatics resources with well-defined invocation protocol - syntax for dynamic specification of the inputs, outputs and run-time control parameters.
these resource xml descriptions may be provided using the pipeline graphical user interface or using machine code according to the pipeline xsd schema.
the pipeline environment provides a powerful and flexible infrastructure for efficient biomedical computing and distributed informatics research.
the interactive pipeline resource management enables the utilization and interoperability of diverse types of informatics resources.
the pipeline client-server model provides computational power to a broad spectrum of informatics investigators - experienced developers and novice users, the computational-resources haves and have-nots, as well as between basic and translational scientists.
the open development, validation and dissemination of computational networks  facilitates the sharing of knowledge, tools, protocols and best practices, and enables the unbiased validation and replication of scientific findings by the entire community.
for specific research domains, applications and needs, there are pros and cons for using the pipeline environment or any of the alternative tools and infrastructures for visual informatics and computational genomics.
examples of powerful alternatives include taverna  <cit> , kepler  <cit> , triana  <cit> , galaxy  <cit> , avs  <cit> , vistrails  <cit> , bioclipse  <cit> , knime  <cit> , and others.
the main advantages of the pipeline environment are the distributed client-server architecture with diverse arrays of grid plug-ins, the lightweight data, tools and services utilization and the dynamic workflow design, validation, execution, monitoring and dissemination of complete end-to-end computaitonal solutions.
v. availability and requirements
• project name: pipeline environment
• project home pages:
◦ loni: http://pipeline.loni.ucla.edu
◦ nitrc: http://www.nitrc.org/projects/pipeline
◦ birn: http://www.birncommunity.org/tools-catalog/loni-pipeline
◦ bioinformatics.org: http://www.bioinformatics.org/pipeline
◦ try the loni pipeline informatics and genomics workflows online without any software installation using anonymous guest account: http://pipeline.loni.ucla.edu/pws
• operating system: pipeline clients and servers are platform-independent, while some features  require the server run on linux/unix os.
the distributed pipeline server  graphical user interface, which installs the pipeline server, grid engine, and computational imaging and informatics software tools, require standard linux os kernels.
the pipeline web start  allows users to start the pipeline application directly from the web browser and run it locally without any installation.
it has all the features and functionality of the downloadable stand-alone pipeline application and allows anonymous guest access or user-authentication to connect to remote pipeline servers.
• programming language: pure java.
• other requirements:
◦ requirements summary: the pipeline client and server can run on any system that is supported by java runtime environment   <dig>  or higher.
windows pipeline servers will not be able to use privilege escalation.
three-tier failover feature is only supported by unix/linux systems.
all other features are available for all platforms.
most distributed pipeline servers require 300- <dig> mb memory, which may depend on the load and garbage collection preferences.
◦ for distributed multicore deployment, the distributed pipeline server  requires a grid manager , which is provided with the dps distribution.
the pipeline server will still work on a platform without a grid manager, however, jobs may not be processed in parallel and performance on multicore machines may be suboptimal.
◦ complete requirements:
▪ client: http://pipeline.loni.ucla.edu/support/user-guide/installation/
▪ server: http://pipeline.loni.ucla.edu/support/server-guide/installation/
▪ dps: http://pipeline.loni.ucla.edu/dps
▪ pws: http://pipeline.loni.ucla.edu/pws
• license: apache-derived software license http://www.loni.ucla.edu/policies/loni_softwareagreement.shtml.
• caution: there are some potential limitations of the pipeline environment and its current collection of data, tools services and computational library :
◦ each new informatics tool which needs to be accessible as a processing module within the pipeline environment needs to be described manually by an expert using the pipeline gui or automatically using a properly configured xml exporter .
then the pipeline xml module description can be shared with other users.
◦ to run available pipeline workflows  on remote pipeline-servers, users need to have accounts on the remote pipeline servers.
in addition,  <dig> types of updates may be necessary in the pipe files - the server-name references of data sources , data sinks , and executables, as well as the path references to the data sources, sinks and executables.
the server-name can be easily updated using server changer tool in pipeline .
user has to edit path references on some or all of the data sources, sinks and executables for their server.
no workflow modifications are necessary for executing these pipeline workflows on the loni pipeline cranium server; however this requires a loni pipeline user-account http://www.loni.ucla.edu/collaboration/pipeline/pipeline_application.jsp.
a proper administrator configuration of the distributed pipeline server  will resolve the need for such revisions by the user.
◦ some computational tools may require wrapper scripts that call the raw executable binaries.
these scripts  are then invoked via the pipeline environment.
example situations include tools that have implicit outputs, or if the tools routinely return non-trivial exit codes, distinct from zero.
such problems may cause the pipeline environment to halt execution of subsequent modules, because of a broken module-to-module communication protocol.
◦ smartlines, which auto-convert between different informatics data formats, need to be extended to handle informatics and genomics data .
◦ access to external informatics databases may need to be customized - e.g., pdb http://www.rcsb.org, scop http://scop.mrc-lmb.cam.ac.uk/scop, genbank http://www.ncbi.nlm.nih.gov/genbank, etc.
◦ native vs. virtual pipeline server: the fully distributed pipeline server  architecture  provides  both the pipeline middleware as well as installers for all computational tools available on the loni cranium pipeline grid server http://pipeline.loni.ucla.edu/support/user-guide/interface-overview/.
the virtual pipeline server and pipeline clients also provide the complete pipeline environment for a virtual vmware invocation http://pipeline.loni.ucla.edu/pnve.
• any restrictions to use by non-academics: free for non-commercial research purposes.
competing interests
the authors declare that they have no competing interests.
authors' contributions
all authors have read and approved the final manuscript.
all authors made substantive intellectual contributions to this research and development effort and the drafting of the manuscript.
specifically, the authors contributed as follows: idd article conception, study design, workflow implementation and applications, data processing, drafting and revising the manuscript; ft study design, workflow implementation, data processing, drafting the manuscript; fm study design, workflow implementation, drafting the manuscript; pp pipeline software implementation, workflow design and drafting the manuscript; zl pipeline software implementation, workflow design and drafting the manuscript; az pipeline software implementation, study design, workflow development and drafting the manuscript; pe pipeline software engineering, workflow design and drafting the manuscript; jp pipeline server grid implementation and management, workflow development and drafting the manuscript; ag study design, workflow implementation and drafting the manuscript, jak and apc study design protocol, data processing, workflow development and drafting the manuscript; jdvh study design protocol and drafting the manuscript; ja workflow applications and drafting the manuscript; ck study design, workflow applications and drafting the manuscript; awt study design, engineering of the pipeline environment, and drafting the manuscript.
acknowledgements and funding
this work was funded in part by the national institutes of health through grants u <dig> rr <dig>  p <dig> rr <dig>  r <dig> mh <dig>  u24-rr <dig>  u24-rr <dig>  u24-rr <dig> and u24-rr <dig>  we are also indebted to the members of the laboratory of neuro imaging , the biomedical informatics research network , the national centers for biomedical computing  and clinical and translational science award  investigators, nih program officials, and many general users for their patience with beta-testing the pipeline and for providing useful feedback about its state, functionality and usability.
benjamin berman and zack ramjan from the usc epigenome center, university of southern california, provided help with definition of sequence pre-processing.
for more information on loni and the loni pipeline, please go to http://www.loni.ucla.edu and http://pipeline.loni.ucla.edu.

 contemporary informatics and genomic research require efficient, flexible and robust management of large heterogeneous data, advanced computational tools, powerful visualization, reliable hardware infrastructure, interoperability of computational resources, and provenance of data and protocols. there are several alternative approaches for high-throughput analysis of large amounts of data such as using various types of shell-scripts and employing tool-specific graphical interfaces. the large-scale parallelization, increased network bandwidth, need for reproducibility, and wide proliferation of efficient and robust computational and communication resources are the driving forces behind this need for automation and high-throughput analysis. there is a significant push to increase and improve resource development, enable the expansion of integrated databases and vibrant human/machine communications, and increase the distributed grid and network computing bandwidth. to meet these needs we present a new visual language programming framework for genomics and bioinformatics research based on heterogeneous graphical workflows. we demonstrate the construction, validation and dissemination of several analysis protocols using a number of independent informatics and genomics software suites - miblast, emboss, mrfast, gwass, maq, samtools, bowtie, etc. these types of genomics and informatics tools were chosen as they span a significant component of the informatics research and, at the same time, they have symbiotic relations which enable their interoperability and integration. the continual evolution and considerable variability of informatics and genomics data, software tools and web-service present challenges in the design, management, validation and reproducibility of advanced biomedical computing protocols. there are a number of graphical environments for visual design of computational protocols, tool integration, interoperability and meta-analysis. most of them aim to enable new types of analyses, facilitate new applications, promote interdisciplinary collaborations, and simplify tool interoperability. compared to other environments, the pipeline offers several advantages, including a distributed grid-enabled, client-server, and fail-over-safe infrastructure, quick embedding of new tools within the pipeline computational library, and efficient dissemination of new protocols to the community. table provides a summary of the synergies and differences between the loni pipeline and several alternative graphical workflow environments. additional comparisons between the loni pipeline and various alternative environments for software tool integration and interoperability are presented here. y = yes, n = no; taverna mir plug-in, mir = mygrid information repository; drmaa = distributed resource management application api. previously, we have demonstrated a number of imaging, brain mapping and neuroscientific applications using the pipeline environment. one shape morphometry analysis protocol, using brainparser several shape manifold models of regional boundary implemented via the pipeline environment is illustrated on figure this is a complete neuroimaging solution which automatically extracts, models and statistically analyzes local and regional shape differences between several cohorts based on raw magnetic resonance imaging data. the results section below includes hands-on examples demonstrating specific informatics and genomics computing protocols for sequence data analysis and interoperability of heterogeneous bioinformatics resources. ii. implementation as an external inter-resource mediating layer, the pipeline environment utilizes a distributed infrastructure model for mediating disparate data resources, software tools and web-services. no software redesign or rebuilding modifications of the existing resources are necessary for their integration with other computational components. the pipeline extensible markup language schema enables the inter-resource communication and mediation layer. each xml resource description contains important information about the tools location, the proper invocation protocol, run-time controls and data-types. the pipeline xml schema http://pipeline.loni.ucla.edu/support/xml-overview/ also includes auxiliary metadata about the resource state, specifications, history, authorship, licensing, and bibliography. using this resource metadata, the pipeline infrastructure facilitates the integration of disparate resources and provides a complete and comprehensive protocol provenance for the data, tools, hardware and results. individual module descriptions and entire protocol xml objects are managed as.pipe files, facilitate the broad dissemination of resource metadata descriptions via web services, and promote constructive utilization of multidisciplinary tools and expertise by professionals, novice users and trainees. in this paper we demonstrate the concrete details for utilizing several informatics and genomics suites of tools and show how such disparate resources may be integrated within the pipeline environment. many additional resources are also available within the core pipeline computational library and more can easily be added to pipeline server libraries following the protocols described below. in addition to presenting a number of bioinformatics and genomics applications using the pipeline environment, this manuscript described some improvements of pipeline version over previous versions, e.g., enhanced user-management, client-server pipeline web-start interface, directory access control, and improved java authentication and authorization service interface. pipeline architecture the pipeline software architecture design is domain and hardware independent which makes the environment useful in different computational disciplines and on diverse hardware infrastructures. the pipeline environment may be utilized in three synergistic mechanisms. the first one http://pipeline.loni.ucla.edu/downloads involves the local use of the pipeline client via connection to a remote server running natively on a hardware system which includes all appropriate plug-ins for system-specific grid managers, file systems, network, and communication protocols. the second type of pipeline server distribution relies on virtualization technology. the virtualized pipeline infrastructure provides end-users with the latest stable pre-compiled environment including all pre-installed open-source informatics tools. the resulting pipeline virtual environment, contains the complete self-contained execution environment that can be run locally or on remote grid computing environment. because the pipeline virtualization environment tightly mirrors that of the loni grid http://www.loni.ucla.edu/twiki/bin/view/infrastructure/gridcomputing, users gain access to all loni image processing, brain mapping and informatics tools and services. version of the virtual pipeline environment is based on ubuntu http://www.ubuntu.com and vmware http://www.vmware.com technologies. the third pipeline distribution mechanism is called distributed pipeline server. this distribution includes a user-friendly graphical user interface for automated native system configuration, installation and deployment of the pipeline server, the available xml computational library, and back-end software tools. the pipeline environment uses a client-server architecture, but each pipeline client may also act as a local server and manage job submission and execution. following proper authentication, the process of a client submitting a workflow for execution to a specified server prompts the server to translate the workflow into parallel jobs and send them to the grid resource manager which in turn farms these to the back-end grid. when a job is complete, the server retrieves the results from the grid resource manager and sends out subsequent jobs from the active workflow. the client receives status updates from the server at regular intervals, figure currently, the pipeline server supports distributed resource management application api interface and java gridengine database interface to communicate to the grid resource manager. these include many of the popular grid resource managers, including sun/oracle grid engine http://en.wikipedia.org/wiki/oracle_grid_engine, gridway http://www.gridway.org, pbs/torque http://www.clusterresources.com. software implementation the entire pipeline source code is in java, including the client, server, network, execution and database components. the pipeline server has a grid plug-in component, which provides communication between the pipeline server and grid resource manager. in addition to jgdi and drmaa support, it supports custom plug-in of grid resource managers. the api for grid plug-in is available on the pipeline website http://pipeline.loni.ucla.edu/support/server-guide/pipeline-grid-plugin-api-developers-guide/. ii. software development user-management as the hardware resources are generally limited and the available pipeline servers are finite, a handful of heavy users may disproportionately monopolize the pipeline services and the underlying hardware infrastructure. to prevent this from occurring, a fair-usage policy is implemented to manage the sharing of these limited resources. this policy, referred to as "user management," ensures that no users may utilize more than a predefined percent of the available resources at a fixed time. different pipeline servers may have different usage percentage values. the number submissions allowed is dynamic and changes after each new job is submitted, as it depends on the number of available nodes/slots and the number of user-specific jobs already scheduled. for instance, a running pipeline server which has only slots available where the limit percent value is set to would allow the first user to utilize no more than slots. the pipeline user-manager calculates the number of slots the user can use with the following formula: where: t is the total number of available slots, u is the number of currently used slots by all the users, uc is the number of used slots by current user only, and p is the limit percent value specified by the pipeline server administrator. this user management protocol significantly improves the server usability and allows each user to submit at least part of their jobs in real time, although it slightly reduces the optimal usage of the server. directory access control the pipeline server allows administrator control over the user access to binary executables, workflows/modules and server files. server administrators may also specify a list of users who can, or cannot, access a list of directories. this is a convenient feature when a pipeline server supports several different categories of groups of users and some server files should not be visible or executable by various users or groups. user authentication using jaas the pipeline authenticates users using the java authentication and authorization service http://java.sun.com/javaee/security/, which allows the server operator to authenticate usernames and passwords against any type of system. when a user connects to a pipeline server, the pipeline tries to create a new jaas object called logincontext and if the creation is successful, attempts to call the object's login method. if the method returns "true", then the pipeline allows the user to continue. otherwise the user is disconnected from the server with an "authentication rejected" message. file permission on shared temporary directories each pipeline server has one directory where all the temporary files are stored. files located in this directory need to be accessible only by the pipeline server administrator and by the user who started the workflow. pipeline server has a special flag in its preferences file which enables this feature and creates special permissions for each file in the temporary directory, which enables safe and secure management of files in the temporary directory. stability the performance of the pipeline server has significantly improved over v. in terms of client-server communication, server reliability and stability. this was accomplished by introducing architectural changes, bug-fixes, and additional new features like failover, grid plug-ins and array job-submission. failover: the server failover feature improves robustness and minimizes service disruptions in the case of a single pipeline server failure. it is available on servers running linux or other unix operating systems. the core of failover is running two actual pipeline servers in parallel, a primary and a secondary, a virtual pipeline server name, and de-coupling and running the persistence database on a separate system. each of the two servers monitors the state of its counterpart. in the event that the primary server with the virtual pipeline server name has a catastrophic failure, the secondary server will assume the virtual name, establish a connection to the persistence database, take ownership of all current pipeline jobs dynamically, and restart the formerly primary server as secondary. grid plug-ins: this feature allows pipeline to run a new grid plug-in process instead of attaching the plug-in to the actual pipeline process. this makes grid plug-ins and resource management libraries isolated from the pipeline server and prevents server crashes if any of the lower level libraries crash. after having this feature, the server up time has been dramatically increased. array jobs: the pipeline server now supports array job submission which improves the total processing time of a workflow by combining repeated jobs into one array job. each job in the array has its own output stream and error stream. this feature increases the speed of job submission especially for modules with multiple instances. depending of the module's number of instances, there is 10-65% speed improvement when using array jobs versus sequential individual job submissions. this feature is configurable and server administrator may set preferences about how array jobs will be submitted on each pipeline server. ii. implementation of genomics and informatics pipeline protocols the following steps are necessary to develop a complete pipeline biomedical solution to a well-defined computational challenge - protocol design, tool installation, module definition, workflow implementation, workflow validation, and dissemination of the resulting workflow for broader community-based testing and utilization. each of these steps is described in detail below and screencasts and videos demonstrating these steps are available online http://pipeline.loni.ucla.edu/support/. protocol design this is the most important step in the design of a new pipeline graphical workflow to solve a specific informatics or genetics problem and typically involves multidisciplinary expert users with sufficient scientific and computational expertise. in practice, most genomics challenges may be approached in several different ways and the final pipeline xml workflow will greatly depend on this initial step. in this step, it may be most appropriate to utilize a top-down approach for outlining the general classes of sequence data analysis, then the appropriate sub-classes of analyses, specific tools, test-data, invocation of concrete tools, and a detailed example of executable syntax for each step in the protocol. below is a hierarchical example of a discriminative design for a sequence alignment and assembly protocol demonstrated in the results section. these steps are not different from other approaches for developing and validating informatics and genomics protocols, however the explicit hierarchical formulation of these steps is only done once by the expert user. all subsequent protocol redesigns, modifications and extensions may be accomplished directly on the graphical representation of the protocol with the pipeline graphical user interface. table contains an example of the specification of an alignment and assembly protocol, which is also demonstrated as a complete pipeline genomics solution on figure this protocol is implemented as a pipeline graphical workflow and demonstrated in the results section. figure shows the corresponding pipeline graphical workflow implementing this genomics analysis protocol. tool installation once the protocol is finalized, the workflow designer and the administrator of the pipeline server need to ensure that all tools are available at the specified server locations. note that tools are installed in specific locations which may be varying for different hardware platforms and sites. the pipeline library manager http://pipeline.loni.ucla.edu/support/server-guide/configuration/ facilitates the portability of the xml-based pipeline graphical workflows by defining a hash-map between different software suites, computational tools, versions and executable locations. thus a well-defined pipeline graphical workflow only references the software suite, its version and the specific tool necessary for the specific computational task. the pipeline server then interprets, maps and constructs the specific executable commands which are relayed to the grid manager as concrete jobs and scheduled for execution. module definition the module definition is accomplished via the pipeline gui. each of the executable processes needs to be described individually and independently as a node in the workflow graph http://pipeline.loni.ucla.edu/support/user-guide/creating-modules/. this step also includes the independent testing and validation of the execution of each of the individual nodes using appropriate data. the result of this module definition step is an xml file representing the tool invocation syntax, which can be broadly used, shared, modified, extended and integrated with other module descriptions to form complex graphical workflow protocols. workflow implementation this protocol skeletonization process is important as it lays out the flow of the data and indicates the complete data analysis provenance http://pipeline.loni.ucla.edu/support/user-guide/building-a-workflow/. after all necessary modules are independently defined and validated, we need to integrate them into a coherent and scientifically-valid pipeline workflow. frequently the developer makes use of module groupings to abstract computational complexity, conditional and looping modules to direct the processing flow, and specify data sources and data sinks. in addition, workflow-wide and module-specific meta-data documentation is also provided in this step. these include appropriate workflow preferences, variables, references/citations, urls, licenses, required vs. optional parameters, inputs, outputs and run-time controls, etc. workflow validation the workflow validation step involves the testing and fine-tuning of the implemented workflow to ensure the results of each intermediate step as well as the final results are reasonable, appropriately captured and saved, and tested against alternative protocols. this automated workflow validation step ensures that input datasets and the output results are well-defined, verifies the provenance information about the pipeline workflow, as well as provides user documentation, limitations, assumptions, potential problems and solutions, usability and support annotation. dissemination validated pipeline workflows may be disseminated as xml documents via email, web-sites and pipeline server libraries, as well as biositemaps objects in xml or rdf format http://www.biositemaps.org. any valid pipeline workflow may be loaded by any remote pipeline client. however, execution of a particular workflow may require access to a pipeline server where all tools referenced in the workflow are available for execution, the user has the appropriate credentials to access the remote pipeline servers, data, software tools and services. in addition, some minor workflow modifications may be necessary prior to the execution of the pipeline workflow although, each pipeline client is itself a server, typical users would not run workflows on the same machine but rather remotely login to a pipeline server to outsource the computing-intensive tasks. pipeline clients can disconnect and reconnect frequently to multiple pipeline servers to submit workflows and monitor the status of running workflows in real time. iii.  we now demonstrate the complete process of installing, xml-wrapping, employing and integrating tools from several informatics software suites - miblast, emboss, mrfast, gwass, maq, samtools, and bowtie. each of these packages includes a large number of tools and significant capabilities. the pipeline xml module definitions for some of the tools within these packages may not yet be implemented, or may be incompletely defined within the pipeline library. however, following this step-by-step guideline, the entire community may extend and improve the existing, as well as describe and distribute additional, pipeline xml module wrappers for other tools within these suites. additional genomics and informatics suites and resources may also be similarly described in the pipeline xml syntax and made available to the community, as needed. iii. miblast ◦ url: http://www.eecs.umich.edu/~jignesh/miblast/ ◦ description: miblast is a tool for efficiently blasting a batch of nucleotide sequence queries. such batch workloads contain a large number of query sequences. these batch workloads can be evaluated by blasting each individual query one at time, but this method is very slow for large batch sizes. ◦ installation: the downloading, installation and configuration of the miblast suite on linux os kernel takes only minutes following these instructions: http://www.eecs.umich.edu/~jignesh/miblast/installation.html. ◦ pipeline workflow ▪ xml metadata description: the module descriptions for each of the nodes in the pipeline workflow took about minutes each. the design of the complete workflow took hours because this suite generates many implicit outputs and is intended to be run each time from the core build directory. this presents a challenge for multiple users using the same routines and generating the same implicit filenames. to circumvent this problem, we added several auxiliary modules in the beginning and at the end. notice that wrapper shell-scripts had to also be developed to address the problems with implicit output filenames. finally, flow-of-control connections, in addition to standard data-passing connections, were utilized to direct the execution of the entire protocol. ▪ name: miblast_workflow.pipe ▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfoblast ▪ screenshots: • input: figure shows a snapshot of the input parameters for the corresponding pipeline workflow. • pipeline execution: figure shows the completed miblast pipeline workflow and a fragment of the output alignment result. • output: table contains the beginning of the output result from the miblast workflow. ▪ approximate time to complete: 20- minutes. iii. emboss ◦ url: http://emboss.sourceforge.net/ ◦ tool: emboss matcher ◦ description: finds the best local alignments between two sequences. it can be used to compare two sequences looking for local sequence similarities using a rigorous algorithm. ◦ installation: the downloading, installation and configuration of the entire emboss suite on linux os kernel takes minutes following these instructions: http://emboss.sourceforge.net/docs/faq.html. ◦ pipeline workflow ▪ xml metadata description: the pipeline xml for a number of emboss tools are available. each of these metadata module descriptions was complete via the pipeline module gui and took about 30- minutes. as a demonstration, only the emboss matcher module is presented here. ▪ name: emboss_matcher.pipe ▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfoemboss ▪ screenshots: • input: figure shows the input parameters for the corresponding pipeline workflow. • pipeline execution: figure demonstrates the completed execution of this emboss module. • output: table contains the beginning of the output result from the emboss matcher alignment. ▪ approximate time to complete: minutes. iii. mrfast ◦ url: http://mrfast.sourceforge.net ◦ description: mrfast is designed to map short reads generated with the illumina platform to reference genome assemblies in a fast and memory-efficient manner. ◦ installation: the downloading, installation and configuration of the mrfast suite on linux os kernel takes only minutes following these instructions: http://mrfast.sourceforge.net/manual.html. ◦ pipeline workflow ▪ xml metadata description: the example includes pipeline module descriptions of the mrfast fasta-indexing and fasta-mapping tools. each of these metadata module descriptions was complete via the pipeline module gui and took about 10- minutes. ▪ name: mrfast_indexing_mapping.pipe ▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfomrfast ▪ screenshots: • input: figure shows the input parameters for the corresponding pipeline workflow. • pipeline execution: figure demonstrates the completed execution of this mrfast workflow. • output: table contains the beginning of the output result from the mrfast workflow. ▪ approximate time to complete: minutes. iii. gwass ◦ url: http://www.stats.ox.ac.uk/~marchini/software/gwas/gwas.html ◦ description: the genome-wide association study software is a suite of tools facilitating the analysis of genome-wide association studies. these tools were used in the design and analysis of the genome-wide association studies carried out by the wellcome trust case-control consortium. one specific example of a gwass tool is impute v. which is used for genotype imputation and phasing https://mathgen.stats.ox.ac.uk/impute/impute_v html. ◦ installation: the downloading, installation and configuration of the gwass suite on linux os kernel takes about minutes. ▪ xml metadata description: here we demonstrate the pipeline module description of impute, which is a program for genotype imputation in genome-wide association studies and provides fine-mapping based on a dense set of marker data. ▪ name: gwass_impute.pipe ▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfgwass ▪ screenshots: • input: figure shows the input parameters for the corresponding pipeline workflow. • pipeline execution: figure demonstrates the completed execution of this gwass impute module. • output: table contains the beginning of the output result from the gwass impute module. ▪ approximate time to complete: 2- minutes. iii. interoperabilities between independently developed informatics resources there are 100's of examples of heterogeneous pipeline graphical workflows that illustrate the interoperability between imaging tools independently developed for different purposes by different investigators at remote institutions. these can be found under the workflows section of the pipeline library, figure as well as on the web at: http://www.loni.ucla.edu/twiki/bin/view/loni/pipeline_genomicsinformatics. similarly, bioinformatics and genomics investigators can construct novel computational sequence analysis protocols using the available suites of informatics resources. below is one example of integrating the informatics tools described above. iii. indexing and mapping in this example, we illustrate how to integrate mrfast and emboss water informatics tools. the left branch of the pipeline workflow uses emboss to align sequences using water. the right branch of the workflow does fast alignment using independent datasets and separate mrfast references. the interesting demonstration here is that the emboss water output is later directly used as derived data sequence which is re-aligned to the mrfast reference sequences. ◦ pipeline workflow ▪ design: this tool-interoperability example illustrates feeding the output of the emboss water module as in input in mrfast indexing module and subsequent mapping using mrfast. ▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfomrfast ▪ name: bioinfo_integratedworkflow_emboss_water_mrfast.pipe ▪ screenshots: • inputs: figure shows the input parameters for the corresponding pipeline workflow. • pipeline execution: figure demonstrates the completed execution of this emboss/mrfast workflow. ▪ output: table contains the beginning of the output result from this heterogeneous workflow. ▪ approximate time to complete: minutes. iii. alignment and assembly another interesting example illustrating the power of tool interoperability using the pipeline environment involved the alignment and assembly computational genomics protocol we presented in the implementation section. in this example, we demonstrate the interoperability between maq, samtools, and bowtie tools. ◦ pipeline workflow ▪ design: this genomics pipeline workflow begins with an optional preprocessing step extracting a sub-sequence of the genomic sequence. then, the illumina sequencing data are converted first to fastq and later to binary fastq file format. the data is then aligned to a reference genome, the map file is converted to bam file and the reference genome is indexed. the alignment map file is converted to samtools format first and then to a binary bam file. next, duplicated reads are removed, the bam file is sorted, md tagged and indexed. in addition the maq-based alignment, bowtie was used to align the data to the reference sequence. such alternative paths in the data processing protocol are easily constructed and modified in the pipeline environment. ▪ url: http://www.loni.ucla.edu/twiki/bin/view/ccb/pipelineworkflows_bioinfomaq ▪ name: maq_samtools_bowtie_integrated_cranium.pipe ▪ screenshots: • inputs: figure shows the input parameters for the corresponding pipeline workflow. • pipeline execution: figure demonstrates the completed execution of this heterogeneous pipeline workflow. ▪ output: table contains the beginning of the output result from this heterogeneous workflow. ▪ approximate time to complete: 4- minutes. iv.  this paper reports on the applications of the pipeline environment to address two types of computational genomics and bioinformatics challenges - graphical management of diverse suites of tools, and the interoperability of heterogeneous software. specifically, this manuscript presents the concrete details of deploying general informatics suites and individual software tools to new hardware infrastructures, the design, validation and execution of new visual analysis protocols via the pipeline graphical interface, and integration of diverse informatics tools via the pipeline xml syntax. we demonstrate each of these three protocols using several established informatics packages - miblast, emboss, mrfast, gwass, maq, samtools, and bowtie. these examples demonstrate informatics and genomics applications using the pipeline graphical workflow environment. the pipeline is a platform-independent middleware infrastructure enabling the description of diverse informatics resources with well-defined invocation protocol - syntax for dynamic specification of the inputs, outputs and run-time control parameters. these resource xml descriptions may be provided using the pipeline graphical user interface or using machine code according to the pipeline xsd schema. the pipeline environment provides a powerful and flexible infrastructure for efficient biomedical computing and distributed informatics research. the interactive pipeline resource management enables the utilization and interoperability of diverse types of informatics resources. the pipeline client-server model provides computational power to a broad spectrum of informatics investigators - experienced developers and novice users, the computational-resources haves and have-nots, as well as between basic and translational scientists. the open development, validation and dissemination of computational networks facilitates the sharing of knowledge, tools, protocols and best practices, and enables the unbiased validation and replication of scientific findings by the entire community. for specific research domains, applications and needs, there are pros and cons for using the pipeline environment or any of the alternative tools and infrastructures for visual informatics and computational genomics. examples of powerful alternatives include taverna, kepler, triana, galaxy, avs, vistrails, bioclipse, knime, and others. the main advantages of the pipeline environment are the distributed client-server architecture with diverse arrays of grid plug-ins, the lightweight data, tools and services utilization and the dynamic workflow design, validation, execution, monitoring and dissemination of complete end-to-end computaitonal solutions. v. availability and requirements • project name: pipeline environment • project home pages: ◦ loni: http://pipeline.loni.ucla.edu ◦ nitrc: http://www.nitrc.org/projects/pipeline ◦ birn: http://www.birncommunity.org/tools-catalog/loni-pipeline ◦ bioinformatics.org: http://www.bioinformatics.org/pipeline ◦ try the loni pipeline informatics and genomics workflows online without any software installation using anonymous guest account: http://pipeline.loni.ucla.edu/pws • operating system: pipeline clients and servers are platform-independent, while some features require the server run on linux/unix os. the distributed pipeline server graphical user interface, which installs the pipeline server, grid engine, and computational imaging and informatics software tools, require standard linux os kernels. the pipeline web start allows users to start the pipeline application directly from the web browser and run it locally without any installation. it has all the features and functionality of the downloadable stand-alone pipeline application and allows anonymous guest access or user-authentication to connect to remote pipeline servers. • programming language: pure java. • other requirements: ◦ requirements summary: the pipeline client and server can run on any system that is supported by java runtime environment or higher. windows pipeline servers will not be able to use privilege escalation. three-tier failover feature is only supported by unix/linux systems. all other features are available for all platforms. most distributed pipeline servers require 300- mb memory, which may depend on the load and garbage collection preferences. ◦ for distributed multicore deployment, the distributed pipeline server requires a grid manager, which is provided with the dps distribution. the pipeline server will still work on a platform without a grid manager, however, jobs may not be processed in parallel and performance on multicore machines may be suboptimal. ◦ complete requirements: ▪ client: http://pipeline.loni.ucla.edu/support/user-guide/installation/ ▪ server: http://pipeline.loni.ucla.edu/support/server-guide/installation/ ▪ dps: http://pipeline.loni.ucla.edu/dps ▪ pws: http://pipeline.loni.ucla.edu/pws • license: apache-derived software license http://www.loni.ucla.edu/policies/loni_softwareagreement.shtml. • caution: there are some potential limitations of the pipeline environment and its current collection of data, tools services and computational library: ◦ each new informatics tool which needs to be accessible as a processing module within the pipeline environment needs to be described manually by an expert using the pipeline gui or automatically using a properly configured xml exporter. then the pipeline xml module description can be shared with other users. ◦ to run available pipeline workflows on remote pipeline-servers, users need to have accounts on the remote pipeline servers. in addition, types of updates may be necessary in the pipe files - the server-name references of data sources, data sinks, and executables, as well as the path references to the data sources, sinks and executables. the server-name can be easily updated using server changer tool in pipeline. user has to edit path references on some or all of the data sources, sinks and executables for their server. no workflow modifications are necessary for executing these pipeline workflows on the loni pipeline cranium server; however this requires a loni pipeline user-account http://www.loni.ucla.edu/collaboration/pipeline/pipeline_application.jsp. a proper administrator configuration of the distributed pipeline server will resolve the need for such revisions by the user. ◦ some computational tools may require wrapper scripts that call the raw executable binaries. these scripts are then invoked via the pipeline environment. example situations include tools that have implicit outputs, or if the tools routinely return non-trivial exit codes, distinct from zero. such problems may cause the pipeline environment to halt execution of subsequent modules, because of a broken module-to-module communication protocol. ◦ smartlines, which auto-convert between different informatics data formats, need to be extended to handle informatics and genomics data. ◦ access to external informatics databases may need to be customized - e.g., pdb http://www.rcsb.org, scop http://scop.mrc-lmb.cam.ac.uk/scop, genbank http://www.ncbi.nlm.nih.gov/genbank, etc. ◦ native vs. virtual pipeline server: the fully distributed pipeline server architecture provides both the pipeline middleware as well as installers for all computational tools available on the loni cranium pipeline grid server http://pipeline.loni.ucla.edu/support/user-guide/interface-overview/. the virtual pipeline server and pipeline clients also provide the complete pipeline environment for a virtual vmware invocation http://pipeline.loni.ucla.edu/pnve. • any restrictions to use by non-academics: free for non-commercial research purposes. competing interests the authors declare that they have no competing interests. authors' contributions all authors have read and approved the final manuscript. all authors made substantive intellectual contributions to this research and development effort and the drafting of the manuscript. specifically, the authors contributed as follows: idd article conception, study design, workflow implementation and applications, data processing, drafting and revising the manuscript; ft study design, workflow implementation, data processing, drafting the manuscript; fm study design, workflow implementation, drafting the manuscript; pp pipeline software implementation, workflow design and drafting the manuscript; zl pipeline software implementation, workflow design and drafting the manuscript; az pipeline software implementation, study design, workflow development and drafting the manuscript; pe pipeline software engineering, workflow design and drafting the manuscript; jp pipeline server grid implementation and management, workflow development and drafting the manuscript; ag study design, workflow implementation and drafting the manuscript, jak and apc study design protocol, data processing, workflow development and drafting the manuscript; jdvh study design protocol and drafting the manuscript; ja workflow applications and drafting the manuscript; ck study design, workflow applications and drafting the manuscript; awt study design, engineering of the pipeline environment, and drafting the manuscript. acknowledgements and funding this work was funded in part by the national institutes of health through grants u rr p rr r mh u24-rr u24-rr u24-rr and u24-rr we are also indebted to the members of the laboratory of neuro imaging, the biomedical informatics research network, the national centers for biomedical computing and clinical and translational science award investigators, nih program officials, and many general users for their patience with beta-testing the pipeline and for providing useful feedback about its state, functionality and usability. benjamin berman and zack ramjan from the usc epigenome center, university of southern california, provided help with definition of sequence pre-processing. for more information on loni and the loni pipeline, please go to http://www.loni.ucla.edu and http://pipeline.loni.ucla.edu.
the interactive pipeline resource manager enables the utilization and interoperability of diverse types of informatics resources. contemporary informatics and genomics research require efficient, flexible and robust management of large heterogeneous data, advanced computational tools, powerful visualization, reliable hardware infrastructure, interoperability of computational resources, and detailed data and analysis-protocol provenance. specifically, this manuscript presents the concrete details of deploying general informatics suites and individual software tools to new hardware infrastructures, the design, validation and execution of new visual analysis protocols via the pipeline graphical interface, and integration of diverse informatics tools via the pipeline extensible markup language syntax. the loni pipeline environment http://pipeline.loni.ucla.edu provides a flexible graphical infrastructure for efficient biomedical computing and distributed informatics research. the pipeline client-server model provides computational power to a broad spectrum of informatics investigators - experienced developers and novice users, user with or without access to advanced computational-resources, as well as basic and translational scientists. this paper reports on the applications of the loni pipeline environment to address two informatics challenges - graphical management of diverse genomics tools, and the interoperability of informatics software. we demonstrate each of these processes using several established informatics packages for basic local sequence alignment and search, molecular biology data analysis, and genome-wide association studies. these examples demonstrate the power of the pipeline graphical workflow environment to enable integration of bioinformatics resources which provide a well-defined syntax for dynamic specification of the input/output parameters and the run-time execution controls. the open development, validation and dissemination of computational networks facilitates the sharing of knowledge, tools, protocols and best practices, and enables the unbiased validation and replication of scientific findings by the entire community.
 contemporary informatics and genomics research require efficient, flexible and robust management of large heterogeneous data, advanced computational tools, powerful visualization, reliable hardware infrastructure, interoperability of computational resources, and detailed data and analysis-protocol provenance. the pipeline is a client-server distributed computational environment that facilitates the visual graphical construction, execution, monitoring, validation and dissemination of advanced data analysis protocols.  this paper reports on the applications of the loni pipeline environment to address two informatics challenges - graphical management of diverse genomics tools, and the interoperability of informatics software. specifically, this manuscript presents the concrete details of deploying general informatics suites and individual software tools to new hardware infrastructures, the design, validation and execution of new visual analysis protocols via the pipeline graphical interface, and integration of diverse informatics tools via the pipeline extensible markup language syntax. we demonstrate each of these processes using several established informatics packages for basic local sequence alignment and search, molecular biology data analysis, and genome-wide association studies. these examples demonstrate the power of the pipeline graphical workflow environment to enable integration of bioinformatics resources which provide a well-defined syntax for dynamic specification of the input/output parameters and the run-time execution controls.  the loni pipeline environment http://pipeline.loni.ucla.edu provides a flexible graphical infrastructure for efficient biomedical computing and distributed informatics research. the interactive pipeline resource manager enables the utilization and interoperability of diverse types of informatics resources. the pipeline client-server model provides computational power to a broad spectrum of informatics investigators - experienced developers and novice users, user with or without access to advanced computational-resources, as well as basic and translational scientists. the open development, validation and dissemination of computational networks facilitates the sharing of knowledge, tools, protocols and best practices, and enables the unbiased validation and replication of scientific findings by the entire community. 