Home → CLC FAQ - CLC Genomics Server and Command Line Tools → Printer Friendly Version
CLC FAQ - CLC Genomics Server and Command Line Tools
- 1. Connecting to a CLC Genomics Server from a Workbench
- 2. CLC Genomics Server
- 2.1. How can I change the memory allocated to a CLC Genomics Server's java process?
- 2.2. Can I limit the number of CPUs used by the CLC Genomics Server?
- 2.3. How can I empty the recycling bin in a CLC Genomics Server File Location?
- 2.4. Why can't I see the groups I expect when setting up permissions?
- 2.5. Why aren't permissions for my Active Directory groups being applied as I intend?
- 2.6. How can I run the CLC Genomics Server as a non-privileged user when it has been run by the root user?
- 2.7. Can I downgrade my CLC Server software?
- 2.8. Why can't I specify an Import/Export directory to be a subfolder of a CLC File Location?
- 2.9. How can I add BLAST database locations to my Server?
- 2.10. How can I set permissions for BLAST database addition and searching on the CLC Genomics Server?
- 3. Working with CLC Job Nodes
- 4. Working with a Grid setup
- 5. CLC Command Line Tools
1. Connecting to a CLC Genomics Server from a Workbench
1.1. How do I log into a CLC Genomics Server from my Workbench?
To log into your CLC Genomics Server, such as the CLC Genomics Server, from your Workbench, you will need the following information:
- The CLC Genomics Server IP address or Hostname
- The CLC Genomics Server listening port (this is 7777 by default, but can be changed by your administrator)
- Your user name on the CLC Genomics Server
- Your password on the CLC Genomics Server
If you do not know what this information is for your CLC Genomics Server, please contact the administrator of your CLC Genomics Server software for help.
Once you have this information please take the following steps:
- Install the CLC Workbench Client plugin in your Workbench
- Restart the Workbench
- Log into the Server by going to File | CLC Server Login
If you have the 'Automatic login' option checked, you will automatically connect and login to the CLC Genomics Server every time you start the Workbench.
1.2. What does the message "Please ensure that the Workbench and Server are compatible..." mean?
This error message occurs when you attempt to log into a CLC Genomics Server that is not compatible with the version of the Workbench you are using.
When this issue occurs, to log in to your CLC Genomics Server you may:
- Use the older version of the Workbench to connect to the CLC Genomics Server. Information about how to find installer files for older versions of the Workbench can be found in the FAQ page: Where can I get installer files for the software?
- Ask your CLC Genomics Server administrator to upgrade to the latest version of the CLC Genomics Server. When new versions are released we send email notification to the owners of CLC Genomics Server licenses along with technical contacts for that license. Please feel free to let the Genomics Server license owner know that they can add others as technical contacts so those responsible receive such email notifications in future. Information about this is in the FAQ page: How can I add someone as a technical or billing contact?
Why do I see this error message?
It is necessary for the Workbench and CLC Server version to be compatible to log into the CLC Genomics Server. For example:
- to connect to the CLC Genomics Server version 11.0.x, the CLC Genomics Workbench must be version 12.0.x.
- to connect to the CLC Genomics Server version 10.0.x, the CLC Genomics Workbench must be version 11.0.x.
- to connect to the CLC Genomics Server version 9.0.x, the CLC Genomics Workbench must be version 10.0.x.
- to connect to the CLC Genomics Server version 8.5.x, the CLC Genomics Workbench must be version 9.5.x.
This compatibility information can be found on the Latest Improvements page. At the bottom of the improvements list for the released version of the Workbench the CLC Server version it is compatible with is indicated:
2. CLC Genomics Server
2.1. How can I change the memory allocated to a CLC Genomics Server's java process?
The default amount of memory set in the case of the is the lesser of 50% of the available physical memory or 50GB. We generally recommend that the setting be left at the default level.
An explanation about why the defaults are recommended can be found in the FAQ entry:
Information on how to change this setting for the CLC Genomics Server, please refer to the Server manual entry:
Please note that the configuration file that needs to be edited for this process is not automatically updated if more memory is installed in the machine. Changes to the memory setting for the CLC Genomics Server must be done manually.
The Genomics Server service must be restarted for changes to the configuration file, 'clcgenomicsserver.vmoptions', to take effect.
When 'clcgenomicsserver.vmoptions' is edited, please make sure the file is saved with '.vmoptions' extention, not as a '.txt' or other.
2.2. Can I limit the number of CPUs used by the CLC Genomics Server?
Restricting the number of cores that the CLC Server can use is covered in our CLC Genomics Server manual here:
2.3. How can I empty the recycling bin in a CLC Genomics Server File Location?
There are two main types of recycle bin visible to users of the Workbench when logged into a CLC Genomics Server:
recycle bins in the Workbench data locations (one bin per location)
recycle bins in the Genomics Server file locations (one bin per location)
Each top level folder that one can see via the Workbench is a "location" of some sort - the ones with blue dots on them are Server file locations. The others are Workbench data locations. Every location has a single recycle bin.
Individuals can delete data within the recycle bins of their own Workbench data locations.
Only members of the administrator group, as defined on the CLC Genomics Server, can empty the recycle bin associated with CLC Genomics Server File Locations. This is because the recycle bin is a shared location for any given CLC Genomics Server file location and many sites do not want all users to be able to access it directly, that is to be able to view things or delete other people's data.
One can thus empty recycle bins in CLC Genomics Server locations by logging into the CLC Genomics Server via a CLC Workbench with a user that is a member of the admin group of the CLC Genomics Server, right-clicking on the recycle bin for a given file system location and selecting "Empty recycle bin" to empty it.
To avoid logging in as an adminstrator to periodically empty all trash, one could set up a script making use of the CLC Command Line Tools to empty Genomics Server recycling bins and run this as a cronjob. This allows the recycling bins to be emptied in a hands-off manner at a defined frequency.
A CLC Command Line Tools command for emptying a recycle bin would be of a form like:
$SERVERCMDPATH -S $HOST -P $PORT -U $USER -W $PASS -A empty_recycle_bin -t clc://server/$LOCATIONNAME
Here, SERVERCMD path refers to the path to the clcserver command, and LOCATIONNAME refers to the name of the Genomics Server file location that the command will empty the recycling bin of. The other parts of the command are explained in the user manual for the CLC Command Line Tools, which can be found here:
The CLC Command Line Tools manual can be found here:
The tools themselves can be downloaded from our website:
2.4. Why can't I see the groups I expect when setting up permissions?
After enabling permissions on a given File Location for a CLC Genomics Server, you should then be able to select the groups you wish to grant access to.
If you do not see the groups you expect presented to you in the list, then this could be because you have not logged into the CLC Genomics Server as a user that is part of the Admin group.
That is, the account used to log into the CLC Genomics Server to set permissions must be an Active Directory - AD or LDAP account and it must be a member of the group entered in the "Admin group name" field of the AD or LDAP authentication configuration on the CLC Genomics Server.
If the user you log in as does not meet that criteria, please log out and log in as a user that does. Please note that the user called "root" that is provided as part of the CLC Genomics Server default installation is not an AD or LDAP account.
If the problem you are seeing persists, please submit your CLC Genomics Server logs. Information on how to do this is linked via the Related Pages at the bottom of this page.)
In the box labelled "Message to CLC bio" box, please supply:
- a description of what you are seeing
- the Distinguished Name of a user account that is a member of the intended "Admin group"
- the Distinguished Name of one or more of the groups that is within the intended "Groups DN" OU
2.5. Why aren't permissions for my Active Directory groups being applied as I intend?
Access to file system locations and their subfolders on a CLC Genomics Server can be controlled via AD groups, including nested groups.
The effective group membership for a user must, however, be entirely through Security groups.
Distribution groups can not be used to assign permissions to shared folders in a Genomics Server data location, in line with the Microsoft description of security vs. distribution groups.
2.6. How can I run the CLC Genomics Server as a non-privileged user when it has been run by the root user?
We recommend that a CLC Genomics Server service is run by a non-privileged system user account for security reasons.
For Linux and Mac OS X systems:
The information here covers the steps to take if you have previously run the CLC Genomics Server using a privileged user account such as root and now wish to switch to running the service using a non-privileged account.
Most of the work here involves changing ownership on files associated with the CLC Genomics Server and altering the service script to ensure the service is started up by the appropriate user. To carry out the steps here, you need to:
- use an account that has administrative privileges for the CLC Genomics Server, AND
- have appropriate privileges on the system where data is stored to change the ownership of the folders and files relevant to the CLC Genomics Server.
For Genomics Servers with job nodes or grid nodes, please ensure that the user you plan to use to run the CLC Server process has the same uid and gid on all nodes.
Steps to take
- Notify users that there will be a temporary service interruption and ask them to stop submitting new jobs to the CLC Genomics Server.
- Take notes of the current File System locations and the Import/Export locations on the Genomics Server interface Admin tab | Main configurations. More information about these two settings can be found in the following manual area:
- (Optional) Backup data associated with the CLC Server File System Locations and Import/Export locations. Backing up data is recommended if backups are not otherwise already in place at your site. The only changes needed to perform here are filesystem ownership modifications, which are not expected to affect the data itself but it is always a good policy to have at least one extra copy of the data to prevent possible data loss.
- Make sure that there are no CLC Server jobs currently running or scheduled to run (queued) on the CLC Server and on any job nodes or grid nodes.
- Stop the CLC Server service by using the startup/stop script as described on the following Server manual page link:
You will need to do that as the user that is currently running the CLC Genomics Server service, or as the administrative user account through the startup/stop script.
Starting CLC Genomics Server v11.0, the service can be shut down or restarted through the web interface:
- Change the ownership recursively on the folders listed in the following section, using a system user account that has the privileges to do this.
The command below is an example of how to change the owner and the group (owner:group) on a Linux system where the installation location of the software was /opt/CLCGenomicsServer. Here, the ownership is being changed so that the folder /opt/CLCGenomicsServer and all folders and files under it will be owned by a normal user account called clcbio with a group called clcbio.
chown -R clcbio:clcbio /opt/CLCGenomicsServer
The folders to apply recursive ownership changes to are:
- The folder where the CLC Genomics Server software is installed.
- All folders configured as File System Locations in the CLC Genomics Server.
- The folder where the CLC Genomics Server software is installed.
- Modify your start/stop script used to start and stop the CLC Genomics Server service so that it refers to the user account that should be running the CLC Genomics Server.
Further information about starting and stopping the service can be found in the manual at:
On some Linux systems, the service script can be found under /etc/init.d/. For example, in the case of the CLC Genomics Server, under
To run the service as a non-administrative user, please ensure that the line starting with CUSTOM_USER is set to true, i.e.
and that the line beginning with USER is set to the user that you wish to use. For example, if that user was called clcbiouser, that line would look like:
- Check that there are no CLC folders and files under the temporary directory called things like
If there are any such files, delete them.
- Start the CLC Server service with the user account you wish to run it as.
- Check the server settings by following the instructions in the manual:
Examine the output report for any errors.
You may also want to consider running a few CLC jobs submitted from the Workbench with the input and output data on the CLC Genomics Server to verify it is working as you expect.
Notes for CLC Servers with grid nodes
For CLC Servers with grid nodes, please make sure the ownership of the following areas is also changed:
- Grid Shared Work Directory
The grid shared work directory is used and should be accessed by both of the CLC Genomics Server and the grid scheduler software and is specified in the grid preset.
- Path to CLC Grid Worker
The path to the CLC Grid Worker points to the directory that holds the executable binaries of the CLC Genomics Server that will be used to run the CLC jobs on the grid nodes.
The path for both of the above locations can be found in the grid preset configurations. More details about the grid preset settings are available on the following Genomics Server manual link:
Some grid scheduling systems may be configured to create temporary files of the CLC jobs as the system user account that runs the grid scheduler software, rather than the user account that runs the CLC Server. Thus, if you are changing the ownership of the CLC Server, please refer to the manual for your grid system and make sure the new CLC Server owner account is used by the grid scheduler for running CLC jobs from start to finish. You may also want to consider to use the same account for running both the grid scheduler / resource manager and the CLC Server service in order to avoid the possible permission issues, as long as the non-privileged system user account is used for running the grid scheduler software.
For Microsoft Windows systems (Windows 7/VISTA/8/10, Windows 2003/2008/2008 R2/2012/2016 server):
After the installation, the CLC Genomics Server service is run by the Local System account by default. The account has full access to the local system but may not have sufficient permission to access network storage share or mapped drives.
We recommend to change the account used to run the CLC Genomics Server service from Local System account to a normal user account. And then make sure this user account has full access to the shared CLC data and import/export locations.
To change the account used to run the service, you can use the GUI windows service snap-in by typing the following in an elevated command prompt (right click on the "command prompt" icon of the Start menu | All Programs | Accessories, then choose Run as Administrator option from the drop-down menu):
- Right click on the service on the service list. The name of the service will reflect the Server product you are using. For example, when running the CLC Genomics Server, it would be CLCGenomicsServer.
- Choose Properties.
- Go to "Log on" tab.
- Choose the radio button for "This account:".
- Click on Browse button to type in the username you would like to use to run the CLC Genomics Server service.
- Once the user account is chosen, please enter the user's password twice in the password fields, then click OK.
You will need to restart the service for the new account to be in effect. To do this, simply right click on the CLC Genomics Server service and choose restart in the service snap-in window.
For a CLC Genomics Server cluster running on windows machines, please make sure the same account is used for running all CLC Genomics Server service on all machines.
Please make sure that this user account has permissions to read and write the data folders used for the CLC Genomics Server (both the master and the job nodes) at both the filesystem level (the Security tab in folder's properties window) and at the sharing level (the Sharing tab in folder's properties window).
Once the CLC Genomics Server service is restarted on all the nodes, you can then use server check setup script method mentioned in the following Server manual link
to make sure that all nodes have sufficient access to the data locations and no error are generated.
2.7. Can I downgrade my CLC Server software?
It is not something that is generally done, but you can downgrade your CLC Server software to an earlier version than the one you are currently running.
While we fully support upgrading of software in place, we do not commit to being able to downgrade in place. While it will often be "just work", but we do not guarantee it. We also do not guarantee that data and workflows generated in a newer version of the Genomics Server or the corresponding Genomics Workbench will be usable in older versions of the software.
To downgrade a CLC Server, we recommend installing to a new location and re-configuring the software in that new location. If you wish to avoid making changes to your CLC Server service script, then you could move the old installation and then move the new installation to the original location. Alternatively, edit your service script to indicate which copy of the CLC Server should be started up when the service is started.
Known downgrading issues
Downgrading from the CLC Genomics Server version 10.0.1 to 9.1.x
On March 14, 20018, version 10.0.1 of the CLC Genomics Server is released. If you upgrade to this version, but later choose to downgrade to version 9.1.x, you can either:
- Install to a new location and reconfigure the Genomics Server, as described above
Before installing the old version of the Genomics Server, delete the file:
from within the installation area of the Server software. Then install the later version of the Server software in place.
Please be informed that this applies only for the supported Linux distributions (system requirements). For any other distribution we do not expect issues with this approach but we can not guarantee it.
Since for Windows, there can be only one "CLCGenomicsServer" service running at a time, the only way to downgrade is to uninstall the higher version first, then install the previous version of the CLC Genomics Server.
2.8. Why can't I specify an Import/Export directory to be a subfolder of a CLC File Location?
The action of setting up an Import/Export location is a means for designating where the CLC Server is allowed read and write access. This is described in the manual page about setting up Import/Export locations:
Import/Export directories should not be located within areas designated as CLC File Locations on the Server.
CLC File Locations for CLC Servers, and CLC Data Locations for CLC Workbenches, should be used for CLC data only. Data should only be added or removed from these locations by CLC software. This is because CLC File Locations and CLC Data Locations contain indexing and other information about the contents of those folders, which is kept up to date by the software.
By CLC data, we mean data that is imported into the software or has been generated by the software itself.
By definition, an Import/Export folder is intended for holding non-CLC data (with extension different from .clc ). Commonly, this would include:
- sequencing data that will be imported, such as fastq files
- data exported from the CLC Server
- BLAST databases
If your aim was to have all data that had any relevance to your CLC software in one place, then you could create two folders at the same level to do this. For example, you could create a folder called CLCDataRelated. You could then have subfolders:
Of course, an Import/Export Location does not need to be in any particular place. It may also be that you already have a place where you store sequence data and plan to import from it. You may also have an existing location where you export your data to. As long the owner or administrator of the file system area approves it, then any folder could be designated as an Import/Export area for the Genomics Server, except for an area already designated as a CLC File Location or a subfolder of such an area.
2.9. How can I add BLAST database locations to my Server?
There are two steps involved in adding BLAST database locations to a CLC Server.
1) Add the location as an Import/Export location.
2) Add it as a BLAST database location.
These actions are done using the web administrative interface of the Genomics Server and are described in more detail in the manual here:
This information is relevant for other CLC Server products that offer BLAST searching.
2.10. How can I set permissions for BLAST database addition and searching on the CLC Genomics Server?
It is possible to configure the CLC Genomics Server so that some members of some groups have permission to create and download BLAST databases in a BLAST database location, while others can run searches, but not add more BLAST databases onto the server. You can also restrict use of the BLAST search tool on the CLC Genomics Server to specified groups.
To carry out the work described here, you must be logged in as a user that has administrative rights on the CLC Genomics Server. Permissions are set a group level, so before proceeding to the steps below, please ensure that users are in appropriate groups. e.g. those who should be able to add BLAST databases to the server BLAST database location are in a particular user group or groups, separate from the groups containing users who should not have such access.
In these instructions, we assume there are three user groups:
- admin - members of this group should have full permissions
- noblast - members of this group should not be able to run BLAST searchers or add new BLAST databases to the server's BLAST database location.
- normal - members of this group should be able to run BLAST searches but not add new BLAST databases to the server's BLAST database location.
1. Create the dedicated BLAST database location on the Genomics Server: How can I add BLAST database locations to my Server?
2. To allow only users in the 'admin' group to add BLAST databases to the server's BLAST database location, grant this group, and no others, permission to run the following algorithms through Admin | Global permissions | Algorithms:
- Create BLAST Database
- Download BLAST Database
3. To allow users of the 'admin' and 'normal' groups to submit BLAST searches onto the CLC Genomics Server, grant permission to both groups for the following algorithm through Admin | Global permissions | Algorithms:
4. (Optional) To block certain group ('noblast' in this case) from accessing a particular BLAST database location completely, click on the "Edit Permissions..." button beside the relevant Import/export directory under Admin | Global permissions | Import/export directories, as shown in the following screenshot.
This may be a useful thing to do in cases where you have multiple BLAST database locations and users in some groups should have access to particular BLAST database locations but not others.
The only other arguable benefit would be to avoid the possibility that users in a particular group could not export other types of data into the folder designated as a BLAST database location. However, this eventuality exists for any users with access to particular Import/export locations. If someone did export data to a BLAST database location, it would not affect the list of BLAST databases available for users who are able to run BLAST searches. There is no export functionality in the Workbench or Server for exporting BLAST databases, and only BLAST databases are listed via the BLAST search and management tools.
3. Working with CLC Job Nodes
3.1. Why is the Attach Node button disabled?
If you are setting up a CLC Server with CLC Job Nodes, you need to set the "Server mode" at the top of the Job Distribution section to 'Master node', rather than 'Single server' or 'Execution node'. Please note that you only need to attach job nodes through the Genomics Server web admin interface on the master node.
The Genomics Server on the master node also requires a valid server license in order for the "Attach Node" button to work. Without a valid server license, any Genomics Server software will be automatically put into the execution / job node mode and it will not be possible to add other job nodes.
If the Attach Nodes button remains greyed out, as shown in the image below, please click on the button labeled Save Configuration in the Server mode section.
After actively saving, the Attach Node button should be available again.
4. Working with a Grid setup
4.1. Should the Server mode be set to "Single" or "Master" in a grid setup?
The Server mode of the Genomics Server is set in the Job Distribution subsection of the Admin tab.
The setting controls whether or not the Genomics Server itself can perform analyses as well as acting as the controlling instance. That is, this option indicates whether users will be able to submit jobs to run directly on the Genomics Server as well as being able to submit analysis jobs to the grid nodes made available via configured grid presets.
If the Server mode is set to 'Master node', the Genomics Server itself will not accept any jobs directly, it will only submit them to the grid system.
If the Server mode is set to 'Single server', then the Genomics Server can also accept jobs directly.
Please note that jobs submitted to the Genomics Server itself, rather than to the grid, will only be processed on the Genomics Server. The grid system and the Genomics Server are not considered a single available pool of processing nodes.
In most situations it will be preferable to have the Server mode set to "Master".
Additional information about the job distribution can be found in the following manual page:
4.2. Can I limit the number of CPUs used by CLC jobs sent to a grid node?
Setting core restrictions for grid nodes associated with a Genomics Server setup is covered in the Genomics Server manual here:
5. CLC Command Line Tools
5.1. Do I need to install the Command Line Tools on the same machine as my Genomics Server software?
The Command Line Tools is a client interface for the Genomics Server, in the same way that the Genomics Workbench is. If you plan to use either of these pieces of client software to connect to a Genomics Server, the requirement is that it be installed on a machine that can connect to the machine your Genomics Server software is running on.
This can be the same machine your Genomics Server is installed on. More commonly it would be a different machine.
The initial parameters you provide (-S, -P, -U, -W) allow you to specify the machine and port the Genomics Server software is running on, and your user credentials to log into the Genomics Server. (Your user credentials are the same as those used to log in to the Genomics Server via a Workbench or the Genomics Server administrative web interface.)
Further details about the use of the Command Line Tools can be found in the user manual:
You can download the Command Line Tools from:
5.2. Are there any general tips about composing a command using the CLT?
Information about how to compose and run CLC Command Line Tools (CLT) commands is covered in the manual, which you can find in HTML or PDF format:
In terms of building up your own commands when starting out, one approach would be to
- refer to the Genomics Workbench manual for the particular analysis you plan to run - as the information there, and reference to the wizard (job setup) interface there can provide a good idea of what the options presented via the Command Line Tools mean, and then,
- look at the usage information that is generated with a basic clcserver command.
How to access the usage information for the Command Line Tools is covered in the manual starting here:
In brief, if you submit a command of the following form:
clcserver -U <username> -W <password or passcode> -S <serverhost>
and your credentials were fine, the top level commands that can be run on your CLC Server should be printed to screen. That is, the commands options that could be provided as arguments to the -A option, as well other major options (-C, -D, -G and so on).
As an example, say you were interested in running a de novo assembly. The information returned above would let you know that the relevant option to use after the -A flag would be denovo_assembly
At that point, the command to find out more about how to set up the details of the de novo assembly could be determined by running a command of this form:
clcserver -U <username> -W <password or passcode> -S <serverhost> -A denovo_assembly
This would return the command line options relevant to de novo assembly.
The location of your input files on your Genomics Server, and the location you wish to write data to can be specified in a number of ways, and for the most part rely on clc urls. This is described in the manual here:
In terms of data input for tools like de novo assembly, where you may have multiple input data objects containing sequence reads, you would enter your each sequence list (sets of reads) after its own
The above is just a short summary of tips for getting started. We highly recommend referring to the manual if you plan to use the Command Line Tools for your own analyses.