Build a Cloud VM
Build a Cloud VM¶
In the following, we explain how to install Quantum Mobile on your own virtual machine in the cloud.
This procedure is automated entirely using an ansible playbook in the quantum mobile repository. It has been used successfully to deploy Quantum Mobile on servers from Amazon Web Services, Google Compute Cloud, Huawei Cloud, Openstack as well as on bare metal servers.
This process essentially a sub-set of the process to build the Desktop Edition (see the instructions here), in that we are provisioning an existing VM, rather than creating one from scratch.
A server running Ubuntu 18.04 LTS Can be hardware or virtual machine (tested on OpenStack, Amazon Web Services and Huawei Cloud).
At least 12GB disk size (including Ubuntu); better 15GB or more. Note: After cleaning temporary files, QM occupies ~6GB of disk space.
Access to server via SSH key as user with
Port 22 open (for SSH access)
You may want to open further ports (optional):
port 8888 to connect to the AiiDA Lab Server (or value of
port 8890 to connect to thr Jupyter Lab Server (or value of
port 5000 to connect to the AiiDA REST API
Provisioning the server¶
To get set up, run the following on your client (e.g. your laptop – not on the server itself):
git clone https://github.com/marvel-nccr/quantum-mobile.git cd quantum-mobile
inventory.yml already contains templates for connecting to servers from different cloud providers, which are easy to populate with the connection details for your server.
For example, to connect to an
aws host, adapt:
aws: cloud_platform: aws ansible_host: 18.104.22.168 # change this ansible_ssh_common_args: -i ./keys/quantum-mobile.pem -o StrictHostKeyChecking=no ansible_user: ubuntu
the IP of the server
the path to your private SSH key for connecting to the server
the user to connect as via SSH
The ansible inventory and playbook variables documentation.
You may also want to “tune” the variables under
inventory.yml, such as:
vm_user: the user for which to install the simulation environment (usually not the admin user you are connecting as)
If you want to pre-load an SSH public key for the
vm_userto connect with (as opposed to the admin key set above), uncomment and set the
Finally, to run the ansible playbook against the required host, use:
pip install tox tox -e ansible -- --extra-vars "build_hosts=aws"
This will take of the order of one hour, after which your server should be fully deployed and operational.
The ansible steps are idempotent, meaning they will essentially do nothing when run again on the same machine.
This means, if the build was interrupted for any reason, you can simply re-run the above command.
You can log in to the server as the
vm_user via the public SSH key you provided.
Saving the image¶
Before creating an image from the disk volume of the server you provisioned:
Remove unnecessary temporary build files:
tox -e ansible -- --extra-vars "build_hosts=aws" --tags cleanup
Clear bash history: SSH to the VM and run
cat /dev/null > ~/.bash_history && history -c && exit
Shut down instance (this will clear temporary data in
Now follow the instructions of your platform for creating an image from your server.
Platform specific guidance¶
Create new keypair from the GUI - note: this will always be added to the
If you intend to publish the image, follow AWS instructions to remove system SSH keys:
sudo shred -u /etc/ssh/*_key /etc/ssh/*_key.pub
Listing an image publicly: After creation of the image, edit its properties and select “public”
GCP lets you create the SSH key-pair by yourself:
Create new SSH keypair on your machine.
user@serverbit at the end of the public key to user “ubuntu” (GCP detects this user and adds the SSH public key for this user in the VM)
add public key to VM
VM IP seems to be stable even between stop/start of VM
Creating the image
GCP distinguishes between “Machine images” (private) and “Images” (can be public). You want the second option.
List image publicly: (not sure whether there is a way to do this via the GUI)
Click on “cloud shell” logo on the top right
gcloud compute images add-iam-policy-binding quantum-mobile-20-05-0 \ --member='allAuthenticatedUsers' \ --role='roles/compute.imageUser'
Slow network connection¶
When provisioning machines in remote networks (e.g. Asia), the time needed for establishing an SSH connection can be prohibitively slow. In such cases, the “mitogen” connection plugin can lead to dramatic speedups (it uses python remote procedure calls).
In order to enable mitogen, execute the following command:
wget https://networkgenomics.com/try/mitogen-0.2.8.tar.gz && tar xf mitogen-0.2.8.tar.gz
and uncomment the corresponding lines in the
In tests, an example task (adding the “max” user to two new user groups) took 15s on an AWS VM in the hongkong region, and 156s on a Huawei Cloud VM in the Guangzhou region (and that’s after turning on “pipelining” to reduce the number of SSH connections). Switching to mitogen brought down the timings to 1s for the AWS hongkong machine and 3s on the Huawei VM (i.e. more than one order of magnitude).