Google Cloud Platform – How to Upload a Folder onto a Virtual Machine

google-cloud-platformgoogle-cloud-storagegoogle-compute-enginescpvirtual-machines

I created a VM instance on the Compute Engine in a Google Cloud project. I want to upload a folder from my computer's local storage onto the boot disk of the virtual machine.

I tried using the scp command in the command line interface of my computer by typing scp /file/path/name external-ip-address-of-server:. But then I got the errorPermission denied (publickey). lost connection

I'm also aware that the wget command can be used to download a file from the internet onto a computer. I thought of using this, but that would require putting my folder online as a download link, which I don't know how to do.

What's the best way to put a folder from local storage onto the boot disk of a virtual machine?

Best Answer

The tool you are looking for is probably rsync.

A possible command line might be:

rsync -av /file/path/name external-ip-address-of-server:

Rsync uses ssh as its default transport, so before it will work easily for you, it looks like you need to get your ssh logins working. But that's another question, and one that StackExchange probably already has an answer for.

Related Solutions

Linux – How to download a file on a server you’ve already ssh’ed into

What you need to answer is why you don't want to log again ... if you logged once, you can log twice ... especially if you've taken the three minutes it takes to create a key and store the key remotely:

$ ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
2b:4c:11:ab:53:d1:15:90:4f:88:a1:42:da:c5:1c:98 user@localhost
$ ssh-copy-id user@remotehost
Warning: Permanently added 'remotehost' (RSA) to the list of known hosts.
Password:
$ ssh user@remotehost
Last login: Mon Dec 14 21:52:45 2009 from 1.2.3.4
[user@remotehost] $

Back in the times before TCP/IP was commonplace, people used kludges such as X/Modem to handle this. Using this nowadays is kinda silly.

How to boot a virtual machine image from the network

You are combining two things: (1) diskless booting which is common in HiPC clusters, where tools like compilers, libraries, etc, are stored on a clustered file system like Lustre or NFS; (2) virtual disk images (vmdk or vdi, or whatever else).

In case (1), you need a tftp server and a dhcp server, along with a modified initrd image for customized drivers, scripts, etc.

http://www.pixelchaos.net/2009/02/15/diskless-booting-with-pxe-and-nfs/

In case (2), given all machines are same, including configurations, you can nfs mount the virtual disk image. Write a script that checks for any changes like whether the existing image is same as nfs mounted; if they are not same, copy this image to boot partition and reboot it. You need to add this script in initrd. You can play with that.

http://www-conf.slac.stanford.edu/afsbestpractices/Slides/afsboot.pdf

The paper above gives you some clues. You better focus on how the whole boot process works, etc.

If you are using ESXi, there is a better way to achieve what you want: export nfs to esxi hosts. Keep the vmdk and -flat.vmdk files on the nfs server, and change vmdk location in .vmx file. During the next boot, it picks up your new image.

Best Answer

Related Solutions

Linux – How to download a file on a server you’ve already ssh’ed into

How to boot a virtual machine image from the network

Related Topic