As a web developer I find trying to sync the local copy of my website with the remote copy to be quite painful. If I change files in two different directories I then have to go to my FTP client, switch directories, switch remote directory, upload, switch, switch, upload.
I have always wanted a way to just hit "Sync" and have everything uploaded for me. So today I made the tool myself. I wrote it in Java, so I guess it'd work on any machine, for you Linux Web Developers 
The tool keeps track of files locally, to determine if they change, and does not use the remote FTP server as a reference. This is because trying to sync with the timestamps on the FTP server is far too difficult.
The tool is not complete. Things to do:
Remote directory removal
CLI
Possible GUI later
Here's the code:
| 1 | import org.apache.commons.net.ftp.*; |
| 2 | import java.io.*; |
| 3 | import java.util.*; |
| 4 | |
| 5 | public class FTP_Sync |
| 6 | { |
| 7 | FTPClient ftp; |
| 8 | |
| 9 | public static void main(String[] args) |
| 10 | { |
| 11 | FTP_Sync me = new FTP_Sync(); |
| 12 | try |
| 13 | { |
| 14 | me.connect("ftp.yourserver.com", "your_user_name", "your_password"); |
| 15 | me.sync("./sync_test/", "/public_html/sync_test/"); |
| 16 | } |
| 17 | catch(Exception e) |
| 18 | { |
| 19 | System.out.println(e); |
| 20 | } |
| 21 | |
| 22 | System.out.println("Sync complete"); |
| 23 | } |
| 24 | |
| 25 | public void connect(String server, String username, String password) throws Exception |
| 26 | { |
| 27 | ftp = new FTPClient(); |
| 28 | ftp.connect(server); |
| 29 | if(!FTPReply.isPositiveCompletion(ftp.getReplyCode())) |
| 30 | throw new Exception("Connect failed."); |
| 31 | |
| 32 | ftp.login(username, password); |
| 33 | if(!FTPReply.isPositiveCompletion(ftp.getReplyCode())) |
| 34 | throw new Exception("Login failed."); |
| 35 | } |
| 36 | |
| 37 | public Properties getTimeStamps(String directory) throws Exception |
| 38 | { |
| 39 | Properties prop = new Properties(); |
| 40 | File file = new File(directory + ".ftp.sync"); |
| 41 | if(file.exists()) |
| 42 | { |
| 43 | InputStream stream = new FileInputStream(new File(directory + ".ftp.sync")); |
| 44 | prop.load(stream); |
| 45 | stream.close(); |
| 46 | } |
| 47 | return prop; |
| 48 | } |
| 49 | |
| 50 | public void writeTimeStamps(String directory, Properties timeStamps) throws Exception |
| 51 | { |
| 52 | OutputStream stream = new FileOutputStream(new File(directory + ".ftp.sync")); |
| 53 | timeStamps.store(stream, ""); |
| 54 | stream.close(); |
| 55 | } |
| 56 | |
| 57 | public boolean shouldUpload(File source, FTPFile[] files, Properties timeStamps) |
| 58 | { |
| 59 | long src_time = source.lastModified(); |
| 60 | long dest_time = new Long(timeStamps.getProperty(source.getName(), "0")); |
| 61 | |
| 62 | if(dest_time < src_time) |
| 63 | return true; |
| 64 | return false; |
| 65 | } |
| 66 | |
| 67 | public void uploadFile(File source, String destination) throws Exception |
| 68 | { |
| 69 | InputStream stream = new FileInputStream(source); |
| 70 | ftp.storeFile(destination + source.getName(), stream); |
| 71 | if(!FTPReply.isPositiveCompletion(ftp.getReplyCode())) |
| 72 | throw new Exception("Failed to upload file: " + source.getName()); |
| 73 | stream.close(); |
| 74 | } |
| 75 | |
| 76 | public FTPFile findFile(String name, FTPFile[] files) |
| 77 | { |
| 78 | for(FTPFile file : files) |
| 79 | if(file.getName().equals(name)) |
| 80 | return file; |
| 81 | return null; |
| 82 | } |
| 83 | |
| 84 | public void sync(String source, String dest) throws Exception |
| 85 | { |
| 86 | FTPFile[] dest_files = ftp.listFiles(dest); |
| 87 | File[] src_files = new File(source).listFiles(); |
| 88 | Properties timeStamps = getTimeStamps(source); |
| 89 | |
| 90 | for(File src_file : src_files) |
| 91 | { |
| 92 | if(src_file.getName().equals(".ftp.sync")) |
| 93 | continue; |
| 94 | |
| 95 | if(src_file.isDirectory()) |
| 96 | { |
| 97 | FTPFile dir = findFile(src_file.getName(), dest_files); |
| 98 | if(dir == null) |
| 99 | { |
| 100 | ftp.makeDirectory(dest + src_file.getName()); |
| 101 | System.out.println("Created " + src_file.getName()); |
| 102 | } |
| 103 | sync(source + src_file.getName() + "/", dest + src_file.getName() + "/"); |
| 104 | continue; |
| 105 | } |
| 106 | |
| 107 | if(shouldUpload(src_file, dest_files, timeStamps)) |
| 108 | { |
| 109 | uploadFile(src_file, dest); |
| 110 | System.out.println("Uploaded " + src_file.getName()); |
| 111 | } |
| 112 | |
| 113 | timeStamps.setProperty(src_file.getName(), src_file.lastModified() + ""); |
| 114 | } |
| 115 | |
| 116 | // Remove files that no longer exist locally |
| 117 | for(FTPFile file : dest_files) |
| 118 | { |
| 119 | // TODO: Special check for this locally |
| 120 | if(file.isDirectory()) |
| 121 | continue; |
| 122 | |
| 123 | String test = timeStamps.getProperty(file.getName(), "Not Found"); |
| 124 | if(test.equals("Not Found")) |
| 125 | { |
| 126 | ftp.deleteFile(dest + file.getName()); |
| 127 | System.out.println("Removed: " + file.getName()); |
| 128 | } |
| 129 | } |
| 130 | |
| 131 | writeTimeStamps(source, timeStamps); |
| 132 | } |
| 133 | |
| 134 | } |
Explanation:
The tool will ensure that your remove web development directory has all the latest files from your local web development directory. It goes through each local file and sees if anything changed since the last time you used the tool. When it finds something different it uploads the changes to the FTP server.
If you find it useful then that's great, but I don't really care because I'll keep working on it no matter what ... I need it.
Enjoy?
It's a common problem, so you are probably doing a lot of wheel re-inventing. I use WinSCP for transferring files because: 1) it's secure and 2) it has a lot of syncing tools built in (like synchronized browsing).
Also there is rsync, which is very nice for keeping two locations identical.
And neither work all that well if all you have is ftp.
It's a common problem, so you are probably doing a lot of wheel re-inventing. I use WinSCP for transferring files because: 1) it's secure and 2) it has a lot of syncing tools built in (like synchronized browsing).
It also has a Norton Commander-like interface, which is always a good thing.
Well, it looks nice, but I can see some potential optimizations. For example, you could use the java.util.TreeSet (or HashSet) to cache the results from the FTP server once instead of running through the list multiple times in your findFile function.
Additionally, you should technically put your stream.close() statements inside of a finally block, otherwise they won't get called upon an exception being thrown.
Looking good though
I wrote a ruby program to do exactly this. Its pure CLI and suits my needs. Check it out if you want: