FTP Sync for Web Developers

FTP Sync for Web Developers

Billybob

Member #3,136

January 2003

As a web developer I find trying to sync the local copy of my website with the remote copy to be quite painful. If I change files in two different directories I then have to go to my FTP client, switch directories, switch remote directory, upload, switch, switch, upload.

I have always wanted a way to just hit "Sync" and have everything uploaded for me. So today I made the tool myself. I wrote it in Java, so I guess it'd work on any machine, for you Linux Web Developers

The tool keeps track of files locally, to determine if they change, and does not use the remote FTP server as a reference. This is because trying to sync with the timestamps on the FTP server is far too difficult.

The tool is not complete. Things to do:

Remote directory removal
CLI
Possible GUI later

Here's the code:

1import org.apache.commons.net.ftp.*;
2import java.io.*;
3import java.util.*;
4 
5public class FTP_Sync
6{
FTPClient ftp;

public static void main(String[] args)
{
  FTP_Sync me = new FTP_Sync();
  try
  {
    me.connect("ftp.yourserver.com", "your_user_name", "your_password");
    me.sync("./sync_test/", "/public_html/sync_test/");
  }
  catch(Exception e)
  {
    System.out.println(e);
  }
  
  System.out.println("Sync complete");
}

public void connect(String server, String username, String password) throws Exception
{
  ftp = new FTPClient();
  ftp.connect(server);
  if(!FTPReply.isPositiveCompletion(ftp.getReplyCode()))
    throw new Exception("Connect failed.");
  
  ftp.login(username, password);
  if(!FTPReply.isPositiveCompletion(ftp.getReplyCode()))
    throw new Exception("Login failed.");
}

public Properties getTimeStamps(String directory) throws Exception
{
  Properties prop = new Properties();
  File file = new File(directory + ".ftp.sync");
  if(file.exists())
  {
    InputStream stream = new FileInputStream(new File(directory + ".ftp.sync"));
    prop.load(stream);
    stream.close();
  }
  return prop;
}

public void writeTimeStamps(String directory, Properties timeStamps) throws Exception
{
  OutputStream stream = new FileOutputStream(new File(directory + ".ftp.sync"));
  timeStamps.store(stream, "");
  stream.close();
}

public boolean shouldUpload(File source, FTPFile[] files, Properties timeStamps)
{
  long src_time = source.lastModified();
  long dest_time = new Long(timeStamps.getProperty(source.getName(), "0"));
  
  if(dest_time < src_time)
    return true;
  return false;
}

public void uploadFile(File source, String destination) throws Exception
{
  InputStream stream = new FileInputStream(source);
  ftp.storeFile(destination + source.getName(), stream);
  if(!FTPReply.isPositiveCompletion(ftp.getReplyCode()))
    throw new Exception("Failed to upload file: " + source.getName());
  stream.close();
}

public FTPFile findFile(String name, FTPFile[] files)
{
  for(FTPFile file : files)
    if(file.getName().equals(name))
      return file;
  return null;
}

public void sync(String source, String dest) throws Exception
{
  FTPFile[] dest_files = ftp.listFiles(dest);
  File[] src_files = new File(source).listFiles();
  Properties timeStamps = getTimeStamps(source);
  
  for(File src_file : src_files)
  {
    if(src_file.getName().equals(".ftp.sync"))
      continue;
    
    if(src_file.isDirectory())
    {
      FTPFile dir = findFile(src_file.getName(), dest_files);
      if(dir == null)
      {
        ftp.makeDirectory(dest + src_file.getName());
        System.out.println("Created " + src_file.getName());
      }
      sync(source + src_file.getName() + "/",  dest + src_file.getName() + "/");
      continue;
    }
    
    if(shouldUpload(src_file, dest_files, timeStamps))
    {
      uploadFile(src_file, dest);
      System.out.println("Uploaded " + src_file.getName());
    }
    
    timeStamps.setProperty(src_file.getName(), src_file.lastModified() + "");
  }
  
  // Remove files that no longer exist locally
  for(FTPFile file : dest_files)
  {
    // TODO: Special check for this locally
    if(file.isDirectory())
      continue;
    
    String test = timeStamps.getProperty(file.getName(), "Not Found");
    if(test.equals("Not Found"))
    {
      ftp.deleteFile(dest + file.getName());
      System.out.println("Removed: " + file.getName());
    }
  }
  
  writeTimeStamps(source, timeStamps);
}

134}

Explanation:
The tool will ensure that your remove web development directory has all the latest files from your local web development directory. It goes through each local file and sees if anything changed since the last time you used the tool. When it finds something different it uploads the changes to the FTP server.

If you find it useful then that's great, but I don't really care because I'll keep working on it no matter what ... I need it.
Enjoy?

Matthew Leverton

Supreme Loser

January 1999

It's a common problem, so you are probably doing a lot of wheel re-inventing. I use WinSCP for transferring files because: 1) it's secure and 2) it has a lot of syncing tools built in (like synchronized browsing).

Also there is rsync, which is very nice for keeping two locations identical.

--
RTFM | Follow Me on Google+ | I know 10 people

Thomas Fjellstrom

Member #476

June 2000

And neither work all that well if all you have is ftp.

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

gnolam

Member #2,030

March 2002

Quote:

It also has a Norton Commander-like interface, which is always a good thing.

--
Move to the Democratic People's Republic of Vivendi Universal (formerly known as Sweden) - officially democracy- and privacy-free since 2008-06-18!

Carrus85

Member #2,633

August 2002

Well, it looks nice, but I can see some potential optimizations. For example, you could use the java.util.TreeSet (or HashSet) to cache the results from the FTP server once instead of running through the list multiple times in your findFile function.

Additionally, you should technically put your stream.close() statements inside of a finally block, otherwise they won't get called upon an exception being thrown.

Looking good though

kazzmir

Member #1,786

December 2001

I wrote a ruby program to do exactly this. Its pure CLI and suits my needs. Check it out if you want:

filesync