Allegro.cc - Online Community

Allegro.cc Forums » Off-Topic Ordeals » Backup Strategy

This thread is locked; no one can reply to it. rss feed Print
Backup Strategy
Fishcake
Member #8,704
June 2007
avatar

The company I work at doesn't actually enforce any backup strategy, and I'm getting a little bit worried. Sure, we have been working on this particular project for 2 years, and no bad things have happened so far. But a couple of times, I was super close to wiping the entire production database :o, so I think it's time to periodically backup our stuff just in case.

We have these data that need to be backed up:

  • Git repositories

  • Perforce repository containing art assets (e.g. PSD, PNG, etc)

  • MySQL database

For now, I'm thinking of setting up a Jenkins job that would simply:

  • bare clone all the git repositories and tar them,

  • clone the art asset repository and tar it,

  • put the server on maintenance, dump the production database, and tar it (it's better to setup a master-slave replication so that I can perform backup without taking the database down, but we don't have the resource to setup an extra instance for the slave)

every night, and then upload them to S3, and maybe to some other place as well for redundancy. Maybe every now and then (say a month), if I'm not lazy, I'll put them on a hard disk.

So, is this sufficient? Or am I doing something wrong? Is there anything I should be concerned of?

Matthew Leverton
Supreme Loser
January 1999
avatar

Spend a few dollars and store your private repos in Bitbucket or Github.

Your database should be replicated in realtime. You can set up automatic backups on the replica to get consistent snapshots and without affecting performance on the master database. Amazon RDS does this (and more). Where are you hosting? The slave DB could cost as little as $5/month.

I'm not familiar with Perforce, but does it do anything more than what you would get out of git (and using Bitbucket/Github)?

None of these things cost very much money.

Fishcake
Member #8,704
June 2007
avatar

Spend a few dollars and store your private repos in Bitbucket or Github.

I'll keep that in mind. Currently, all of our repos are stored in our company's Gitlab, which is installed on a physical server stashed in a dark corner somewhere. Knowing the company well, I don't think it is being backed up.

Quote:

Where are you hosting?

We are hosting on AWS.

Quote:

I'm not familiar with Perforce, but does it do anything more than what you would get out of git (and using Bitbucket/Github)?

The reason Perforce is used is because the art assets are mostly large PSD files, and I don't think git handles large binary files very well. Git also lacks access control, which is needed for binary files since they can't be merged.

Quote:

None of these things cost very much money.

Sure, it doesn't cost much, but if you consider the fact that we're a small team that has a strict budget, working on a project that so far has not made any money, I don't think I'll able to convince my boss to spend extra money on something that we can do ourselves. :P

But I do agree that it's better to pay someone else to do it for us instead of rolling it ourselves.

Chris Katko
Member #1,881
January 2002
avatar

Github, Gitolite, and Gerrit all have an access control layer on top of git.

-----sig:
“Programs should be written for people to read, and only incidentally for machines to execute.” - Structure and Interpretation of Computer Programs
"Political Correctness is fascism disguised as manners" --George Carlin

StevenVI
Member #562
July 2000
avatar

Fishcake said:

Knowing the company well, I don't think it is being backed up.

It sounds like you're getting a little bit overzealous. Maybe make sure that there isn't already a plan in place for all of this?

Quote:

I don't think I'll able to convince my boss to spend extra money on something that we can do ourselves.

There's a real dollar figure for every minute of time you spend doing something, and I'm sure your boss is aware of that.

Anecdotally, I recall for a game I was working on once our animator estimated a few weeks to animate a model of a horse. (We didn't even have a few weeks to spare for that work anyways.) Then we realized that we could buy a pre-made animation for only like $200. We used the pre-made one and saved thousands of dollars as a result, and everyone was happy.

__________________________________________________
Skoobalon Software
[ Lander! v2.5 ] [ Zonic the Hog v1.1 ] [ Raid 2 v1.0 ]

Fishcake
Member #8,704
June 2007
avatar

StevenVI said:

It sounds like you're getting a little bit overzealous. Maybe make sure that there isn't already a plan in place for all of this?

Maybe I am. :P But I'm just trying to be cautious, having read this article.

Quote:

There's a real dollar figure for every minute of time you spend doing something, and I'm sure your boss is aware of that.

I'll keep that in mind. :)

Github, Gitolite, and Gerrit all have an access control layer on top of git.

I think you misunderstood. By access control, I mean the ability to be able to lock a file so that no other people can edit the file while I have the lock. If an artist is editing a PSD file, I don't want another artist to be editing the same file. But it is kind of an overkill to pick Perforce over that feature, since we only have 3 artists, and they sit beside each other. :P But I still don't think Git is good for really large files. (maybe I'm wrong?)

Go to: