Allegro.cc - Online Community

Allegro.cc Forums » Off-Topic Ordeals » Wednesday Moan: I've decided SVN sucks

This thread is locked; no one can reply to it. rss feed Print
 1   2 
Wednesday Moan: I've decided SVN sucks
Neil Walker
Member #210
April 2000
avatar

Here I am migrating some stuff from one system (StarTeam) to another. So after conversion (it creates a dump file) I load the dump file. I then purge unnecessary files and move files around. Then I create a dump of that.

I know I can use svndumpfilter (but it's crap), but there is no permanent deletion option (so all the files that shouldn't be there are lying around consuming space and wasting time in loading). When I watch the process of 'svnadmin load' you can see why it's painfully slow - it creates your repository as it was then applies all the changes afterwards (like my moving of folders/files and deletion of files).

I like SVN, but svn load/dump and purging is dreadful.

Neil.
MAME Cabinet Blog / AXL LIBRARY (a games framework) / AXL Documentation and Tutorial

wii:0356-1384-6687-2022, kart:3308-4806-6002. XBOX:chucklepie

Thomas Fjellstrom
Member #476
June 2000
avatar

I assume it wouldn't be that hard to parse the dump file and just delete crap you're not interested in.

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

gnolam
Member #2,030
March 2002
avatar

I assume it wouldn't be that hard to parse the dump file and just delete crap you're not interested in.

Which is pretty much what svndumpfilter does. :P

--
Move to the Democratic People's Republic of Vivendi Universal (formerly known as Sweden) - officially democracy- and privacy-free since 2008-06-18!

bamccaig
Member #7,536
July 2006
avatar

Subversion does suck, though I think this is the least of its problems. If you can, I suggest migrating to Mercurial or Git. At the very least, acquaint yourself with them so that you can recommend them to your company.

** EDIT **

Neat. :) Git actually has the ability to do this relatively easily: http://help.github.com/removing-sensitive-data/

I'll be trying it out in a minute so I'll let you know how efficient it is. :)

** EDIT **

I actually had to edit a password out of a file so I couldn't just delete it, but Git did it just fine. :) My project is quite small so it was fast, but I'm guessing it would be quite slow on a large repository (understandably). Apparently you can speed it up by specifying tmpfs as the device to work on.

Anyway, my command line was basically this:

git filter-branch --tree-filter \
        'find -name SourceFile.java -execdir \
        sed -i s/sensitive\ string/placeholder\ string/' master

git log -Ssensitive\ string
[Should be no output if it worked happily]

[Make sure the repository looks happy]

git push --force origin master

Unfortunately, apparently the sensitive information still exists in the repository, but we've basically hidden it. If somebody has the SHA1 hash for the now hidden branch then I think they can still recover the data. You'd have to create a whole new repository to completely get rid of it. This is fortunately pretty easy since you already have a complete clone of the repository (though I'm not sure whether or not pushing into the new repository will again push that sensitive information). I used this to fix a project on GitHub that had a relatively worthless password in the source code, but a password nonetheless:

http://github.com/bamccaig/bambot/blob/154a389aa0002e18e7684625663fb68a91552bb6/src/org/castopulence/bambot/BamBot.java

Note line 40. :)

BAF
Member #2,981
December 2002
avatar

Your quit function is overly verbose and unnecessary. You can just quit with your quit message. :P

bamccaig
Member #7,536
July 2006
avatar

BAF said:

Your quit function is overly verbose and unnecessary. You can just quit with your quit message. :P

It doesn't seem to actually work anyway. At least, when I've tried it in the past it would just quit without giving a reason. :) If you're referring to parting from each channel before quitting then I suspect you are correct. :P I was more or less experimenting when I wrote the existing code. I haven't actually begun writing anything remotely useful yet.

BAF
Member #2,981
December 2002
avatar

Yeah, I've noticed that the quit message randomly doesn't go through on Freenode. Try testing on a network that doesn't use buggy IRC servers. :P

Anyway, it doesn't really matter if the quit message goes through or not IMO, especially if you're writing a bot.

Steve Terry
Member #1,989
March 2002
avatar

We use CVS at the j0rb and I think it's pretty useful. I've had to make branches of projects before and merge them up after making drastic changes with little problems (.NET). I have to say though that checking out and committing changes is a little slow, but I'm getting paid for it so why complain :P.

___________________________________
[ Facebook ]
Microsoft is not the Borg collective. The Borg collective has got proper networking. - planetspace.de
Bill Gates is in fact Shawn Hargreaves' ßî+çh. - Gideon Weems

bamccaig
Member #7,536
July 2006
avatar

You just have to try out Subversion and you'll quickly see that CVS doesn't work very well. Then try out Mercurial or Git and you'll quickly see that Subversion doesn't work very well either. ;D

Billybob
Member #3,136
January 2003

bamccaig said:

Then try out Mercurial or Git and you'll quickly see that Subversion doesn't work very well either.

Git didn't have a good GUI when I tried it (a year ago). TortoiseSVN is amazing 8-)

bamccaig
Member #7,536
July 2006
avatar

There is a TortoiseGit now, but I think it's still somewhat immature and installing it isn't as simple as with the other TortoiseSCMs (you have to manually install msysgit first, which in itself isn't completely non-trivial). That's only relevant to Windows though. Linux and presumably other unices have had gitk for a long time, I think.

Not that it matters. The GUI wrappers all suck compared to the command line interface. GUIs are fundamentally flawed when it comes to non-graphical work because they require 10x longer to design and develop and 50x more work to use for non-trivial things (and they don't interface with one another easily). GUIs are nice for graphical data, like a branching graph, for example, or graphic design. Get used to using the command line. Even in Windows it's a lot faster and cleaner for 99.99999% of things (especially when you install Cygwin or MSYS to supplement it).

Thomas Fjellstrom
Member #476
June 2000
avatar

I think KDE has a git plugin now. So you can see git status from Dolphin, the file dialogs, or konqueror. :D

--
Thomas Fjellstrom - [website] - [email] - [Allegro Wiki] - [Allegro TODO]
"If you can't think of a better solution, don't try to make a better solution." -- weapon_S
"The less evidence we have for what we believe is certain, the more violently we defend beliefs against those who don't agree" -- https://twitter.com/neiltyson/status/592870205409353730

Kibiz0r
Member #6,203
September 2005
avatar

bamccaig said:

The GUI wrappers all suck compared to the command line interface.

...which also sucks.

Git is an excellent system, but goddamn can you tell it was written by coders for coders. I don't know why you'd decide to improve the mechanical side of version control without even keeping the same level of usability.

Also, for my OS X brethren, GitX is an okay GUI for Git. I still use the command line most of the time though.

Matthew Leverton
Supreme Loser
January 1999
avatar

Billybob
Member #3,136
January 2003

bamccaig said:

There is a TortoiseGit now

I tried it a year ago, and it was too buggy to use.

Quote:

The GUI wrappers all suck compared to the command line interface.

Exactly, that's the problem.

Quote:

GUIs are fundamentally flawed when it comes to non-graphical work

I value having a GUI interface to whatever SCM I use. Therefore, Git, from my point of view, sucks, because it doesn't have a good GUI. This isn't about what is the most effective way to use a tool; I am simply more comfortable using a GUI.

Also, updating a Git repo from one machine to the rest of my machines was too difficult. I tried out Git over a year ago, so my memory is vague, but I remember spending a few hours trying to set up every machine just right, and still couldn't get it to work fluidly. I eventually just gave up and setup a project on Github, but then that kind of defeated one of the key features I liked about Git (being server-less).

Elias
Member #358
May 2000

bamccaig said:

You just have to try out Subversion and you'll quickly see that CVS doesn't work very well. Then try out Mercurial or Git and you'll quickly see that Subversion doesn't work very well either.

But if you ever get to try out Perforce, you will realize that even CVS is great.

--
"Either help out or stop whining" - Evert

Neil Walker
Member #210
April 2000
avatar

I tried perforce a while ago and it was ok, but after using tortoiseSVN I prefer the non-thick client approach, either the explorer plugin like tortoise or direct IDE integration. I don't see why I should have to use another program just to manage files.

Neil.
MAME Cabinet Blog / AXL LIBRARY (a games framework) / AXL Documentation and Tutorial

wii:0356-1384-6687-2022, kart:3308-4806-6002. XBOX:chucklepie

bamccaig
Member #7,536
July 2006
avatar

Billybob said:

Also, updating a Git repo from one machine to the rest of my machines was too difficult.

Actually it's incredibly easy from Linux to Linux. You probably won't even need to worry about setting up daemons because Git can work directly over SSH. For example, if you have a repository at /home/user1/src/project1 then you could use the URI user1@host1:src/project1.

git remote add my_remote user1@host1:src/project1
git push my_remote master

And just like that, it will be updated with changes to branch master from your local copy of that repository. When you clone, you typically get a remote named origin already set up for you that points to the cloned repository.

It seems getting a daemon running with the git protocol is a little bit more work. There is git-daemon for this, but it doesn't support authentication so you'll probably only want to use it read-only (the default). I haven't quite figured out the daemon aspect of Git yet.

Billybob
Member #3,136
January 2003

bamccaig said:

Actually it's incredibly easy from Linux to Linux.

Which is awesome. SSH in Linux seems to come in handy on countless occasions. However, some of my machines run Windows. :-/

To be clear, I want to use Git. From what I have read about it, it has features I would enjoy. I just seems to have its head up its butt and refuses to support anything other than Linux with a CLI. So, here I am, stuck with SVN.

bamccaig
Member #7,536
July 2006
avatar

Billybob said:

I[t] just seems to have its head up its butt and refuses to support anything other than Linux with a CLI. So, here I am, stuck with SVN.

Unfortunately, you're just wrong. :)

Repositories can be published via HTTP, FTP, rsync, or a Git protocol over either a plain socket or ssh. Git also has a CVS server emulation, which enables the use of existing CVS clients and IDE plugins to access Git repositories. Subversion and svk repositories can be used directly with git-svn.

Basically, Git supports way more protocols than Subversion does. :P It was developed by UNIX programmers for UNIX programmers. I really don't think Linus Torvalds could care less about Windows (for good reason). In the beginning, the only way one could use Git (and therefore begin to care about it) was if they were a UNIX-like user. :)

Since some of us have to work with Windows anyway (some of us you might even choose to), other people are taking on the challenge of porting Git to Windows. They've done a good job and it is mostly usable in Windows now with msysgit. However, it just hasn't reached a point yet where anybody has put a simple Windows installer together, let alone a simple server bundle for Windows. It will happen, but you have to realize that it will take some time.

Honestly, if the code you're working on is intended to be released as open source then you're best off just using a free service, like GitHub. If proprietary, you always have the option to pay one of those services to host a private repository for you too. Or you can just run a Linux box somewhere and get a "central" repository working on it to use as a go-between. If you try hard enough, you can probably just get a service running in Windows to work as well. It seems you can use Apache and WebDAV if you're inclined to do so.

With a bit of bastardization, you could even host a Subversion repository and then use git-svn to communicate with it (though I wouldn't call that a full-fledged Git experience).

Tobias Dammers
Member #2,604
August 2002
avatar

Billybob said:

I just seems to have its head up its butt and refuses to support anything other than Linux with a CLI. So, here I am, stuck with SVN.

There's always mercurial; it's slightly less feature-laden than git, and maybe (although I haven't noticed yet) a bit slower, but while git is an amalgamation of C programs and shell scripts, and thus harder to port to non-unix-like platforms, mercurial is written in python, and the windows version is pretty much on par with the various *nix ones. There's a really nice painless introduction written by the Joel from joelonsoftware, which should help avoid a lot of confusion, especially for those coming from a central-repo SCM system.

but after using tortoiseSVN I prefer the non-thick client approach, either the explorer plugin like tortoise or direct IDE integration. I don't see why I should have to use another program just to manage files.

A CLI program is about as thin as it gets. An explorer extension is in fact just another program (or a whole bunch of them), and so is an IDE plugin; you just access them from within another program. Most IDE plugins I have worked with are even worse than Tortoise{insert SCM}. There's another reason people like CLI interfaces: They're scriptable. A distributed SCM also gives you a sophisticated incremental backup mechanism: Just write a little script that pushes all your local repositories to a backup location, put it in a cronjob, and that's it.

Oh, and before bashing any SCM, just try to remember what it's like when you have to work without one. You don't want to go there - everyone working on the same network share? bi-hourly zipped backups? keep track of versions in Excel? mailing around individual source files and puzzling them together in nerve-wrecking all-nighters? No thank you. I'd even take CVS over that.

---
Me make music: Triofobie
---
"We need Tobias and his awesome trombone, too." - Johan Halmén

bamccaig
Member #7,536
July 2006
avatar

You don't want to go there - everyone working on the same network share? bi-hourly zipped backups? keep track of versions in Excel? mailing around individual source files and puzzling them together in nerve-wrecking all-nighters? No thank you. I'd even take CVS over that.

I have it on good authority[1] that tarballs and patches are actually more sophisticated than CVS. ;) They certainly sound more friendly to me. Now, if it was a choice between working on the same network share, tracking changes with Excel, or using CVS then I would probably give in and use CVS... :'(

References

  1. Linus Torvalds'...
Trent Gamblin
Member #261
April 2000
avatar

You could probably write a VCS in Excel though. People have wasted countless hours writing all kinds of crap with that thing.

BAF
Member #2,981
December 2002
avatar

Aww, how cute. Little bam bam wants to be just like linus when he grows up. He even believes that tarballs and patches are better than CVS.

Arthur Kalliokoski
Second in Command
February 2005
avatar

BAF said:

Aww, how cute. Little bam bam wants to be just like linus when he grows up. He even believes that tarballs and patches are better than CVS.

You use Visual SourceSafe, I suppose?

[EDIT]

I found this googling:

"SourceSafe was a perfectly adequate source control system in the late 90's. Unfortunately, SourceSafe was never updated architecturally to reflect modern source control practices. Even the latest version, SourceSafe 2005, absolutely reeks of 1999. And, to be fair, some of the same criticisms apply to CVS. CVS is no longer a modern source control system, either; it doesn't even support the concept of atomic checkins"

http://www.codinghorror.com/blog/2006/08/source-control-anything-but-sourcesafe.html

“Throughout history, poverty is the normal condition of man. Advances which permit this norm to be exceeded — here and there, now and then — are the work of an extremely small minority, frequently despised, often condemned, and almost always opposed by all right-thinking people. Whenever this tiny minority is kept from creating, or (as sometimes happens) is driven out of a society, the people then slip back into abject poverty. This is known as "bad luck.”

― Robert A. Heinlein

 1   2 


Go to: