|
|
|
This thread is locked; no one can reply to it.
|
| Scalability and Youtube: |
|
nonnus29
Member #2,606
August 2002
 |
Here's a google tech talk from one of the youtube guys. They had about 6 developers grow the site from nothing to 100 million videos served a day when they were bought by google last year. One interesting thing was they forked lightppd to serve their thumbnails. Also concerning thumbnails they maxed out ext3 files per directory: http://video.google.com/videoplay?docid=-6304964351441328559 |
|
le_y_mistar
Member #8,251
January 2007
 |
google tech talks are usually interesting, iirc, lighttpd is used by a lot of big sites such as wikipedia to throw out static content such as images. ----------------- |
|
Goalie Ca
Member #2,579
July 2002
 |
Yay. I"m always on the lookout for this kind of thing. Thanks for showing ./me just started watching it. 52 mins later maybe i'll have a comment. edit: Heh in the sidebar.. it must be my lucky day (earlier today i met up with departmental chair from JHU ) ------------- |
|
nonnus29
Member #2,606
August 2002
|
Yeah, I was going to watch the llvm one too, only I had to watch the Harry Potter, St Augustine + Evil one first!
|
|
Goalie Ca
Member #2,579
July 2002
|
The youtube guy needs some help with public speaking and I skipped the part on DB's but the thing i got away from him was that "things will fail" and that "it's only a matter of time" so design the system assuming it will fail at some point. That reminded me a lot of google's map/reduce paper where they assumed some of those boxes would fail at any given instant. One of the problems i'm having is that the cluster i need to run code on ALWAYS FAILS.. clearly this is a problem Also there's this good talk on software transactional memory. I haven't actually bothered to understand it yet.. so hopefully this guy is smart. Well he must be becasue he has an accent and uses big words ------------- |
|
Matthew Leverton
Supreme Loser
January 1999
 |
I watched / listened to the video in its entirety. It's interesting, but I've heard it all before. DB too slow ... Replicate, Replicate, Replicate ... Writes overtake the Reads ... Tweak, Cache, Tweak ... Split DB up into partitions. Some of their hacks were clever, but I wonder why he even mentioned them. Sure it's cool idea to read the rows before writing them to ensure that they are pulled from memory (to avoid the stalled writes on replication), but he really needed a disclaimer, "Don't try this at home!" Living on hacks is just going to make life a nightmare. I'd rather just see a presentation on how to scale correctly. |
|
Archon
Member #4,195
January 2004
 |
Surely Allegro.cc's database is far more advanced... |
|
Matthew Leverton
Supreme Loser
January 1999
|
Yeah, I scaled it this way: Database 1 has 23yrold3yrold's post. Database 2 has everything else. |
|
Epsi
Member #5,731
April 2005
 |
Quote: Yeah, I scaled it this way: Database 1 has 23yrold3yrold's post. Database 2 has everything else.
Heh. That made me laugh ___________________________________ piccolo: "soon all new 2d alegro games will be better. after i finsh my MMRPG. my game will serve as a code reference. so you can understand and grab code from." |
|
23yrold3yrold
Member #1,134
March 2001
 |
Quote: Database 1 has 23yrold3yrold's post. Singular? All my posts have been consolidated into one? Is this going to be released as a 3-part novel series? -- |
|
le_y_mistar
Member #8,251
January 2007
|
Quote: Singular? All my posts have been consolidated into one? Is this going to be released as a 3-part novel series? thankfully, it wouldnt sell because not many of care about your neck beard WoW tales;D ----------------- |
|
23yrold3yrold
Member #1,134
March 2001
|
I don't post those on this forum anyway. -- |
|
|
Well we returned that lemon to sender and are now looking around again. 