Dec 14 2005
power failure
Our datacenter has had full power outage, and we are now recovering from the aftermath. About half of the machines are still down and we are working on bringing them back as soon as possible. To keep things moving, I’ve turned off RSS and some other features so that posting will still work and so that you will still have access to your bookmarks.
5pm EST Update: Everything is back up. We apologize for the inconvenience. It looks like a number of slave servers are corrupted. We’re rebuilding them now and taken the site down so this goes as quickly as possible.
Update: Some servers are coming up. Should be going much faster from here on in.
PS — we are obviously not yet on the Yahoo! infrastructure, and this is exactly the kind of stuff they can do a lot better for us (and you.)
Update: Most stuff is up, finally. Tag intersections are currently not working, and search is not updating — a few servers would not come up. We’ll bring them up tomorrow when we return to the east coast.
Joshua Schachter · joshua Tags: service-alert Bookmark this
49 Comments
New User… | Dec 14 2005 at 10:40 am
Lets say: yahoo! ;-)
BTW, sending the verification email doesn’t work at the moment.
shekay… | Dec 14 2005 at 11:29 am
clueless RSS user question: Is there a way for me to subscribe to a feed for “service alert”? I have a special folder in my aggregator set aside for these types of announcements (my ISP, mail provider, etc.). The main del blog feed wouldn’t aptly go there.
Bramus!… | Dec 14 2005 at 1:44 pm
Posting works, but:
- the pagecount doesn’t increase.
- the tagcount doesn’t increase, nor does the adding of newly defined tags.
- the verification email sending doesn’t work.
Hope you get everything back up real soon.
Best of luck!
B!
teo… | Dec 14 2005 at 2:27 pm
Please, let us know what datacenter it is so we never host anything important there.
Yasha… | Dec 14 2005 at 2:45 pm
I always hated yahoo :) Why not google? ;) With google you’l never have power failure.
Judson… | Dec 14 2005 at 2:51 pm
they’re probably not in the yahoo server system a few days after the announcement ;)
Brock Tice… | Dec 14 2005 at 3:41 pm
Is this why I can’t rename tags?
Ivan… | Dec 14 2005 at 4:10 pm
Why are there mysql errors all the time?
Steve… | Dec 14 2005 at 4:18 pm
Seen anyone in a Google t-shirt with a pair of wirecutters recently?
Wyclif… | Dec 14 2005 at 5:03 pm
Don’t make the mistake of thinking that just because Google is big they do not have power difficulties:
http://hardware.silicon.com/servers/0,39024647,39154982,00.htm
Again, the problem is scale. You can solve the hardware scaling problem, but you’re still dependent on California’s power grid.
ZEO… | Dec 14 2005 at 5:33 pm
…I’m starting to realize that del.icio.us is really important to me.
hillary… | Dec 14 2005 at 5:38 pm
glad to hear all the servers are back up. however, still getting lots of mySQL errors, and the tagging problems others have listed above are not resolved.
any outlook on when things will be functioning normally?
Judi Sohn… | Dec 14 2005 at 10:30 pm
These things happen. but could you do a little bit better than “del.icio.us is down for maintenance. we’ll be back in one hour.” Without a timestamp, that’s meaningless.
Jared White… | Dec 14 2005 at 10:48 pm
Ack, I want my del.icio.us! Anyway, I’m sorry to hear about your woes and wish you the best of luck in getting everything back up and running. I know how awful this stuff can be sometimes.
As one tech guy to another,
Jared
joga… | Dec 14 2005 at 10:55 pm
del.icio.us down
del.icio.us is down and that puts a cramp in my net style.
…
Richard Soderberg… | Dec 14 2005 at 10:57 pm
A link to the blog post about the downtime would be handy on the downpage, the next time there’s an outage. Don’t spend any time on it this time, though – outages suck :(
A View from Home… | Dec 14 2005 at 11:38 pm
Del.icio.us fall down
Just got back from a trip to DC (lots more to blog about tomorrow), went to my del.icio.us page (at around 9 pm EST) and saw this: del.icio.us is down for maintenance. we’ll be back in one hour. Now it’s…
Michael Kebbekus… | Dec 15 2005 at 2:00 am
del.icio.us:
Where are you? I miss you.
nowak… | Dec 15 2005 at 3:05 am
Is there an updated ETA on this? It’s been a long “one hour.”
Jackie… | Dec 15 2005 at 3:10 am
Yahoo is copying all the links to MyWeb2.0 site! In the mean time, you are fooled to believe it’s a power failure! Good that i made a offline backup yesterday!
m.mashima… | Dec 15 2005 at 4:50 am
I wanted to know what’s going on.
I think that it is necessary to separate to read BLOG at any time.
Jerome… | Dec 15 2005 at 5:16 am
So long and thanks for all the fish!
Tollie… | Dec 15 2005 at 5:20 am
(LOL at Steve’s comment)
Hey guys, besides what they said about linking the ‘we got trouble’ page to this blog/status page, will ya remember to put the time of the update when you edit the post to say ‘Update:’ ? Just a simple (“7pm EST Update: keep your pants on”) type thing. Thanks.
Lloyd D Budd… | Dec 15 2005 at 6:02 am
No co-lo?!
yukkie(JA)… | Dec 15 2005 at 6:24 am
del.icio.us seems to be recovered completely for me.
Thanks your best efforts!
toydi… | Dec 15 2005 at 6:32 am
Congratulation to del.icio.us! It seems to up again here too!
Btw, is there a good way to manage an offline backup from del.icio.us?
(Please send tag for:toydi to me, if you have a good solution, thanks in advance!)
Kevin… | Dec 15 2005 at 8:48 am
I was really scared back there… no del.icio.us? I shudder at that thought! :D
706GL… | Dec 15 2005 at 9:06 am
I picked the wrong day to see what this whole del.icio.us thing was about. Still haven’t received my email to set up an account.
Son Nguyen… | Dec 15 2005 at 11:31 am
I thought it was being transferred to Yahoo and the transition went bad. We have had one of our data centers down because of power failure recently. Energy is actually becoming a bigger problem for web hosting, just as Google warned the other day.
Dave… | Dec 15 2005 at 2:09 pm
My tag counts are wrong.
Linda… | Dec 15 2005 at 3:05 pm
My tag counts are wrong too and new tags don’t show up in the list.
Nick… | Dec 15 2005 at 5:42 pm
Intersection pages are getting cleared for me every time I add a tag to a link that is included in that intersection. Is there an ETA on when those issues are to be resolved?
RichS… | Dec 17 2005 at 8:49 pm
You know, people, it’s nice outside: you could leave the internet alone for maybe a couple hours, and then when you came back to it, if it wasn’t all better you might be at least have calmed down and got some perspective.
Francisco… | Dec 18 2005 at 6:19 am
I can’t loggin to my account! (SyncMaster)… please help me!
george kronberg… | Dec 18 2005 at 6:29 am
on my page: “All your items (11) …[list_of_my_tags]… No items”. ??
Francisco… | Dec 18 2005 at 8:19 am
I can loggin now. Thanks for fixed the problem.
Jean-François… | Dec 18 2005 at 5:02 pm
down again
Will… | Dec 18 2005 at 5:16 pm
I’m getting down for maintenance on Sunday 12-18. What’s the story?
Jonathan… | Dec 18 2005 at 6:21 pm
I’m getting down for maintenance too. Good luck getting everything back up and running asap.
-Jon
http://philoneist.com
keith… | Dec 18 2005 at 6:25 pm
Any info would be great. Did godzilla attack the data center?
eileen… | Dec 18 2005 at 9:28 pm
Still down for me too (Sunday, 12/18/05 6:32 PM PST). I use an RSS feed to show my podcast’s archived shows (so folks can surf for the audiobooks easier).
I noticed during the last outage you guys went back up but didn’t enable RSS? Is that goign to happen this time too?
I’m gearing to send some emails out about my podcast but will hold off if the RSS feeds won’t be showing.
Robert… | Dec 18 2005 at 10:54 pm
Perhaps it was King Kong who attacked the data center… Seriously, guys… ANY info about the cause of this ‘emergency maintenance’ and outtage would be great.
~R
Paul… | Dec 18 2005 at 11:43 pm
So bad. Soooooo bad. How much was this company bought for?
Karthik's Rambling!… | Dec 19 2005 at 1:00 am
Yahoo acquires del.icio.us
MuPa… | Dec 19 2005 at 5:40 pm
del.icio.us continued hiccups
Sembra che del.icio.us soffra di problemi. Ieri sera non riuscivo a caricare alcunché. Il messaggio segnalava errori generici, poi oggi una nota sul blog segnala che hanno qualche problema di corrente.
Il problema risale a qualche giorno fa (sempre dal b
Gerd Müller… | Dec 27 2005 at 6:47 am
Looking for a Bookmark Manager to store your private Bookmarks?
No social bookmarks, no tags, no endless clicking and searching.
Just your own bookmarks – accessible all over the World, in any internet
browser.
Try Web Favorites. http://www.webfavs.de/?lang=en
Greetings.
Nina… | Jan 23 2006 at 5:31 pm
Congratulation to del.icio.us! It seems to up again here too! Btw, is there a good way to manage an offline backup from del.icio.us? (Please send tag for:toydi to me, if you have a good solution, thanks in advance!)
alaa… | Mar 29 2006 at 9:02 pm
Thanks for fixed the problem.
dafphee… | Apr 28 2006 at 6:54 pm
my country is still grapling with power problem so very often there is power outage. I have a ups as backup but the problem is sometimes when there is an outage, my system goes of instantly and starts rebooting. I thought the ups was bad, I changed it and still the problem remains. I have just changed it again and my problem still stays.
I have changed vertually everything from casing to power pack. mother board. processor and my hard disks are crashing like ‘no man’s business’.
can somebody help.
My mother board is gygabyte p4 and 3.20 processor speed.
some body pls help before I crash along side the my pc.
dafphee