Tirea Radio is currently ON AIR

Started by Seze, December 06, 2010, 09:28:18 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

Seze

So after a power failure that ended up causing one of my servers to die a slow and horrible death, Tirea Radio is now, finally, back online.  I'm still having some issues getting the now playing stuff working again on the website, but thats a small issue compared to getting the stream back online. 


Learn Na'vi Mobile App - Now Available

Seze

After a long and tedious 4 hours, I've finally got the server back online.  For some reason the filesystem got corrupted and when I first tried to fix it, the server crashed on me.  Glad I was home when this happened, I was about an hour away from leaving on a business trip.  Anyways, so I got the filesystem repaired and after a long battle with kernel panics and mdadm, the server is finally back online.  All services should be back online as well.  If you can't get anything to work, please let me know and I will try to get it fixed ASAP.

Irayo


Learn Na'vi Mobile App - Now Available

Carborundum

We learn from our mistakes only if we are made aware of them.
If I make a mistake, please bring it to my attention for karma.

Seze

Quote from: Carborundum on December 12, 2010, 10:05:13 AM
It appears to be down again. :-\

That is correct.  The server has been having some filesystem issues lately.  I think my server found out I was coming home today, so it decided to crash to give me something to fix while I'm at home.  The server will be offline for another hour or two while I rebuild the server from backups.  I think I screwed something up with the raid array, so no matter what I do, it keeps erroring out after a few days, so I'm just going to rebuild the filesystem and raid array from scratch to blow out whatever it is that I screwed up.  I'll update this thread once I get things working again, which should be sometime in the next hour or two...


Learn Na'vi Mobile App - Now Available

Seze

Oeru txoa livu, its taking much longer than expected.  Turns out, when I went to rebuild my raid array, it was built the a newer version of the raid utility (mdadm) which makes the rebuilt raid arrays I made today, completely incompatible with my current system, thus resulting in a non booting server.   CentOS can be really annoying when problems like this come up.  Its nice and stable, unless you use something a tad bit newer and it tanks your whole system...  I'm getting really tired right now, been up all night debugging this issue.  I'll get this fixed tomorrow morning when I get up.


Learn Na'vi Mobile App - Now Available

Seze



Learn Na'vi Mobile App - Now Available

Carborundum

Thanks for working so hard on this. Know that it is appreciated. :)
We learn from our mistakes only if we are made aware of them.
If I make a mistake, please bring it to my attention for karma.

Tirea Aean

Quote from: Carborundum on December 13, 2010, 03:36:04 PM
Thanks for working so hard on this. Know that it is appreciated. :)

very much so THIS.

Seze

Just a heads up, I'll need to take the server down sometime in the next few days.  When I was working on the server recently, I split up the HDD a bit more and I never got a chance to fully integrate the extra partitions into the system.  I won't be able to work on it until I make it home again, which won't be till the end of this week.  I'll probably try to work on it either Sunday or Monday.  I'll try to give a days notice before I work on it.  If anyone has a time they absolutely can't go without Tirea Radio, just speak up and I'll try to work around that time.


Learn Na'vi Mobile App - Now Available

Seze

#9
I'm moving my server into a new rack and as such, I need to power it down in order to move it.  I'm starting the transfer process around 1200 UTC/GMT today.  I expect this process to take up to an hour to complete...


Learn Na'vi Mobile App - Now Available

Seze

Had a power failure this morning which took Tirea Radio offline.  Once power was restored, the server failed to boot back up again.  I'm currently working to get services transitioned over to one of my other servers.  I'm not sure how long its going to take to get this fixed, so expect that it may be offline for a few days.


Learn Na'vi Mobile App - Now Available

Tirea Aean

Quote from: Seze on July 08, 2011, 02:48:49 PM
Had a power failure this morning which took Tirea Radio offline.  Once power was restored, the server failed to boot back up again.  I'm currently working to get services transitioned over to one of my other servers.  I'm not sure how long its going to take to get this fixed, so expect that it may be offline for a few days.


thanks for the heads up man. Even in times like this, I thank you for what you are doing and the time/resources you sacrifice for the Tirea Radio project.

Seze

I replaced all the batteries in my APC Smart-UPS 3000 today.  The ones that were in it were estimated to be 4 or 5 years old, which is way past their estimated life-span.  There was a 2 hour power outage today and I was a little surprised that the UPS wasn't able to handle that, so thought it might be a good time to do a battery upgrade.  Replaced all 8 of the 12v 5Ah batteries for $170...


Learn Na'vi Mobile App - Now Available

Tirea Aean


Seze

#14
Quote from: Tirea Aean on July 08, 2011, 07:34:25 PM
nice. So we up?

No...

The server that was hosting all Tirea Radio services is dead, and I'm not planning on reviving it.  Instead I'm going to move Tirea Radio to my other server.  I just updated the DNS settings for tirearadio.com to point to the other server, so I should be able to get the website back up sometime soon, but the actual radio stream may take a little while longer to get back online.  I've been trying to install liquidsoap on the other server for the past couple of nights, I'm about to give up try an older build that will work with the software versions I have access to.  I keep running into compile errors with the current build they have out now.  I think its mostly due to not having access to the cutting edge builds of different software packages but I'm not 100% certain.  I still need to transfer all the files over to the other server, but before I can do that I have to figure out how to get the data off the server's hard drive.  Mac doesn't read ext3 very well out of the box...


UPDATE:
> The hard drives were intact and I am now pulling the data off of them and putting it on the server. 
> The website should be back up now.  Somethings will appear broken due to the stream being offline
> I still need to get liquidsoap compiled properly on the server, and that is the last big item left before we are back online...


Learn Na'vi Mobile App - Now Available

Seze

So after a power failure that ended up causing one of my servers to die a slow and horrible death, Tirea Radio is now, finally, back online.  I'm still having some issues getting the now playing stuff working again on the website, but thats a small issue compared to getting the stream back online. 


Learn Na'vi Mobile App - Now Available

akiwiguy

YES!

Ma Seze, nga lu "fracking" txantsan nì'aw.
*hugs*

Tirea Aean

Update:

palulukan down.

All webserver and radio streaming services are currently (and hopefully temporarily) offline.

Seze

Oe lu skxawng...  I was doing some work in the server rack last night and the ethernet cable that goes to my main server (palulukan) came loose.  Everything should be back online now... 


Learn Na'vi Mobile App - Now Available

Seze

Just giving you all a heads up, I'm getting a larger block of static IPs from Comcast, which means things will go offline for a bit after the new block is assigned.  I was told to expect the change to occur sometime in the next 5 to 7 days.  Everything should remain online until someone at Comcast gets around to updating my account.


Learn Na'vi Mobile App - Now Available