@navi_wotd. Na'vi word of the day on Twitter.

Started by Puvomun, March 27, 2011, 10:23:03 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

omängum fra'uti

It looks like the update script got confused by my lack of putting a date on the final column of the spreadsheet, and so it didn't get that there was a recording made for today.
Ftxey lu nga tokx ftxey lu nga tirea? Lu oe tìkeftxo.
Listen to my Na'vi Lessons podcast!

Puvomun

Aye, I use that info to create the link. I'll add the date and re-upload things, then it should be fine. Can you please add the date in your file as well? It will be downloaded again tomorrow, erasing my edit of the local copy.
Krr a lì'fya lam sraw, may' frivìp utralit.

Ngopyu ayvurä.

omängum fra'uti

The date I just add in so I know the time frame, so I can plan out when to record a new set.  I'd prefer if you didn't rely on it being there.  Only the first 5 columns have I been updating with any regularity.  The 6th and 7th are on an as-I-get-around-to-it interval.  However, when neither is set it changes the formatting slightly, in that the 5th column is now the end of line, and so has an EOF on it.  (I noticed before I recorded the first few, you had an extra new line in your data captured from those lines, so your parsing of the file isn't splitting off the final line ending.)

I've put a placeholder value in the remainder of the rows, so if I don't get it updated it won't cause parsing errors again.
Ftxey lu nga tokx ftxey lu nga tirea? Lu oe tìkeftxo.
Listen to my Na'vi Lessons podcast!

Puvomun

Thanks, I'll change things on my end so you can leave the date-part out. Might take a few days, but I am sure that's not a problem.
Krr a lì'fya lam sraw, may' frivìp utralit.

Ngopyu ayvurä.

omängum fra'uti

That should be fine, right now I've got about a week's worth of dates filled in ahead, but have been doing the new words the night before on account of being a bit sick.  I keep hoping I'll feel better and not have to record them through a stuffed up nose.  So far it hasn't seemed to be a huge problem as far as the recording quality goes.
Ftxey lu nga tokx ftxey lu nga tirea? Lu oe tìkeftxo.
Listen to my Na'vi Lessons podcast!

Puvomun

I hope you get well soon. Being sick is hardly any fun.
Krr a lì'fya lam sraw, may' frivìp utralit.

Ngopyu ayvurä.

omängum fra'uti

Looks like something is broken again.  At http://naviwotd.nlpagan.net there are no words currently.
Ftxey lu nga tokx ftxey lu nga tirea? Lu oe tìkeftxo.
Listen to my Na'vi Lessons podcast!

Puvomun

#47
Quote from: omängum fra'uti on June 11, 2011, 01:14:53 PM
Looks like something is broken again.  At http://naviwotd.nlpagan.net there are no words currently.
ARGH! Fixing. I know what happened.

Thanks for the quick note!

Edit: fixed. Damn, one typo in a last small fixup.    :'(
Krr a lì'fya lam sraw, may' frivìp utralit.

Ngopyu ayvurä.

omängum fra'uti

I tend to check my phone first thing in the morning to make sure it has the new word.  If it doesn't, I know something in the process broke and I can figure out why, so I'm pretty quick to notice. :)

It occurred to me today how complex this all is to get the WoTD banner updated...  Off the top of my head I can think of 10 seperate components which are involved in the full pipeline from the source data all the way to the WoTD sigbars.  Fortunately there's only 2 parts that can break taking the whole thing out (Unfortunately one of those two broke a few days ago, heh) and they are probably the least fragile components, the rest would just keep it from updating.
Ftxey lu nga tokx ftxey lu nga tirea? Lu oe tìkeftxo.
Listen to my Na'vi Lessons podcast!

Puvomun

Quote from: omängum fra'uti on June 12, 2011, 03:09:17 AM
I tend to check my phone first thing in the morning to make sure it has the new word.  If it doesn't, I know something in the process broke and I can figure out why, so I'm pretty quick to notice. :)

It occurred to me today how complex this all is to get the WoTD banner updated...  Off the top of my head I can think of 10 seperate components which are involved in the full pipeline from the source data all the way to the WoTD sigbars.  Fortunately there's only 2 parts that can break taking the whole thing out (Unfortunately one of those two broke a few days ago, heh) and they are probably the least fragile components, the rest would just keep it from updating.

Part of the problem, i am sure, is that it started with an idea (the new twitter feed) and things got bolted on top and bottom. I plan to rework the part on my end, so that is less complex. The way to get the link-check in now is too complicated for my taste, I plan on sticking most of that in a program. The bash script is getting way out of hand for all this.

I'll work on a testset that cannot break the current setup, and will make sure the part I get and send will remain as they are.
Krr a lì'fya lam sraw, may' frivìp utralit.

Ngopyu ayvurä.

Swoka Ikran

Quote from: omängum fra'uti on June 12, 2011, 03:09:17 AM
It occurred to me today how complex this all is to get the WoTD banner updated...  Off the top of my head I can think of 10 seperate components which are involved in the full pipeline from the source data all the way to the WoTD sigbars.  Fortunately there's only 2 parts that can break taking the whole thing out (Unfortunately one of those two broke a few days ago, heh) and they are probably the least fragile components.
I'm guessing the "one of those two" was skxawng.lu's server. I still find it ironic that the IPv6 support quit on IPv6 day...

Also, I'm curious as to what components I'm missing. I can only think of Puvomun's CSV+shell script, Twitter, AppSpot server, the script appspot is running, skxawng.lu server, PHP program that produces the images, and my SQL database (one of the tables in there is a cache for WoTD).
2010 was the year of the Na'vi.Vivar 'ivong Na'vi!


 
Avatray | NWOTD Sigbars | Sacred's Sigbar Tool | My collection of Avatar merchandise

omängum fra'uti

The components I'd come up with were 3 parts...

1. Data sets (2)
WoTD base data, Google Docs WoTD data
(These two are merged for the WoTD updates)
2. Data conduits (6)
Puvomun's server, twitter, app engine, Google Docs, skxawng.lu, naviwotd.nlpagan.net
3. Scripts (3)
Puvomun's shell script (Shell, really?  Ouch!), app engine script, sig bar generating PHP

Ok so that's 11...  I think I wasn't counting Google Docs before.

Povumon, if you're re-working it, at some point I'd like to move the app engine code to no longer have to poll twitter, and instead get pinged directly from your script when it's done updating the WoTD.  It would just be a direct URL access to a special URL I'd give you.
Ftxey lu nga tokx ftxey lu nga tirea? Lu oe tìkeftxo.
Listen to my Na'vi Lessons podcast!

Puvomun

Quote from: omängum fra'uti on June 12, 2011, 11:30:37 AM
Puvomun's shell script (Shell, really?  Ouch!), app engine script, sig bar generating PHP

Povumon, if you're re-working it, at some point I'd like to move the app engine code to no longer have to poll twitter, and instead get pinged directly from your script when it's done updating the WoTD.  It would just be a direct URL access to a special URL I'd give you.

Yes, it is a shell script in Bash. That is why I desperately need to rework it, you do NOT want to see this beast. (*I* nearly don't want to see it!!)
If you have the URL, throw it my way, I can stick it in the current script at the end too, so you're notified the easy way.
Krr a lì'fya lam sraw, may' frivìp utralit.

Ngopyu ayvurä.

Swoka Ikran

#53
Quote from: omängum fra'uti on June 12, 2011, 11:30:37 AM
The components I'd come up with were 3 parts...

1. Data sets (2)
WoTD base data, Google Docs WoTD data
(These two are merged for the WoTD updates)
2. Data conduits (6)
Puvomun's server, twitter, app engine, Google Docs, skxawng.lu, naviwotd.nlpagan.net
3. Scripts (3)
Puvomun's shell script (Shell, really?  Ouch!), app engine script, sig bar generating PHP

Ok so that's 11...  I think I wasn't counting Google Docs before.
Dang...didn't realize there were so many parts to it.

Using the above the data path for my app is:

* Shell script merges data sets, posts to twitter.
* App engine+Script reads twitter, produces key=value output.
* Skxawng.lu runs PHP which reads app engine output and produces a PNG.

Three different systems, using 3 different programming languages, located in 3 different parts of the world. (4 systems if you count Twitter...)

EDIT: Fixed typo & added something.
2010 was the year of the Na'vi.Vivar 'ivong Na'vi!


 
Avatray | NWOTD Sigbars | Sacred's Sigbar Tool | My collection of Avatar merchandise

Sіr. Ηaxalot

Quote from: Swoka Ikran on June 12, 2011, 11:01:02 AM
Quote from: omängum fra'uti on June 12, 2011, 03:09:17 AM
It occurred to me today how complex this all is to get the WoTD banner updated...  Off the top of my head I can think of 10 seperate components which are involved in the full pipeline from the source data all the way to the WoTD sigbars.  Fortunately there's only 2 parts that can break taking the whole thing out (Unfortunately one of those two broke a few days ago, heh) and they are probably the least fragile components.
I'm guessing the "one of those two" was skxawng.lu's server. I still find it ironic that the IPv6 support quit on IPv6 day...

Also, I'm curious as to what components I'm missing. I can only think of Puvomun's CSV+shell script, Twitter, AppSpot server, the script appspot is running, skxawng.lu server, PHP program that produces the images, and my SQL database (one of the tables in there is a cache for WoTD).

Heh, ironic indeed. Actually, IPv6 have broke a few times earlier, but then I have rebooted the server to fix it, this time I completly killed IPv6 support to stop the problems.. I'm probably moving the server to another location this summer where IPv6 wont be available at all, so it wont be much difference.

Swoka Ikran

#55
Quote from: Sir. Haxalot on June 12, 2011, 04:45:56 PM
Heh, ironic indeed. Actually, IPv6 have broke a few times earlier, but then I have rebooted the server to fix it, this time I completly killed IPv6 support to stop the problems.. I'm probably moving the server to another location this summer where IPv6 wont be available at all, so it wont be much difference.
The good news is it's fixed. IPv6 wasn't all that important anyway...seeing that 98+% of the traffic was IPv4 according my access log.

Also, why would the server be moving?
2010 was the year of the Na'vi.Vivar 'ivong Na'vi!


 
Avatray | NWOTD Sigbars | Sacred's Sigbar Tool | My collection of Avatar merchandise

Puvomun

#56
Right. One more point of failure, it seems:

While testing the new script:

access failure on: http://api.twitter.com/1/statuses/update.json
to automatically wait for a connect, use -hold.
paul@Kelutral:~/src/navi_wotd/dev$ ping -c1 http://api.twitter.com
ping: unknown host http://api.twitter.com


Was something weird in bash. Fixed
Krr a lì'fya lam sraw, may' frivìp utralit.

Ngopyu ayvurä.

Plumps

Ma mesutan a-srung-susi,

I have to say this project is one of the best I've seen around here. Wonderful idea and although I'm not a twitterer ... or whatever you call the person who tweets :o I keep track of these. Amazing! Karma up-up-up! :D

Sіr. Ηaxalot

Quote from: Swoka Ikran on June 12, 2011, 05:31:03 PM

Also, why would the server be moving?

Because it looks likely that I will be getting 100/100 FTTH connection at home. If that will be the case I will move the server here which will grant me full control an ability to guarantee at least 80/80 compared to the 10/10 I have guaranteed now.

Swoka Ikran

Quote from: Sir. Haxalot on June 14, 2011, 07:23:28 PM
Quote from: Swoka Ikran on June 12, 2011, 05:31:03 PM

Also, why would the server be moving?

Because it looks likely that I will be getting 100/100 FTTH connection at home. If that will be the case I will move the server here which will grant me full control an ability to guarantee at least 80/80 compared to the 10/10 I have guaranteed now.
Oooh. That's a nice connection. Speed improvement will be nice :)

As for moving, where has the thing been all this time? Data center?
2010 was the year of the Na'vi.Vivar 'ivong Na'vi!


 
Avatray | NWOTD Sigbars | Sacred's Sigbar Tool | My collection of Avatar merchandise