A cautionary tale

[heyrick]

Hi,

I spent part of this evening making a few tweaks to the view-scheduled stuff. Nothing difficult, just:
1. "Today" is displayed in white (to make it clear)
2. The date is yyyy/mm/dd (because for a European, US dates can be extremely irritating - 03/04/2011 is the third of April, right? <sigh>). At least descending precedence is clear and unambiguous.

This wasn't a simple "make viewschedule" or anything. No, I learn all of this random code is stuffed into a huge library called "libneux"...
...which seemed to require pretty much everything else to be built, again. I think the OSD devs (what's that, all four of us?) will cheer if greyback succeeds in making it work with a build system that doesn't have a penchant for wiping and rebuilding everything.

Besides, all of this in one big library? <raises eyebrow>

Right. Library built, copied, transferred to some scratch space on the OSD.

Don't chuck the old, just rename it in case the new code doesn't work correctly.
mv libneux.so.1.0.0 libneux.orig.so.1.0.0
(no path - trust me, you do NOT want to do this)

Segmentation fault
Oh, and the thing has stiffed.

Okay, no big. Maybe something has it in use (perhaps main-menu keeps it active?) and Linux wasn't smart enough to say "no, dummy, this one is in use".

Reboot.

It hangs on the "Neuros OSD - The power of open" (or whatever the last message actually says).

It just won't go further.

So I use the emergency escape route into Arizona (when the LED fast-blinks, press 'home' and then 'enter') and I tell it to just set up OSDng again. Probably the mini_fo overlay is cocked up, that's the only r/w part.
Needs internet. Says check the website for offline instructions, which are "Coming soon". Hmmm...

So I lug the netbook, TV capture card, and the OSD into the living room and plug it into the livebox.

Suffice to say the OSDng setup doesn't appear to cope with partial files. So I have to look to see where it thinks it is writing to, and then delete them. This involves lots of power-cycling to force a reboot as setup failure stiffs the thing (so no telnet access either to do it 'kindly').

Finally it is all installed, set up, working again. Reconfigured from scratch. And a waste of several hours when I could have been watching zombies go "grrr argh".


Lessons:
1. Don't dick with important looking files on a live filesystem.
1. Don't dick with important looking files on a live filesystem.
1. Don't dick with important looking files on a live filesystem. [three times for emphasis, you listening, Rick, you numpty!]
2. Don't expect Linux to be smart enough to prevent you from moving an open file (jeez, the lowly BBC Micro could handle that!).
3. Modifications should be built on a netbooted system, or something. Leave the internal filesystem WELL alone.
4. If I/greyback/we/anybody rebuilds new firmware and implements read/write functionality, we'll need to consider a way to recover from such a filesystem corruption. Yes, in hindsight, renaming an active library (with a load of symlinks pointing to it) was epic stupid, but then I am surprised that the OS didn't say "duh!". It just went ahead and trashed itself. Useful.


Now I'm gonna watch Fractale and eat chocolate to commiserate my stupidity.


Best wishes,

Rick.

PS: Every cloud has a silver lining, right?
       Helluva way to clear the cruft out of my schedule!

[ChadV]

Lessons:
1. Don't dick with important looking files on a live filesystem.
1. Don't dick with important looking files on a live filesystem.
1. Don't dick with important looking files on a live filesystem. [three times for emphasis, you listening, Rick, you numpty!]

Agreed.  I have a web/mailserver right now running without a kernel because I was dinking around.  If the power goes out before I get the new one ready, I'm hosed.  >.>

[greyback]

Ouch guys, and I thought I was having a bad computer-day

Rick: I am a tad surprised how badly your experience was though. Sure if the file is open at the time, replacing it will cause a whole world of hurt, but corrupting the whole thing? Did mini_fo go nuts or something?

In theory with the mini_fo overlay system, if something screws up, just deleting the overlay will restore it to normal. But I guess if you can't boot the system to delete the overlay, you're screwed - but having it on a CF card would be nice, just do a boot without CF inserted, get telnet going, plug in card & delete away. One slightly awkward option.

In some better news, I'm having success with the OSD and the buildroot build system at last. I have a working EABI kernel, and toolchain with gcc4.3.3. Buildroot gives me lots of applications and libraries for free, I just need to customize the system for the OSD's specifics.

I chose buildroot because it's been around a long time, is compact and straight-forward (compared to the monsterous beast that is OpenEmbedded), and there was talk in their mailing list about it being able to generate a basic SDK automatically. That's not there yet, but I'm watching closely.

I still have no idea if the closed Ingenient modules will work, they might need a recompile with the new toolchain. But one step at a time.

When it looks to be reasonable shape, I'll document it all, upload my files and you can all join in
-G

P.S. I'm not hugely interested in incorporating all Neuros' stuff into this. And unfortunately anything built with this system will not run on the original firmware. So I dunno how useful this will actually be. I want to make a better UI, have playback & record work via command line, have a proper scheduler. Make it easy to script and for people to hack at. I'll try to make it as easy as possible. Maybe breathe a little life into what I still think is a great little piece of hardware.

[heyrick]

Rick: I am a tad surprised how badly your experience was though. Sure if the file is open at the time, replacing it will cause a whole world of hurt, but corrupting the whole thing? Did mini_fo go nuts or something?

If it went nuts, it did it silently. But, then, isn't that the way it happens?

Thing is, OSDng couldn't boot as it needed the root_rw to be mounted, and without that working... Sadly the setup isn't as simple as deleting it and letting the system recover. I perhaps could have done it in less hassle if I knew how (makefs, something like that?), but I did not.
Then... then came into play the problem with the OSDng setup. It only seemed to want to work correctly when there was no OSDng folder on the CF. Hmmm...

Anyway, thank God for the drop-to-Arizona option. It allowed me to get into the system (gee, and I even remembered the default login password!). If you see the time of my posting, you'll understand why I didn't want to flash in the recovery Arizona, and then upgrade to OSDng from scratch. Flashing this thing takes forever. Doing it twice at 4am, would be stressful.

In theory with the mini_fo overlay system, if something screws up, just deleting the overlay will restore it to normal.

I remembered you said that. Thing is, what happens if you delete the overlay and then something goes looking for it?


If it is any help, I tried to get "main-menu" running from the command line. It reported "Bus error". <shrug>

Also, I was able to telnet into the stalled boot, it looked as if most stuff had started. The dmesg report ended after the video codecs reporting, the G735 (G7-something). Could it not set up the display? If I remember (I'm not in my room) it has a bunch of dimension stuff following. But not in this case. Sadly there was no "ERROR <blah>" in the report.

But I guess if you can't boot the system to delete the overlay, you're screwed

As I said, telnet did work. That was about the only thing that worked. But getting it to init pretty much needed a reinstall of the OSDng "read/write" system.

but having it on a CF card would be nice, just do a boot without CF inserted,

I think in whatever we do with the OSD, it should - if at all possible - be self-repairing. Would it be viable to run a check each boot? Start-up is slow enough as it is. Perhaps some code in Flash where booting without a CF card will pop up a message saying:

  You need a Compact Flash card inserted to use this system.

  Please insert CF card and press ENTER to continue booting.

  For recovery, please insert CF card and press the HOME key.

I'm not sure how this would be handled on the OSD1.5...

I have a working EABI kernel, and toolchain with gcc4.3.3. Buildroot gives me lots of applications and libraries for free, I just need to customize the system for the OSD's specifics.

Nice.

I still have no idea if the closed Ingenient modules will work, they might need a recompile with the new toolchain. But one step at a time.

Does Neuros have the Ingenient source code? The build_helper.sh script implies that they might. If so, this could be a possibility if things do not work with the newer kernel, for Ingenient is no longer, it is now an Indian company called "Sasken Inc.".

P.S. I'm not hugely interested in incorporating all Neuros' stuff into this.

Hmm... The organisation seems a little... difficult. I come from a background where supervisor-level code (stuff that talks to hardware), things that need to remain active across task swap-outs, and shared libraries are implemented as modules. Everything else is "the program".
Take for example the YouTube fetcher. Why is the YouTube module not a part of the YouTube program? And why is the Linux system polluted with stuff like:
  blah.so  ->
  blah.so.1 ->
  blah.so.1.0 ->
all of which are symlinks to blah.so.1.0.0 ?

So I dunno how useful this will actually be.

Depends upon the feature set. Give people a reason to upgrade, many will.

I want to make a better UI,

Yes. There are many little tweaks I would like to make (in lieu of rip-and-retry). But given how much of it ends up in libneux <cough><cough>

have playback & record work via command line,

There is an option in the headers for writing video as:
  NONE
  MP4
  ASF
  AVI

Does the AVI one actually work? I don't have any useful tools to directly convert MP4 to AVI by changing the wrapper. AVIDemux can sort-of do it, but it screws up keyframes.

There's lots of stuff I have which can understand H.263 MPEG4-SP inside AVI files.
Most of which can't tell an MP4 file from a TXT file...

have a proper scheduler.

There appear to be "quality" settings (Economic/Normal/Fine/Superfine) which don't appear to be usable from the UI. Do these work? It might be nice to have access to it so I can trade off bitrate (and hence filesize) with quality. I record stuff that I don't need high quality in 768kbit, but this might trigger "economic". Would things be nicer looking at 768kbit/sfine?

.../linux-r3-main-app/neuros-qt4-libs/neux/settings/nvrecorddata.h

Make it easy to script and for people to hack at.

Yes, I think scripting support could add a lot to the OSD. It may run a little more slowly, but if tweaking some readable script allows for easy changes, then it'd be worth it.

Maybe breathe a little life into what I still think is a great little piece of hardware.

"Still think"?

Tell you what, read this guy's blog -> http://worldcadaccess.typepad.com/gizmos/vmate_from_sandisk/
and you'll see that the OSD is so far ahead of the competition that it is ridiculous.
My own experience - little Chinese "iPodRecorder". Based around the DM320/TVP5150/AIC23 (sound familiar?). Clumsy bitmapped (damn-ugly!) UI. Tragic record timer (you need to stop at midnight and put the other part from midnight the following day, at least it records in one lump). Files are broken into hour-long chunks. NO control over video quality (seems to be around 800kbit VBR). CANNOT PLAY BACK ITS OWN RECORDINGS! (jittery, and they're only about 800kbit!). Prone to randomly stop recording. Sometimes doesn't start scheduled recordings. Damn thing is hardwired to output an NTSC signal (hence I would only see it in black and white, blog pictures are in colour as the computer does NTSC but the telly doesn't).

The OSD is nice. It does its job. It does it well. That doesn't mean it can't be improved, but, you know, compared to other stuff I've heard of, it is far better.

[ so let's make it better! ]


Best wishes,

Rick.

[heyrick]

Small suggestion: EPG support. Well, why not? It has network access...

This will be problematic as the variety of countries, channels, and so on.

For British people, however (and possibly Ireland too, G.), take a look at:
  http://xmltv.radiotimes.com/xmltv/channels.dat

Those are the available channels.

I will, later this evening, watch a dopey low-budget flick on "Horror Channel". This is in the list as:
2476|The Horror Channel

So you'll get the schedules for this channel from:
  http://xmltv.radiotimes.com/xmltv/2476.dat

Replace the number with the channel you're interested in.

Entries look like this:
Satan's School for Girls~~~2000~Christopher Leitch~Beth Hammersmith/Karen Oxford*Shannen Doherty|The Dean*Kate Jackson|Alison Kingsley*Julie Benz|Mark Lantch*Daniel Cosgrove~false~true~false~false~false~false~false~false~2~12~Film~A girl enrols at college to probe her sister's apparent suicide - but finds herself forced to bring her natural talent for magic into play as she battles a satanic cult. Horror, starring Shannen Doherty, Kate Jackson and Julie Benz.~false~19/02/2011~22:55~00:50~115

(see? I told you it was a dopey low-budget film!)

How to interpret this stuff is described here:
  http://www.birtles.org.uk/phpbb3/viewtopic.php?f=5&t=245


Best wishes,

Rick.

[greyback]

Small suggestion: EPG support. Well, why not? It has network access...

Yes, this is on my list too. The XMLTV project does need perl, which is a big of a space-hog, but this project plus a nice UI would be sooo good to have on the OSD.
-G