A solution for lost recordings, but I need a little help...

[Weston]

Hello. I'm working on developing my own solution for recovering lost recordings... like many people with this issue, I'm using my Recorder 2+ to record video from my race car and the recorder lost power before finalizing the file. I had two recordings on the same card that were lost this way, and they both show up as 0 byte files but the .thm thumbnail files are ok. One of them should have caught some pretty good action, and even though there was dirt on the camera lense, I feel it's worth the effort to recover the files. Besides, it would be nice to have a solution ready to go the next time this problem occurs...

By directly accessing the SD card and searching for part of the file headers (HEX 00 00 00 1C 66 74 79 70 4D 53 4E 56 00), I have been able to crudely recover the video files (plus a lot of garbage on the end), but they have some problems... the only thing that I can get to play them is "mplayer" under Linux, but I frequently have to skip over problem spots in the file so that it doesn't crash, and it doesn't even attempt sound. I haven't dug into MP4 file format specs yet, and I'm hoping to find a shortcut, because it seems that this should be a simple solution. I mostly just need to figure out if this corruption I'm seeing is caused by simple FAT16 file fragmentation, incomplete MP4 header or stream information, or both...

What I need to know is...

1) Should I be digging through FAT and trying to deal with fragmentation, or should that be a non-issue in this case? The files show up as 0 bytes when I mount the SD card, but the disk space seems to be somewhat allocated for them, so I'm thinking that maybe the files were written in order rather than fragmenting, so maybe I don't have to worry about adding code to deal with FAT... I'm just trying to keep this as simple as possible now and address the main cause of the file corruption first.

2) What exactly is the Recorder 2 finalizing when you properly stop the recording? What MP4 header or other fields is it writing? It's my understanding that this format should be fairly resistant to corruption and incomplete writes, but it still seems to need something more than I have...


Thanks for any info! I'd be happy to share my completed program with everyone if I can get it working reasonably well...

[nekenk]

Hi Weston,

I found a solution only to recover the video stream, I explained the method here :
http://forums.3ivx.com/index.php?showtopic=86104&st=0&gopid=421222&#entry421222

the main problem is that track MP4 atoms for video and audio are written on the SD card at the end of the recording (like the start sector and the length of the file by the way). the free software mp4creator is able to recover the video from raw data but I didn't find anything for the audio stream.
If you are more lucky than me, please, tell me.

PS : my SD card was not fragmented at all but some THM files were inside MP4 files (according to sectors number)

Nekenk.

[Weston]

Thanks for the info. It would seem that the ideal solution is to set the length on that MDAT atom and then add a MOOV atom after it, which would resemble what we see in a completed recording, but constructing a MOOV from scratch is easier said than done. I tried the quick and dirty approach of just copying a MOOV from a good recording... it did make most video players at least start to play the video, but the length was limited to that of the good recording, and there were still some problems. I'll need to do some reading if I persue that method...

I have had some success with making my program add an ESDS atom (extracted from a good recording) after the FTYP atom, and also set the length of the MDAT atom (MDAT length = file size - offset of MDAT atom). A length of zero seems to be fine when it's the last atom in the file though. mplayer in Linux plays it the same as before the changes (ie it plays it, but not well), and mp4creator will be able to somewhat make sense of it after the changes... enough to make a new file that most MP4 players can play. I recovered two videos this way and both have no sound, but mostly intact video (there are some distortions). The second file is quite large (1.5mb, and most of that should actually contain video) and I can't seem to recover more the first 477mb of the MDAT data... if I don't truncate that atom to about 477mb, mp4creator will either have a fatal floating point error or a seg fault. If I skip beyond that problem area of the file, I can tell that there is video, but it doesn't display right at all... I can see elements of the video at the top of the screen, but that's it. I suspect that there's some corruption around there that causes something to get out of sequence. Maybe it's a file fragment, maybe my SD card lost it's connection (it wasn't enturely secure in the recorder when I came off of the track), or maybe it's just plain corruption. The Windows defrag program says that my SD card isn't fragmented at all, but I wonder if it's not even looking at the 0kb files since they probably don't have complete FAT data. Maybe I'll read up on FAT16 and see what I can find...

What I'm tempted to do at this point is read up on the format of the video and audio streams and see if I can locate and extract them from the file, while skipping the unrecognizable junk that seems to be giving mplayer and mp4creator crashes and other problems.

[nekenk]

Maybe I'll read up on FAT16 and see what I can find...

I'm not a FAT16 or 32 expert but the free space of the SD card with corrupted files from NR2+ takes into account the corrupted files. So, I think somewhere in the FAT there is the information that a sector is not free or something like that.
For your problem with your long file, I would say that there is a sector that does not belong to your MP4 file (a thm file or an old file...)

Today, I'm working on a solution to avoid that kind of stuff : electronic DIY to wait 5-10 seconds after the power off :
switch on -> camera on ->  5 sec later NR2+ on
switch off -> camera off (NR2+ writes correctly the file) ->  5 sec later NR2+ off

The solution is very cheap with 2 transistors, some resithors and capacities,
I will write something on it if I have enough time.

Nekenk.

[924RACR]

Have you been able to make any progress on creating such a utility?

More specifically, I'm also having trouble actually pulling the 0kb file off the card - what utility are you guys using?

I have, I believe, the video from a very big, unique race (2007 American Road Race of Champions, came in 5th, good action on-track and really want to recover the video)... long-term solution will of course be either to somehow improve power supply or quit screwing with the R2 and go to a more robust solution better designed for this application.

[Ex-Navy]

www.chasecam.com MPEG2 recorder

[924RACR]

I know, but dayum spendy - even the bare recorder, heck, that's nearly 3 tires!!!