Ok, this is a bit of a techie article that will interest only a small handful of the people who view this site. Basically during an afternoon coffee-break (and a quick read of the WOS forums), I got to thinking about the influx of large capacity storage solutions for “old” computers such as the ZX Spectrum. Well, specifically the ZX Spectrum as that’s where my interests lie. Yes, this is the kind of thing I sometimes think about when I have a free moment. Sometimes it’s naked women, but this time it was old computers.
One of the most requested features for DivIDE firmware is the ability to write files. Sure loading files is great an’ all, but it would be nice to be able to save files onto, say, our FAT16 partition from the ZX. The problem is how do we store all of the extra information that comes with a ZX-specific file? It’s a problem that’s been discussed at length on the forums, and yet a good solution has yet to be found. The formats that currently exist (.TAP for example) make use of checksums and the like, making creating or updating files a bind. Either that or they are designed to represent a part of a disc image, and so store the actual file size in the header, while padding out the actual file size to fill a virtual “sector” (I’m looking at you Hobeta format). Other solutions included “ZXLFN”, which stores the additional information in the filename… not ideal, tho’ quite easy to use.
But the Hobeta format actually makes some sense. You can tell the file type from the extension ($b is a BASIC file, $c is code, $m is rather pointless in my opinion, but it represents music). Then the header at the start of the file gives you all the information you need, if a little jumbled up, along with a lot of information you don’t need.
So let’s take Hobeta as the basis for our file storage. All of that information in the header that’s not needed we can remove. For example we can take the filename from the, well… filename. A file called “MYFILE.$B” would be used as a file called “MYFILE” on the ZX – the fact that it’s a BASIC program can be gleaned from the $B. Checksums, while useful, are a little overrated – if the data gets corrupt (perhaps due to a trashed disc) you will soon know about it because you won’t be able to see the file. And why would we need to store the length of the file? We can work that out from, you know, the length of the file? This just leaves us with one thing to store in addition to the data in the file – the start. This would be the LOAD address of a block of code, or the start LINE of some BASIC. A two-byte overhead, with no CRC to stop us from adding to the file.
So that’s my proposed solution to the whole “saving to a large-capacity device” problem. A simple, new format with little overhead. The filetype (after the last “.” in the filename) denotes the type of file we’re dealing with. The first two bytes, in little-endian format, give additional information depending on the filetype. A screen$ would be recognised by having a CODE address of 16384, and the total filesize being 6146 or 6914. An optional extra would be to have a number prior to the last “.” in the filename to denote the loading order of files with the same name – that’s sensible as the sorting could then be alphabetical.
File types available would be:
- .$$B : BASIC file, first two bytes denote start LINE number.
- .$$C : Code file, first two bytes denote LOAD address.
- .$$? : Any other types we think we need.
So there you go. It’s not something I expect anyone to pay attention to, I’m just getting the idea out there.
18 Comments
I was just checking my blog to write an article about Vampires and I realised I’d written this a while ago but not actually posted it. So here it is.
I disagree that checksums are unnecessary baggage. Suppose that the corrupted file is the last level of a difficult multiloading game, or anything you don’t see very often: you could easily go for years without knowing it was wrong, and perhaps distributing your apparently-valid copy to others.
Please, please, please don’t use $ signs in the extension. They’ll make life unnecessarily painful for anyone using a proper shell which uses $ to designate variable names.
Other than that, fairly sane, but I still don’t see what the problem is with doing one simple “make” to create a tape image.
The other issue is how to deal with LOAD “” or equivalent.
Ah yep, agree with the $ thing – I just took that as an idea from the Hobeta format.
I should perhaps point out tho’ that this wouldn’t be a replacement of .tap files, and certainly not for distributing games, etc — my makefiles all produce .taps for my work.
When I wrote it I was thinking more of how to save files from a ZX onto a HDD. For example, the ZX command SAVE “file” SCREEN$… yes it could easily be saved to a .tap, but that makes editing tricky due to the header information, CRC. I think for other formats the size, CRC (or checksum, more accurately), etc are great.
From a ZX perspective, the only thing needed in order to use a file is the load/start address.
As a side-note, I should perhaps check how this is done in esxdos… they have implemented file saving and I think it was to “raw” files (that’s to say, not .taps). I’ll have a go when I get a chance.
Just reading through this again (I wrote it a while back) I would like to disagree with my comment in the article about “An optional extra would be to have a number prior to the last “.” in the filename to denote the loading order of files”.
That’s a fairly sucky way to do it, but I don’t really have a suggestion other than “alphabetical” for how to do LOAD”".
Inventing America? What about ZXLFN, pal! ;) I think Baze defined it quite good.
Well, now I see you’ve written something about ZXLFN. I think it’s not optimal, but the best solution how to store extra information on the guest filesystem. If EsxDOS saved such files instead of raww format, it would be nice. It has many advantages. No need for converting those TAP/TZX/whatever stuff. Of coure, emulators should support it, too. A good ground for a new file format?
…but yeah, the loading order is a problem, when LOAD “”CODE in Basic loaders was persent, it could do nasty things.
Although I’m not tech-savy and even feel afraid to participate with my view, here’s my idea:
1-The DivIDE firmware should allow the Spectrum to treat IBM formatted HDs (or any other media) just like we read/write disks on ms-dos. We should be able to format, read, delete, copy from one kind of media to the other, copy from one tape to the other, rename files (in certain programs), etc.
2-Instead of tape files or image files of some sort, maybe each virtual tape/disk could be stored inside a folder. The name of the folder would be the name of the tape/disk.
3-If load”" was used, the DivIDE would check if external media was connected, list the ‘tape/disk’ folders, and ask for one to be selected in order to load it. And if no HD, flash or whatever was detected, load”" would simply wait for real tape input.
4-All the files that are usually inside a .tap or .tzx file would be inside their own folder. This would give us the option to freely edit the files.
5-Any tape/disk information that’s absolutely necessary (order of files inside a tape, for example) could be stored together with the folder name, after the name, or by using a .div or .nfo file that would be used solely by the DivIDE O.S., not to be considered part of that tape or disk. If any files inside that folder were modified, the .div file would be automatically updated.
/me hides from Masters of Spectrum
While I can see the advantages of something like ZXLFN, I’m against storing too much information in the filename. It’s something that can easily get garbled (especially when moving to an old system that uses 8.3 filenames).
My thoughts came from me thinking how I would try to implement a simple saving system, either as part of a DivIDE firmware or from a possible OS-style application. I just figured that much of the information stored in header is due to the fact that from tape we load a header then a block, sequentially. From disc we don’t need to do this.
As for the idea of .div/.nfo, I actually thought about support for a kind of “extended” descript.ion (basically the same thing), but that requires additional work for the interface/device.
As for LOAD”", I was thinking yesterday… unless you’re using an MB02+, you can’t normally do LOAD”" from a disc device on the ZX, so why try to support it? If you wanted to do that for some released software (for example), then you’d make a .tap of it anyway.
Maybe.
I think Bruno’s idea is not bad! It’s something like plugins in Total Commander, you just press enter on the file (TAP file in our case) and it would open its contents. Hmm, something like Spectrum Commander for DOS. Exactly.
Erm, Spectrum Navigator, of course :)
Regarding the implementation in ESXDOS, it’s kind of like +3DOS, files saved have a header which contains all the info in the original speccy header, plus room for some esxdos specific bits. However, you can load RAW screen$ and code files sort of transparentely (if you dont specify load address for code files it defaults to summit i don’t remember).
Since the filesystems are modular in esxdos, one can write a ZXLFN enabled fat driver if it is needed.
greetings from africa :)
lordcoxis
coxis: That’s quite a sensible approach I guess, basing the format on something that already exists. Of course a raw file could be loaded, I’m not sure if I’d load a “headerless” file to a standard address tho’ – perhaps issue an error instead? Dunno. It’d be fun to see and “R” error loading from disc ;p
Years ago we had the choice to use the PlusD, Opus Discovery, FDD3000, the +3, etc. All these options allowed us to access disks and perform standard disk operations without much effort.
Now that we have the DivIDE, we should be able to do the same thing, but with Flash cards, HDs, etc. We shouldn’t be limited to a read-only solution that can only launch image files.
I remember Rich Mellor mentioning that future DivIDE firmware versions will allow writing to FAT32, but I wonder what kind of options we’ll have when that happens.
Anyway, I pre-ordered my DivIDE and I’m eager to receive it.
ResiDOS uses a similar system to esxdos. SAVEd files start with a standard +3DOS header (a 128-byte header including all the normal tape header information, a signature and a checksum). This doesn’t seem very much overhead to me and is a fairly good solution. The next release of ResiDOS (which I hope to get out on Sat 31st March) will also allow headerless files to be LOADed as CODE files, defaulting to a start address of 32768, IIRC.
Well the latest DivIDE Plus comes pre-installed with several firmwares, which can be switched between with a simple OUT 23,x command. Included in the firmware is ResiDOS v2.00 which allows reading and writing to FAT 16 partitions. Now all we need is to encourage Garry Lancaster to work on a FAT 32 version !!