RetroNAS

[edit]
This has grown legs and started to walk. Project is here if you want to test it:
https://github.com/danmons/retronas/wiki
[/edit]

OCAU Retro Crew, please give me your knowledge, thoughts and opinions.

I want to build a "Retro NAS" (Network Attached Storage) device installation tool. The goal is to be a cheap system you can load up with minimal technical skill (simple menu-driven installer) to serve as a universal storage system for as much retro hardware as possible, and give old computers access to the Internet. Targeting a Raspberry Pi at first, but planning on other NAS devices as time goes on. Plug in any storage and share it over a network to any device you can think of.

Goal is simplicity and universal access first. Security is almost non-existent due to very old protocols. Open Source projects are almost mandatory due to this being aimed at an RPi and being distributed for free on GitHub.

For this I need your help. My knowledge of retro is limited, so could you please all share with me the neat little bits of software you use to assist with your old devices - whether they're computer, console or something else.

Here's what I do know exists that I can work with to start. I'm going to try to offer read-only guest access on all protocols where possible, extending with simple username/password based read/write where possible.

* Samba, allows for all SMB protocols right back to NTLM/LanMAN, SMBv1, NetBIOS/NetBEUI, etc. This will cover things like mapped drives and UNCs in Windows9X/2K/XP, MS-DOS with a TCP/IP and a LANMan client, ISO streaming to OpenPS2Loader on a PlayStation 2, smb:// and cifs:// on MacOSX from 10.0 and up, and countless other devices.

* Netatalk, allows for Apple Filer Protocol (AFP) and AppleTalk over TCP. I don't know how far back this goes currently, but I think I can definitely support Classic Mac OS 8 and 9, and all versions of Mac OS X. I don't know enough about pre-TCP AppleTalk/AppleShare (if you do, please comment), but if that works it might open up access for much older Macs (I see random articles about AppleFiler support right back to Apple II GS, which would be amazing if it works).

* EtherDFS, a super lightweight Layer 2 file sharing protocol for MS-DOS 5.x and up, and clones like FreeDOS. Works with simple packet drivers and doesn't need a heavy weight TCP/IP stack for very old computers. Provides a TSR client that lets you mount drive letters in MS-DOS to copy files back and forth like any other second drive or floppy.

* FTP, native clients in tonnes of OSes and devices. I also want a simple FTP client that can push games to devices like modded Classic XBox, XBox360, PS3, etc (maybe web based so you can drive it from your phone?)

* TFTP, aka "Trivial FTP" even lighter weight protocol than FTP. Lots of old computers and embedded hardware have native clients for this.

* NFS for ye olde Unix systems.

* HTTP, again native tool supported by loads of computers old and new, can export all files so they can be downloaded through a simple web browser. Can also be supported by Nintendo3DS HomeBrew to install FBI/CIA files via QR codes (I can generate these QR codes too).

* ps3netsrv - network game loading for modded PS3s

* WebOne HTTPS proxy. This allows very old web browsers to proxy through and reach new websites that only support HTTPS with SSL3.x/TLS1.X and newer. SSL can be downgraded for older machines (think Win95 Internet Explorer and 40bit encryption), or potentially stripped all together for plain text HTTP. Great for downloading from archive.org straight from an old computer. It won't help with CSS rendering, HTML5, etc. But hopefully it can at least be used to get to files on the Internet hiding behind modern HTTPS connections. WebOne advertises support back to Netscape 3.0.4 Gold and IE 1.5.

They're all on my "must have" list to start. I've got some stupid ideas like a web based WAV/audio player that an push audio tape data out the RPi's sound card to tape-based computers, but that will come later. Likewise DHCP/firewalling ideas for making your RPi a locked off retro "bubble" LAN because all of these old protocols are very insecure. But they're way down the list.

From here, what else is missing? I have a huge blind spot around 8 and 16 bit microcomputers especially (C64, Atari 8bit, Atari ST, Amiga, etc). I know simple HTTP/FTP will likely work for those, but if there are better tools that exist for these that allow software copying/streaming more natively, please let me know. Are there, I dunno, things you can Telnet to to download files from an internal "BBS" or something? Native protocols for other systems I don't know about? Even if they're proprietary or you don't know about server side software, please let me know so I can start following breadcrumbs.

Same goes for ANYTHING else. If there's a tool that you think can help, please list it. Doesn't matter how specific or obscure. I might not get to it straight away, but I want to know about all the possibilities.

 

Couple more ideas that came to mind:

* DreamPi - a small project that lets you plug a USB attached analogue dial up modem into your RPi, and then allow dialup clients to use it to connect to Ethernet. Designed to work with Dreamcast, but it works equally well with old Mac and PC computers/laptops that only have dialup options and no Ethernet ports. Potentially MS-DOS too?

* Still thinking about BBS (Bulletin Board System). I've seen people host DOSBox or VM systems running DOS on Linux, and run native MS-DOS BBS tools inside that, forwarding connections in to the DOSBox/VM setup. I could auto-script the DOS VM to mount up the file shares, and then expose that through the BBS. Combine with DreamPi to enable "dial up" from your real MS-DOS machine (or C64 or Amiga or other stuff maybe?) to the BBS on the RPi and pull files down. If there's native Linux BBS server tools, even better.

* SLIP/PPP for serial connected machines that don't have Ethernet ports. Like the DreamPi idea for getting other non-Ethernet physical connections on to the TCP/IP network.

* Sega NAOMI netboot, allows game streaming to Sega NAOMI arcade hardware.

 

Watching with interest. Some notes on my DOS networking experiences:

- I had a short fling with EtherDFS and it seemed very cool. But I had issues with the TP-Link TL-WR802N nano router that I use to get retro machines onto wifi. Turns out it doesn't like raw ethernet. Just something to keep in mind for anyone trying it.

- I currently use mTCP for FTP. In my case I do it somewhat backwards, where the retro machine is the server, and I manage the whole system with FileZilla on a modern PC. Great set of tools which also contains an FTP client of course.

On the hardware side, one of my NAS requirements will be dual NICs. That way my retro machines get to run their wild west protocols, while I still have a secure connection to my modern PC.

 

Feature-wise, a "sub-secure" Wireless Access Point would be good, there are a bunch of Amiga wireless adapters (and Nintendo consoles?) that don't support WPA2 or radio standards that one might find in a modern household WLAN.

Meta-wise, I'm interested in how you'd compose and support the image - with a good traditional OS distribution I think the needs of this thing would be served well by eg. Ansible, but anything from "build.sh" to maintaining your own hermetic container images (eg. Flatpak) would work.

 

Yeah, I'm working with someone else on this who's good at making Internet videos. I've emphasised to him that a much more difficult task of this will be providing good quality documentation and videos.

I'm planning on whipping all of this up over Christmas. WAY harder than the code is spending a year writing documentation and making videos to explain things like "layer 2 file sharing isn't like layer 3, and really needs wired Ethernet on the same physical switch". That by itself is a very complex concept for people who aren't system or network admins, and I really want this to be as easy as possible for people without requiring that knowledge. But that means lots and lots of clear and well organised documentation.

I'm familiar with this when I did some mucking about trying to make this project:

https://stickfreaks.com/software/dosisoify

Design was a pre-packaged floppy image with NIC drivers, mTCP and wget that let you pull an ISO down from a HTTP server and then mount it via SHSUCD as a virtual drive letter.

Ended up being a bit too complex I think, but I could try to revive that as a set of easy-to-use client tools for MS-DOS users. Not sure.

Definitely a good question, and I've been thinking a lot about it. Like you, I prefer Ansible, but that's a heavyweight setup with lots of pre-requisites.

I was thinking about a "PiKiss" or "raspi-config" style dialog-driven "TUI" (text interface, but menu driven) which is what a lot of RPi users are familiar with already. they could then choose individual components or just go with the all inclusive "do the whole lot" option.

Whether I do the work then through Ansible or just write some little shell scripts, I'm not sure. I *really* like the Ansible option though, for extensibility to many other things outside of RPi.

 

TFTP - https://en.wikipedia.org/wiki/Trivial_File_Transfer_Protocol
Hello 1980s File Sharing.

6502 TCP/IP
https://cc65.github.io/ip65/


Amiga
https://github.com/obarthel/amiga-sana-ii-tftpclient

Atari
https://github.com/anodynesoftware/pftp

sudo apt-get install xinetd tftpd
https://linux.die.net/man/8/xinetd
https://linux.die.net/man/8/tftpd

tftp will also work with esp32 and esp8266

 

I'd suggest PXE support but maybe that is beyond the scope of this project. Also now that I think of it, older network adapters don't typically come with a PXE capable boot ROM and installing one isn't usually a simple matter. Still, if you're using tftpd, you'll be half way there anyway. Then you could have menu options to boot various images like that dosisoify image that you made. UBCD is another option that is easy to get going over PXE.

I think I mentioned in the past that I was using a DOS PXE image a UNDI network driver and the ether-DFS client to map a network drive. Most software I tried worked really well but there were some that just didn't work and seemed to mess up the ether-DFS client so that it thought every directory was empty. I never tried but it'd probably be possible to install earlier versions of Windows over the network using a setup like this.

 

My home network currently uses PXE, which sends the iPXE "undionly.ko" ROM, chainloads into iPXE, with full support for iSCSI installs and the like for retro OSes (usually a manual process of boot from iSCSI, format/sys c:, copy files, boot from local disk and install), as well as modern OS installs like Linux over HTTP or Windows Vista through to 11 inclusive via WinPE ISO for BIOS computers and WIMBoot for UEFI computers (in both cases, extracted by wimlib).

I supply a many-levels-deep iPXE menu with different families of operating systems that you can navigate around and choose your OS (screenshots in the thread linked below).

That's definitely on my "stretch goal" list for this project, however the critical piece there is that I need to be careful about the legalities of distributing OS images (even if they're "abandonware", I want the core project here to be legal).

With that said, I've also got a few scripts lying around for generating either images or iSCSI LUNs from an ISO, so I could do that as a way of giving people a simple way to have MS-DOS or WIn9X installers if they supply their own floppies/ISO, and I do the rest with a script.

I could certainly have FreeDOS there as a test case for anyone who wants to try the process with an existing OS before going to the effort of MS-DOS or Win9X.

I can generate floppy and USB images with very small iPXE ROMs on them (usually a few KB at most, far less than the size of a floppy). I can also make the floppy images Gotek/FlashFloppy compatible for people who have that, so they can boot that way and PXE load from any Ethernet card with or without a ROM on it.

You can see a small part of what I have running at home here:
https://forums.overclockers.com.au/threads/network-install-windows-10-8-7-vista-too-at-home.1283899/

That's specifically for Vista and up. There's vast differences required for older Windows OSes and DOS, but I can definitely have a wide variety of options available.

 

When I wrote "DOS" before, I should have writen "FreeDOS". That's what I've been using. I was thinking just provide network access from FreeDOS for installation and let people worry about putting their own OS installation files on the RPi server using Samba etc.

A very good point. I totally forgot about booting from floppy drive. I haven't done that much in a very long time and I've only used iPXE over network.

 

Began bashing some Ansible playbooks together yesterday. Will hide that behind some dialog/ncurses menus for ease of use (people familiar with Ansible can bypass that if they want). Maybe see what I can integrate with cockpit as well maybe? I dunno.

Focusing on getting the storage stuff out first (mostly the first post), with other services like netboot and PXE afterwards. Anything that will conflict with a standard home user's setup (DHCP, etc) will need to have some more clever thought (maybe offer DHCP forwarder via dnsmasq, but include some extra options for PXE, etc?).

Not a lot of time to write this just now. Hoping to do a big sprint over Christmas and get it up on GitHub after that.

 

This is all pretty much a strange unfamiliar world to me, I don't do networking and never have (poor choice I know, but unless y'all have a time machine handy...).

So I look forward to watching developments, learning and benefiting from what this has to offer.

Keep in mind that any commentary from me here is likely to be results-based rather than technical.

I don't quite understand this. I'm, no doubt wrongly, envisaging a frankly-irritating process of an old computer slowly transferring in 100-700MB each time, as a non-trivial delay time added to each play. Probably totally the wrong view. Or does this copy only happen the first time, and the ISO is then retained on the target system for the next play? /noob

 

"That depends".

The diversity of old PCs is enormous, so really there's a million options. I was looking at a fairly decent quality MS-DOS machine (386 era, MS-DOS 6.X, 100 Mbit/s (10MB/s) Ethernet, good size spindle / SSD / CF-IDE storage). With that setup you could copy a standard ~700MB ISO to your machine in under a minute and leave it on your hard disk. From there, SHSUCD could mount it as a drive letter any time you like and access the contents directly.

If you've got an optical drive and relevant drivers already, then just burning the ISO to a CD-R is probably easier. If you're on a VM or in DOSBox, ISO mounting is native. But the design there was to offer the same flexibility that something like a Gotek/FlashFloppy device offers - a modern way of tackling legacy physical formats without needing the physical hardware.

Really old devices with low physical disk space and 10Mbit/s (1MB/s) network? That's going to suck. Likewise if you already have a physical CD-ROM in your retro machine, and a CD burner somewhere else, that's going to be easier. Same argument for people who have real floppies and drives - just use that. Tools like a NAS+SHSUCD, Gotek/FlashFloppy, etc are all there for futureproofing or convenience.

The tools I'm building in this thread are high on the "convenience" side of things. I get told on the daily that "nobody needs them" because physical media and SD cards and floppies and the rest are the "one true way", but the reality is options are good, and the biggest hurdle for network based options is the complexity of setting them up. (Console users - there are people working on WiFi options for flash carts and ODEs, so don't think that SD cards are your only option forever!).

My goal here is to offer something cheap and simple - download my software, run a menu driven tool to install things, point your retro computers/consoles at it and off you go. If you need to repeatedly reinstall things, or need quick access to games and files, or have more games than you have local drive space (think eXoDOS/eXoWin3x at many TB of space), or you have multiple old computers and want to share data (want to copy from MS-DOS to Apple Mac Classic?), this will help.

As I said above, more difficult than writing this software (which is quite easy, honestly) is making comprehensive documentation. I'm hoping to get a lot of volunteer help there from various communities, and especially focusing on videos to show people, as I can make the server side easy, but there's always complicated client-side stuff that's wildly different on old computers and consoles.

 

Thanks for explaining the rationale.

Conversely my viewpoint is that of a person with a non-networked array of machines which are now arranged more-or-less as I want them. I see your project here and my goal then becomes "find reasons to use this". Several of which you have outlined above, acknowledged.

My own small data point: my baremetal [FAT32] DOS machine's storage is a 32GB SD card. Over the 30+ years I've been PC-compatibling*, I've played all the nominally-known DOS games in existence and curated my way down to the general subset of "pretty much all I'll ever play, give-or-take half a dozen games per year". That SD card is currently about 10GB full. With that not-uncommon $30-total storage solution, I could pack C:\ISO with quite a bit more stuff before I'd be looking outside for storage.

But then, what happens when I want to set up from scratch again? etc. I can see many use cases for what you're doing, even from my limited perspective.

*the best kind of bling

 

This project very likely won't be for people who don't want a network, or are perfectly happy with "sneaker net".

I remind folks though that this project is aimed at a wider array of devices than just computers. There are dozens of retro consoles now with network capabilities. For example, the Nintendo 2DS and 3DS family uses the "FBI" package installer to install "CIA" (internal 3DS package format) packages. One way to install these is to power down the 3DS, remove the very fiddly MicroSD card from the 3DS (annoyingly difficult on models like the New 2DS XL), copy your CIA package to it, power back up, open FBI installer, navigate to the file, choose install.

An alternative install method is to scan a QR code that points to a http:// link, and it installs. The difficult bit? Have a simple NAS system where you can dump your CIA files into a SMB share and have the QR codes auto-generated a minute later. And guess what I'm making?

As above, there's way more I'm going to try to cover. SMBv1 for Win9X (deprecated by most NAS devices and Windows desktops) and PlayStation 2, ps3netsrv for PlayStation 3, AFP for classic Macs, FTP and TFTP for countless retro OSes as well as Classic XBox and XBox 360, EtherDFS for MS-DOS, dial-up emulation for computers with only a dial-up modem to connect them to your Ethernet network, and hopefully many, many more.

Not to mention the retro console ODEs coming (I'm assuming next year, no official word though) populated with ESP32 connectors that will allow transferring ISOs and save states over WiFi. New games is discovered (or new homebrew game created), and no sneakernet needed. Just download it to your NAS and it's ready on your console.

MiSTer fans will be well catered for. If you want to play everything MiSTer supports, you're up for TB of space. This project will help you out (and offer things like inline compression and deduplication for advanced users).

I use all of these tools for myself already, and the time and effort savings are enormous for me. When I recommend them to others, the number one complaint is the complexity of setting it up. This project aims to dramatically reduce that complexity.

Definitely a big plus for anyone who rebuilds retro machines frequently. Network connections mean only installing network drivers, and then everything else is available on a storage system far, far larger than what the native computer can even deal with in a removable device.

The example for MS-DOS fans is eXoDOS. There's non way that complete pack will ever fit on a real DOS machine with real hardware. But we can definitely make it accessible on-demand over network. That same logic applies for console ODE fans. Full Redump sets are enormous, and having it all stored on a NAS is very convenient.

My ultimate goal is getting this into museum style setups. Retro Man Cave just opened his up as an example, but I'm helping 2 other places around the world with their museum setups. I'd love to make something to help these people reduce their setup and maintenance time.

 

You have my attention
I know you can't have roms or links to roms, but maybe like the updater script you can "find" some helpful scripts that go and grab "stuff"

Happy to be a beta guinea pig on the mister side of things.

 

There's a group of people out there who are working on a method of making this happen, but for obvious reasons I can't talk much about it here.

What I can tell you is that this looks to be the tool of choice:
https://syncthing.net/

Hopefully that gives you some hints as to where it's going. I'll be integrating that into the RetroNAS thing (n.b.: name not set in stone - and please give me suggestions for a name if you hate "RetroNAS". I'm not a marketing person, so I'll take any advice on board).

When that gets up and running, I'll let you know. More willing beta testers will be great.

 

NINJARN
NINJARN Is Not Just A Retro NAS



Edit - I also had NINARN, is not another retro nas
And SFERNM - stick freak elvis's retro nas machine

 

With a smidge of spare time up my sleeve, I put a little work into this over the weekend. Most of the "must have" features are in, and I've done some very basic testing.

Guides and howtos are going to take way longer, but maybe I can knock something text up soon, and visual up Christmas.

Hoping to have something alpha release quality late this week on GitHub, if there are any willing volunteers who want to test parts out after that.

 

Rent a beta tester ?

 

Ha. Bloody Google Swype.