RetroNAS

Discussion in 'Retro & Arcade' started by elvis, Nov 27, 2021.

  1. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    New tool add - "gogrepo". Backs up your entire GOG library - all installers, manuals, addons and extras, for DRM-free offline storage. Based on this:
    https://github.com/eddie3/gogrepo

    Install it via "Install things", run it via the new "Tools / Scripts" entry in the menus.

    You'll need to run the "Configure GOG account" once to log in securely and store a cookie. Then choose your OS downloader (Windows/Mac/Linux), or choose ALL to grab the lot.

    I'll try and make a menu for it soon to choose individual games, instead of grabbing everything by default. If you've got a very large GOG library, it could get tedious.

    Wiki:
    https://github.com/danmons/retronas/wiki/gogrepo

    [edit]

    Currently it drops the GOG cookie and manifest in your exported RetroNAS dir, which in hindsight is insecure if other people can read it (they can't do much other than download your games, but that's still not very cool). I'll change that tomorrow to drop the cookie and manifest in the user's home directory, but keep the games in the main storage dir.
     
    Last edited: Jan 15, 2022 at 12:45 AM
    MUTMAN and Pierre32 like this.
  2. greencamel65

    greencamel65 Member

    Joined:
    Jan 17, 2003
    Messages:
    670
    Location:
    Asgard
    Cheers, Running it now, Easy enough - However i had to install the python3-html5lib package manually on my debian vm after installing via the retronas menu, I'm not sure if a fresh pi install would be missing the package or not.
     
    elvis likes this.
  3. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Cheers, will add to the dependencies list.
     
    greencamel65 likes this.
  4. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Balls, just found a typo. New commit just now.

    Re-install it to fix the script. Existing cookies/manifests will be untouched.
     
    Last edited: Jan 15, 2022 at 6:57 AM
    greencamel65 likes this.
  5. Pierre32

    Pierre32 Member

    Joined:
    Oct 13, 2019
    Messages:
    1,551
    Location:
    Sydney
    That's cool, I'll definitely give it a run. I've been manually downloading GOG installers like some kind of caveman. I'd better do a storage upgrade first though, as that 120GB SSD I had laying around is filling up fast. That'll also give me a chance to use Cockpit to configure the new volume with separate partitions.
     
    elvis likes this.
  6. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    I'll try and work on a per-game menu soon. It'll grab everything for one title (all installers, manuals, extras, etc), but at least let you cherry pick the specific game you care about as a priority.

    Gotta get the Samba and AFP videos out today though.
     
    greencamel65 and Pierre32 like this.
  7. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Samba video done.

     
    Daft_Munt and Pierre32 like this.
  8. MUTMAN

    MUTMAN Member

    Joined:
    Jun 27, 2001
    Messages:
    10,134
    Location:
    4109

    I see a heap of vids done :)
    Great work there elvis :thumbup::thumbup:
     
    elvis likes this.
  9. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Yeah I figured I'd tag them all as visible now. AFP is rendering and uploading shortly.

    I figure that's the basic stuff done for now. Enough info for someone cluey to get a Pi and install a couple of protocols to play with. From there they should get the hang of it and be able to do other stuff.

    Still TONNES to document of course, but it's a start. I need to completely re-do Netatlk2 from scratch now that I've found a much better source port. A fair few hours of testing there to be done to make sure 30 years of Macs can access the server, and maybe an extra video for ye olde System 6. Then get the GOG downloader sucking less. Then videos for EtherDFS and GOG. Then the rest of the ideas:
    https://github.com/danmons/retronas/blob/main/IDEAS.md

    I feel like for every milestone I hit, I create three more, hahaha. :lol:
     
  10. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Netatalk / AFP / AppleShare / AppleTalk video up.

    I've had a tonne of breakthroughs since this went up. This is essentially OS8/OS9 with TCP level Macs and up, but since recording this video I've conquered all the way back to Apple II GS with GS/OS (aka System 6) and M68K Macs. I'll make a separate video for those once I grind through some more testing tomorrow.

     
    Daft_Munt, Pierre32 and MUTMAN like this.
  11. Pierre32

    Pierre32 Member

    Joined:
    Oct 13, 2019
    Messages:
    1,551
    Location:
    Sydney
    New storage procured; a Sandisk 480GB portable SSD. On watching the RetroNAS Cockpit and Storage video, I realised that we still have information to come on setting up a separate volume for EtherDFS. But I wanted to play with Cockpit's storage setup, so I forged ahead, knowing I'd have to run EtherDFS manually if I wanted to use it.

    The plan is a 40GB VFAT partition with the remainder as NTFS, which will allow me to plug it into the Win10 box too.

    Again we get into some pretty novice bumbling from me, but I'm not scared to look silly if it helps with documentation.

    I created the partitions to plan, adding iocharset=cp437 to the custom mount options for the VFAT partition, per EtherDFS docs. No problems.

    I then moved the drive to Win10 to start transferring some content. The drive was not appearing in Explorer, but it was in Disk Management (with all context menu options greyed out apart from Delete Volume). I figured I was in science mode, so let's recreate those partitions here and see what it looks like to the Pi.

    [ Sidebar: Rebooting the Pi without the storage attached, it wasn't coming up on the network. So I plugged a monitor in, and saw that it was hanging when trying to mount the missing volume (like this) and halting in emergency mode. From here I was able to edit fstab to comment out those entries, reboot successfully, then plug my new drive in again. ]

    I had labelled my partitions "vfat" and "ntfs" when formatting in Windows, so these auto-mounted as:

    Code:
    /media/pi/vfat
    /media/pi/ntfs
    I ran RetroNAS and set my top level directory to

    Code:
    /media/pi/ntfs/
    I reinstalled Samba, EtherDFS and ProFTPd. Then from Windows I hit the Samba share, and all was good. I saw the EtherDFS /dos folder created in there. Not the mount I'll be using for DOS stuff of course, but it lets me know stuff's working. I created a text file in there as a test, started EtherDFS on the DOS machine, and saw my text file.

    Now I killed EtherDFS and restarted it with my desired mount, /media/pi/vfat. This works, but as expected I'm told that it's not happy with the format of the volume.

    Code:
    WARNING: the path '/media/pi/vfat' doesn't seem to be stored on a FAT filesystem! DOS attributes won't be supported.
    So into Cockpit -> Storage to once again to reformat that partition as actual VFAT. Then I plugged the drive back into Windows again. The "ntfs" partition showed up of course, but "vfat" did not. My original 120GB SSD (single VFAT partition) does work in Windows though. So I need to have a look at the properties of each and see if I can uncover what I did differently with that one.

    Honestly, Windows is just irritating in this regard. I do want this functionality though, and I'm sure others will too.
     
    MUTMAN and MR CHILLED like this.
  12. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Check the warnings about maximum RPi USB amp draw on 5V-only 2.5" drives attached via simple USB to SATA connectors. If you find storage disconnecting under heavy loads, you can get a USB Y-splitter and supplement the power to your disk.

    RE: FAT problems, if you run "sudo blkid" on the partition name in Linux, what does is say about the header?
     
    Last edited: Jan 15, 2022 at 9:42 PM
  13. Pierre32

    Pierre32 Member

    Joined:
    Oct 13, 2019
    Messages:
    1,551
    Location:
    Sydney
    I was already powered down and had everything attached to the Windows machine again. But, we can mark it solved!

    EtherDFS happily plays with a FAT32 volume that has been formatted in Windows. But that's not what I had, because Windows doesn't allow you to FAT32-format a volume over 32GB (not in the GUI anyway). The option you get is exFAT, and that's how I'd formatted my new 40GB partition. But my old 120GB drive is FAT32, because I had formatted it with Paragon Partition Manager. Just did the same on the new drive, and both systems are happy.

    Something that occurred to me while doing this is that I can't see my DOS volume from Win10. Probably something you're going to cover when you get to the EtherDFS vid. Will it be possible to see that over SMB?

    [edit] For posterity, adding a separate issue I had with write permissions on the FAT32 volume.

    Once I had my volumes all visible on the network I hit another issue: Only root had write access to the FAT32 volume. This made it unmanageable from DOS & Windows. RetroNAS has a "Fix permissions" tool but this doesn't work on Windows-formatted FAT32 drives (AFAIK the chown command doesn't compute on such a volume).

    The solution was to add uid & gid parameters to that volume's line in /etc/fstab:

    Code:
    UUID=2192-604A /media/pi/FAT32 auto defaults,uid=1000,gid=46 0 0
     
    Last edited: Jan 18, 2022 at 11:05 AM
    MUTMAN likes this.
  14. greencamel65

    greencamel65 Member

    Joined:
    Jan 17, 2003
    Messages:
    670
    Location:
    Asgard
    My mapping is a bit differnet to yours but i can access the EtherDFS share over SMB.
    I'm running as a vm but i'd imagine the same is doable on baremetal

    /data/retronas/ 12TB btrfs passthrough drive
    /data/retronas/dos/ 40GB vfat vdisk

    Now i'm worried this might cause issues down the line? My linux experence is somewhat lacking but i fumble my way through.
     
  15. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Should be fine if your FAT partition is mounted underneath the root share.

    If not, I can guide you through creating another share to point to it.

    The design of it all was generic protocols (SMB, AFP, FTP, HTTP) sharing out of the top level, and specific tools all one level deep if they need any special setup or isolation for given reasons.
     
  16. Pierre32

    Pierre32 Member

    Joined:
    Oct 13, 2019
    Messages:
    1,551
    Location:
    Sydney
    My top level is set to my NTFS partition mount, \media\pi\ntfs.

    If I set it one level higher to \media\pi, I see both of my partitions on the network: \ntfs and \FAT32 - that works fine. But then any subdirs created by services (\dos, \ps2 etc) are created directly under \media\pi.

    Instead I've made this change /etc/samba/smb.conf:

    Code:
    [retronas-ntfs]
    include = /etc/samba/retronas.conf
    
    [retronas-fat32]
    comment = EtherDFS partition
    path = /media/pi/FAT32
    browsable = yes
    read only = no
    create mask = 0775
    directory mask = 0775
    guest ok = no
    valid users = pi

    And this to /usr/lib/systemd/system/etherdfs.service:
    Code:
    ExecStart=/opt/retronas/bin/ethersrv-linux -f enx00e04c68022e /media/pi/FAT32
     
  17. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    Yup that should work fine.

    For regular users, I'm going to do a guide on mounting a FAT volume at /data/retronas/dos and demonstrate Linux's ability to put mounts under mounts.

    But you can obviously do whatever you want. It's open source baby, and you've got all the control. :)
     
  18. Pierre32

    Pierre32 Member

    Joined:
    Oct 13, 2019
    Messages:
    1,551
    Location:
    Sydney
    That'll be a useful guide. Tweaking config files is fun, but not tweaking them is also fun. At some point I'll probably do a fresh install, so I can enjoy all the effort you're putting into simplicity.

    Currently giving gogrepo a run.

    [edit] A little gogrepo tip while we await a menu to select which games to download. If you log into your library on the website and set games to Hidden, they won't be downloaded by gogrepo.
     
    Last edited: Jan 16, 2022 at 12:34 PM
    MUTMAN and elvis like this.
  19. MUTMAN

    MUTMAN Member

    Joined:
    Jun 27, 2001
    Messages:
    10,134
    Location:
    4109
    thats gunna hurt heads like mine :lol: I'll stick to a box for MiSTer ROMs/ISOs for now :tired:

    No pressure as I figure the core usage is best concentrated on for now, but that torrent client would be super handy for me (add a simple OpenVPN setup to it to please :) )
     
  20. OP
    OP
    elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    45,640
    Location:
    Brisbane
    PlayStation 2 + OpenPS2Loader + SMB video:

     
    Pierre32 and MUTMAN like this.

Share This Page

Advertisement: