Scripts and File Globs

Discussion in 'Other Operating Systems' started by Primüs, Jun 11, 2018.

  1. Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,348
    Location:
    CFS
    Bit of help needed.

    I want to script some multi-command actions, but I need to be able to hand it a file glob i.e.

    dir1*/*.pdf

    What the script is essentially doing is changing permissions, changing owner, then want to run an installed binary. All the commands (chmod, chown, <othercommand>) all work perfectly on CLI issuing the wildcard, but when running in the script it only really grabs the FIRST result.

    I've looked into it and I've read that scripts dont really understand globs, and you need to go through looping it manually to find your files and apply, this is OK for the chmod and chown, but the 3rd binary is kind of a batch processor where it wants everything passed through not spawning a new instance for each file.

    Is there a way to get around this glob processing?

    Using bash but happy to change language / runtime to something that will work.
     
  2. GumbyNoTalent

    GumbyNoTalent Member

    Joined:
    Jan 8, 2003
    Messages:
    6,174
    Location:
    Briz Vegas
    find all your files and output to a tmp file then read the file as a list and loop process.
     
  3. OP
    OP
    Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,348
    Location:
    CFS
    Gives the same problem unfortunately
     
  4. GumbyNoTalent

    GumbyNoTalent Member

    Joined:
    Jan 8, 2003
    Messages:
    6,174
    Location:
    Briz Vegas
    So when you pass to it it should have an option for a text file list of files, or a comma separated list, preformat the command line .
     
  5. waltermitty

    waltermitty Member

    Joined:
    Feb 19, 2016
    Messages:
    745
    Location:
    BRISBANE
    If you store your glob in a variable it should be expanded, yeah? Just do that and pass it to the batch process
     
  6. GumbyNoTalent

    GumbyNoTalent Member

    Joined:
    Jan 8, 2003
    Messages:
    6,174
    Location:
    Briz Vegas
    can you show example of what you are globbing (remove sensitive info first)?

    PS what shopt's are you running?
     
    Last edited: Jun 12, 2018
  7. OP
    OP
    Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,348
    Location:
    CFS
    No it doesnt, i've tried passing it through as variable and tried escapign it and passing it to the binary as string it always seems to just expand to first match only.

    basically using filebot for media renaming in this example (although come along other similar examples before and looping was acceptable for those ones)

    basically in script it runs:

    The glob im passing would be similar to "Show.Name.S01*/*.mkv" to match all mkv files only in Show Name S01.

    If I run the command with the glob in my shell, it works perfectly, but in script it will only every match the first.

    Other use case is passing '.' (or defaulting to '.' if no glob /filepath given) and in script that works to match every file in current folder, but i'd imagine '.' is handled differently to a wildcard glob.

    my shopt's are default for Ubuntu Server 16.04:

     
  8. GumbyNoTalent

    GumbyNoTalent Member

    Joined:
    Jan 8, 2003
    Messages:
    6,174
    Location:
    Briz Vegas
    try adding
    Code:
    shopt -s extglob
    after
    Code:
    #!/bin/bash
     
  9. grs1961

    grs1961 Member

    Joined:
    Jan 21, 2005
    Messages:
    488
    Location:
    Melbourne
    Does something like
    ...
    command `echo 1*/*.pdf`
    ...
    Note the back ticks, do what you want?
     
  10. OP
    OP
    Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,348
    Location:
    CFS
    no help :(

    Tried that and putting my command in command `` and no luck. sigh.
     
  11. GumbyNoTalent

    GumbyNoTalent Member

    Joined:
    Jan 8, 2003
    Messages:
    6,174
    Location:
    Briz Vegas
    Code:
    #!/bin/bash
    for i in `ls dir1*/*.pdf | sort -V`
    do
        chmod 755 $i
        chown user:user $i
        filebot -rename $i [opt]
    done
    
     
  12. OP
    OP
    Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,348
    Location:
    CFS
    That will spawn a new instance of filebot for each file, vs if i ran filebot -rename dir1*/*.mkv it'd batch process the whole lot.

    In terms of processing, there is a very big difference. This would make it quicker to still do it manually.
     
  13. GumbyNoTalent

    GumbyNoTalent Member

    Joined:
    Jan 8, 2003
    Messages:
    6,174
    Location:
    Briz Vegas
    is only a loop will execute child to completion before progressing to the next item in the list.

    OR

    https://www.cyberciti.biz/open-source/command-line-hacks/linux-run-command-as-different-user/

    run the script as appropriate user.
     
  14. OP
    OP
    Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,348
    Location:
    CFS
    Yes but if i'm feeding it 20-30 + files it will have to do the full processing on each file individually, where-as feeding it as a glob gets a lot of the processing done straight away. Anything over 3 or 4 episodes I may as well run it manually to make it quicker (What i've been doing lately)
     
  15. grs1961

    grs1961 Member

    Joined:
    Jan 21, 2005
    Messages:
    488
    Location:
    Melbourne
    What does a simple "echo dir1*/*.pdf" give from the command line?

    it should give something like this:
    $ echo Doc*/*.pdf
    DocNOT/1.pdf DocNOT/2.pdf DocNOT/3.pdf Documents/01_cover.pdf Documents/0884831916_000001145783433.pdf Documents/Altronics_Catalogue_2013.pdf Documents/BC-1-img-905123847-0001.pdf Documents/Belkin_SMB_KVM_p74902ea_f1dp108a.pdf Documents/Benchmarking_ServiceDesk_Success_US.pdf Documents/Document_20140815163524.pdf Documents/E35332.pdf Documents/viking_cli_config_guide.pdf
    $
     
  16. OP
    OP
    Primüs

    Primüs Member

    Joined:
    Apr 1, 2003
    Messages:
    3,348
    Location:
    CFS
    OK my thinking was completely wrong - I was lead to believe that the glob passes in a single argument value constituting a full ARRAY of the glob results, hence why it'd only catch the first result (defaulting to 0 index)

    What actually happens is the glob expands the array before entering the script so the script becomes:

    Code:
    ./myscript dir1/file1 dir1/file2 dir2/file1 etc etc etc
    
    So I can access it using $@ and just got to be mindful in future of extra arguments I may add to it.

    I still dont understand why it cant just pass and process the glob through as if you were on the commandline, since I use bash and im running this as bash, but thats an argument for a different day.

    Thanks for the help guys I really appreciate it.
     
  17. GumbyNoTalent

    GumbyNoTalent Member

    Joined:
    Jan 8, 2003
    Messages:
    6,174
    Location:
    Briz Vegas
    That my friend is up to the programmer of filebot to address globs properly, few do thus why we still use recursion feeding singular values when doing file renaming.
     

Share This Page