1. OCAU Merchandise is available! Check out our 20th Anniversary Mugs, Classic Logo Shirts and much more! Discussion in this thread.
    Dismiss Notice

The general *nix chat thread

Discussion in 'Other Operating Systems' started by Asteroid, Jul 1, 2012.

  1. Rass

    Rass Member

    Joined:
    Jun 27, 2001
    Messages:
    3,179
    Location:
    Brizbekistan
    stipud question. what am I doing wrong with the following?

    this gets no results:
    grep -f index.txt searchfile.csv

    this gets me the results I was after:

    #!/usr/bin/perl
    #use strict;
    #use warnings;

    open (IndexFile, 'index.txt');
    while (<IndexFile>) {
    chomp;
    print `grep $_ searchfile.csv`;
    }
    close (IndexFile);



    Is there a limitation to the size of files which grep can use for strings? index.txt is about 3k and searchfile.csv is around 66mb.
     
  2. elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    46,621
    Location:
    Brisbane
    What does "file index.txt" say? My guess is index.txt was created on Windows, and is full of extra DOS end-of-line markers.
     
  3. Rass

    Rass Member

    Joined:
    Jun 27, 2001
    Messages:
    3,179
    Location:
    Brizbekistan
    yeah, I just opened it up and showed symbols, and each line has a <cr><lf>... so the chomp was un-breaking it.

    Thanks!
     
  4. elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    46,621
    Location:
    Brisbane
    Ah Windows, is there anything you can't fuck up? :p

    Most Linux distros come with a tool called "To From DOS", which is amusingly abbreviated to "tofrodos" (To Frodo's?). It will trim the extra end of line character, and is safe to re-run on files that don't need it. I tend to run it over input text files in scripts just by policy, given that I don't trust Windows users to know what the hell they're doing ever.
     
  5. ex4n

    ex4n Member

    Joined:
    Oct 5, 2011
    Messages:
    2,242
    Location:
    Perth
    Thanks guys, I am still learning how to troubleshoot this stuff, looking at the logs.. I should have thought of that.. I am used to pretty useless windows logs. I don't get any error when starting, just [fail] so here is what I got in the logs.

    Code:
    Aug 30 12:14:05 kashmir named[2958]: starting BIND 9.8.1-P1 -u bind
    Aug 30 12:14:05 kashmir named[2958]: built with '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc/bind' '--localstatedir=/var' '--enable-threads' '--enable-largefile' '--with-libtool' '--ena$
    Aug 30 12:14:05 kashmir named[2958]: adjusted limit on open files from 4096 to 1048576
    Aug 30 12:14:05 kashmir named[2958]: found 4 CPUs, using 4 worker threads
    Aug 30 12:14:05 kashmir named[2958]: using up to 4096 sockets
    Aug 30 12:14:06 kashmir named[2958]: loading configuration from '/etc/bind/named.conf'
    Aug 30 12:14:06 kashmir named[2958]: /etc/bind/named.conf.options:26: undefined ACL '.none.'
    Aug 30 12:14:06  named[2958]: last message repeated 5 times
    Aug 30 12:14:06 kashmir named[2958]: /etc/bind/named.conf.options:27: undefined ACL '.none.'
    Aug 30 12:14:06 kashmir named[2958]: loading configuration: failure
    Aug 30 12:14:06 kashmir named[2958]: exiting (due to fatal error)
    so I look into /etc/bind/named.conf.options and checked lines 26 and 27.. then I remembered a few days before I added something to this (probably never rebooted since).

    Code:
    26: // allow-transfer {.none.;};
    27: // allow-recursion {.none.;};
    28: recursion no;
    29: version "Not Disclosed";
    
    so I commented out these 2 lines, and now it works again. nothing to do with the hostname :) I was adding those lines as someone told me it will stop my server being used for DDOS requests, now I just need to fix this and test it again.

    Thanks for the help I now know where to look.. I feel very enlightened.
     
    Last edited: Aug 30, 2013
  6. Unframed

    Unframed Member

    Joined:
    Mar 30, 2010
    Messages:
    9,394
    Location:
    Hella south west
    Hey elvis, did you get around to playing with the wife's laptop? Just curious as I still haven't managed to get optimus running yet, seeing if you had some magic up your sleeve.
     
  7. elvis

    elvis Old school old fool

    Joined:
    Jun 27, 2001
    Messages:
    46,621
    Location:
    Brisbane
    Nope, haven't had time yet. :(
     
  8. Unframed

    Unframed Member

    Joined:
    Mar 30, 2010
    Messages:
    9,394
    Location:
    Hella south west
    All good, thought I'd ask as most answers I find online are a dead end.
     
  9. Fred Nurk

    Fred Nurk Member

    Joined:
    Apr 5, 2002
    Messages:
    2,257
    Location:
    Cairns QLD
    Anyone had much experience with GNU's coreutils on a Win32 machine? Got some configuration files that would be handled reasonably easy (well, it'd cut down a lot of tedious effort at any rate) using coreutils but not quite sure just how compatible it is.
     
  10. kripz

    kripz Member

    Joined:
    Sep 29, 2004
    Messages:
    2,834
    Location:
    Near Frankston
    I copied grep out of it, didn't even require any dependencies, just worked. I don't have any experience with the other programs. Also be wary of windows new line.
     
  11. IKT

    IKT Member

    Joined:
    Feb 4, 2007
    Messages:
    4,277
  12. GreenBeret

    GreenBeret Member

    Joined:
    Dec 31, 2001
    Messages:
    19,370
    Location:
    Melbourne
    Ended up moving ~/.cache to /var/cache/org/user/ instead of RAM, cos can't mount anything under NFS-mounted /home/user. NFS server load dropped from medium to absolutely nothing (~120 clients on a 2.4Ghz quadcore, 16GB RAM, also runs Samba for about 150 PCs). So it was Firefox, Chrome and Google Talk/Hangout smashing it badly.

    The other candidate for a similar migration is ~/.local, particularly gvfsd-metadata.

    Bonus for both: reduced backup time / size.

    :thumbup:

    I still use ~/.cache on tmpfs for machines that don't have NFS /home, esp the ones with SSD (longevity++).

    Yeah I've already got /scratch for the power lusers to write their big temp data sets to. Occasionally there's still a newbie dumping it to /home though. :lol:
     
  13. tr3nton

    tr3nton Member

    Joined:
    Oct 14, 2009
    Messages:
    1,281
    http://www.linux.com/news/featured-...ideos-from-linuxcon-and-cloudopen-new-orleans

    11 Keynote Videos from LinuxCon and CloudOpen New Orleans

    The Linux kernel developers panel with Tejun Heo, Greg Kroah-Hartman, Sarah Sharp and Linus Torvalds. The session is moderated by Ric Wheeler.



    Eben Upton, Raspberry Pi



    Gabe Newell, Valve



    "Developers, the new super power." Jonathan Bryce, OpenStack



    "New frontiers." Dirk Hohndel, Intel



    "The Technium." Kevin Kelly, Wired



    "What five years of Android has meant." Chris DiBona, Google.



    "The state of Linux." Jim Zemlin, The Linux Foundation



    "Before I die." Candy Chang, Artist and TED Fellow



    "Linux, cloud and next generation workloads." Brad McCredie, IBM



    "Open matters." Brian Aker, HP.

     
  14. IKT

    IKT Member

    Joined:
    Feb 4, 2007
    Messages:
    4,277
  15. gcflora

    gcflora Member

    Joined:
    Jun 22, 2012
    Messages:
    665
  16. ex4n

    ex4n Member

    Joined:
    Oct 5, 2011
    Messages:
    2,242
    Location:
    Perth
    I assume this is more of a CSS trick than a vulnerability? Still good advice, if you just blindly copy paste without looking.

    Code:
    <p class="codeblock">
          git clone
          <span style="position: absolute; left: -100px; top: -100px">[201~/dev/null; clear; echo -n "Hello ";whoami|tr -d '\n';echo -e '!\nThat was a bad idea. Don'"'"'t copy code from websites you don'"'"'t trust!<br>Here'"'"'s the first line of your /etc/passwd: ';head -n1 /etc/passwd<br>git clone </span>
          git://git.kernel.org/pub/scm/utils/kup/kup.git
        </p>
    edit: just realised you would have to paste into a text editor first, or it would just start to execute the code before you got a chance to read it and press enter.
     
    Last edited: Oct 7, 2013
  17. Braedz

    Braedz Member

    Joined:
    Jun 30, 2010
    Messages:
    299
    Location:
    Adelaide
    Tried Archlinux for the first time over the weekend. I think this is going to end up being my favorite distro.

    Managed to get a Archlinux VM to boot only using 28Mb of RAM. That is nuts.
     
  18. IKT

    IKT Member

    Joined:
    Feb 4, 2007
    Messages:
    4,277
    The AMD Radeon Performance Is Incredible On Linux 3.12

    http://www.phoronix.com/scan.php?page=article&item=amd_linux312_major&num=1
     
  19. Daemon

    Daemon Member

    Joined:
    Jun 27, 2001
    Messages:
    5,474
    Location:
    qld.au
    I've come across a real niggling little issue which is proving to be annoying to resolve. I've setup a CentOS 6 VM running BIND (9.8.2) as a secondary NS and run into a very frustrating file permission error.

    As it's CentOS / RHEL, the bind process runs as "named" with user / group both named. Zone files are stored in /var/named, again standard config.

    Initially running it the zone files refused to write so of course I double checked permissions. The file ownership is correct and I've confirmed that the process is running as named.

    Now for the gotcha, I ran named manually (/usr/sbin/named -u named) and it worked perfectly.

    The run line in the init.d script is as follows:

    daemon --pidfile "$ROOTDIR/$PIDFILE" /usr/sbin/"$named" -u named ${OPTIONS};

    Both appear exactly the same from a process list, so it has me stumped! I've double checked and both are reading the same config so therefore trying to access the same files.

    I've run strace to see if I can identify anything quirky but nothing stands out. I can't figure out why they're they're different yet running as the same user.

    Any ideas?!
     
  20. cvidler

    cvidler Member

    Joined:
    Jun 29, 2001
    Messages:
    16,145
    Location:
    Canberra
    SELinux.

    try, setenforce 0 (to temporarily disable SELinux).
    see if the problem goes away if it does, you probably want to do a relabel.

    edit: did you install bind via yum? it should've put the right labels on the folders for everything to work.
     

Share This Page

Advertisement: