Warning: this is a rant. Turn away now if you are easily offended. The last three years have left me scratching my head. I'm at a complete loss as how to explain the complete lack of quality and professionalism in an industry I used to be proud of. I make a living as a sysadmin. What does that mean, to be a sysadmin? Well, where I come from it means knowing a lot. It means knowing how to config routers and networking equipment, it means advanced firewalling, DNAT, SNAT, it means knowing how to do traffic sniffing and deciphering packet-level information, it means knowing how to build and configure common services like SMTP/IMAP/POP/mail via a dozen different pieces of software on three different families of operating systems, it means knowing how to build clusters for high availability and high performance, it means knowing when to use CIFS, NFS, SMB, GFS and when not to and what the difference is between them all, it means knowing hwo to configure iSCSI, fibre channel, SANs, direct and non-direct storage, it means knowing SQL and getting information out of databases, it means knowing how to program in a dozen different languages and how to script and automate events in any OS to make life easier, it means understanding authentication and security settings, how to configure any directory service from LDAP to AD to NIS, it means understanding DNS is more than just a optional addon to look up system names occasionally, it means understanding encryption, knowing what terms like Diffie Hellman, AES, SHA1 and others mean, and what parts of the encryption process they apply to, it means being able to make everything you do completely redundant and fault tolerant, right down to you own job, and it means so much more. Why is it then, that over the last three years I've seen fewer and fewer people who call themselves sysadmins understand these things? Why is it that I've been surrounded by "IT professionals" from junior sysadmins to CTOs who don't have a goddamn clue about one tenth of the above? Why is it that in three years I've met ONE person in professional IT who I would consider worthy of sitting down and having a conversation with? Why is it that professional IT services today consist of service reps who tell you the things you are doing are untested, dangerous, unsupported, different, not usual, or a host of other words meaning they are scared shitless and unwilling to learn something new? Why is it that I spend my time building things people tell me for 6 months during build and test "will never work", only to have them go into production and work ten times faster for one tenth the cost of the old system? Why is it that IT professionals today choose brand labels over intelligence, and post-justify it by hiding behind "board confidence" when providing a solid, working, profitable system is the best thing to boost confidence from the board? I tried switching industries. I've done IT in engineering, architecture, film and TV, retail, medical, finance and superannuation. Some of the places I've worked for have been fortune 500s. Some have been Fortune 5s. Did the quality of the IT staff go up? No. Was I met with people who were open minded and willing to learn new things to better the workplaces they were in? No. Was I met with fear, close-mindedness, and nothing but people who rattle off marketing bullshit as an excuse for not knowing technical information? Yes. And every time I leave, I hear the same things. Some new guy comes in to replace me. Within days/weeks he's broken something necessary for production, lost terabytes of data, destroyed the backup/DR/recovery systems, spent hundreds of thousands replacing something that met the businesses' every need with some proprietary/generic piece of rubbish that performs half as well when there were dozens of other things that could have been improved instead. And all because they didn't take the time to understand the business, it's needs, and the solutions currently in place. My latest job is no different. I've walked into a place that holds and controls financial data for over 6 million Australians, and around 50 million Americans. A place where data integrity is paramount, and system stability and performance is of the highest regard. The hardware is provided by a tier 1, namebrand hardware provider (number 2 worldwide in server sales, I hear). The support guys who come on site are paid absolute buckets of cash and are supposedly the best of the best. These guys come out and utterly bollocks up installs. They constantly tell you things are impossible to achieve, only to stare slack-jawed in amazement three weeks later when they are achieved and working faster than their setups were supposed to provide. They rant and spit when I build things for zero-dollar licensing cost that their multi-hundreds-of-thousands-of-dollar hardware is supposed to be the only stuff that can do the job (my latest GFS/CLVM cluster outperforms their SAN snapshotting, and is free of charge compared to their pay-a-license-per-snapshot "solution"). And of course, their golden trump card is to say "well that's fine, but we don't support it" when you offend them. Watch the CIOs scramble when their hardware vendors threaten to not offer support! Yet ask them when they last called on the "professional" support (other than simple break/fix/replace stuff), and most can't answer. I'm disgusted. I'm pissed off. Quite frankly, I'm over IT. I don't consider myself smarter than the average bear, and I don't consider that I have higher expectations than is realistic. I expect that "professional" IT people are professional. I expect that they have a desire to learn, a technical competence to achieve the tasks they set out to do, and a constant need to push the envelope of what's achievable and always move forwards. I expect everyone to have a goal to leave something in a better/faster/more efficient way then they found it. Yet it seems that the last few years have shown people in IT are by and large the complete and polar opposite. Don't get me wrong - I'm not some gung ho cowboy. I've met plenty of those (and sacked a few along the way). Being conservative with sensitive material is always a smart option. But letting it deteriorate is utter ignorance. Many moons ago, I used to have a mentor. A man who quite frankly I considered genius level. I don't throw around words like "genius" frequently. In my life I've met three people who would rightly qualify as geniuses. Only one I've had the pleasure to work with, and more importantly learn from. In the small amount of time I worked with the man my rate of learning tripled. He had the right amount of sage advice coupled with the sense to let you make your own mistakes from time to time. Sadly the company in question got bought out, and the new owners were typical of all of my criticisms above. Within three months 50% of the IT staff left (myself and my mentor were two of the first). Within 9 months they'd spent 10 times our annual budget on a variety of completely unnecessary infrastructure, and completely ignorant and underqualified consultants (all of whom got the work via personal ties to the new owners, of course), and the company was brought to the brink of destruction. From what I hear this week, they'll be liquidated by the end of the year. So when did this happen? When did "the IT guy" turn from the person who was cross trained with the breadth and depth of knowledge across a wide variety of systems and procedures turn into a drivelling half-wit who sees more value in a commercial certification than actually learning and building things, and who decides to be "the Microsoft guy" or "the UNIX guy" or "the Cisco guy" and learns nothing but one brand-name item to the ignorance of all others, and often poorly because they can't separate concepts and ideas from brand names and marketing acronyms? When the hell did professional IT people stop being professional? I've had a gut full. Something must come of this. The industry as a whole is in for a rude shock if it keeps going the way it does. We keep packing IT departments full of more people who know less. Things break constantly because unqualified people manage them, and departments stop communicating because the connecting technologies are always "somebody else's problem". The industry gets flooded with cowboys who have no concept of system and data integrity, who don't take care with the systems they are put in charge of, who don't bother securing things in a proper fashion so that data doesn't leak everywhere. It's almost a daily event to hear of some horrendously scary security breech that affects millions of innocent people who put their trust in these idiots. Will there be a crash? Will there be a bottom to this rapidly declining curve? Will it get to the point where IT is just so shithouse that the people relying on it start to demand a certain level of competence? I look at other professional industries - lawyers, doctors, engineers, architects. All of these people need to maintain a certain level of education and prove a certain minimum of competence to practice their arts. My guess is if the same thing happened in IT, 2/3 of the workers would be out the door, and rightly so before they destroy something important. Maybe I just need to get the hell out of this industry. I keep getting offered roles that are "bigger and better", but they just lead to paperwork hell. I want to stay technical and hands-on, but at the same time am just sick to death of the ignorance and stupidity I see every waking hour in this profession. Were it not for mortgage and children, I would have been out half a decade ago. But maybe it's seriously time to begin working on an exit strategy. And of course, it doesn't help at all that I've been reading "Atlas Shrugged" these last few weeks. If ever there was a book to convince you to say "fuck the world", that's it. If you've read this far, good for you. If you think I'm some sort of uppity, pompous, self-righteous know it all, then you really don't know me. But flame away all the same. Like everyone else, you'll give me a list of reasons why I expect too much, why not everyone is good enough to know everything, blah blah blah. A long list of excuses as to why mediocrity is acceptable and nobody should strive for anything other than average. Your comments will be filed away with the rest of them. End rant.