Splunk

Splunk is available as a free download that lets you log 500MB of data. Enterprise versions start at $5,000.

One of a systems administrator’s most tedious tasks is digging through log data to determine the cause of a problem. This kind of manual searching is time intensive and error prone. A better solution is an  search engine such as Splunk. But Splunk is to say the least unimpressive as a help in log analysis: on large volumes it is slow on small volumes it is redundant.  In both cases help that it provides is trivial.

The whole idea of Splunk is too much ado about nothing: to create a framework for using Python based modules in pipes to process stream of log data in the same way as typical admins uses Unix filters. But somehow due to relentless marketing they managed to stay afloat. 

Michael Baum, Splunk CEO and co-founder claims that his company created a search engine that lets you search and report on logs and data from any application, server, or network device similar to what Google can so on the Web.  But this analogy is wrong. Syslog records are more like database then the Web.

It is not completely useless. Splunk 3.0 does contain some useful features such as structured analysis and reporting. Splunk’s  reporting capability lets you use the product’s built-in library of graphs, charts, and reports to analyze the data that Splunk gathers.

The new scripting feature provides an easy method for porting data to Splunk. Users can write simple command-line or shell scripts that call another program, then send the output to Splunk for indexing.

NEWS CONTENTS

• 20090704 : People Over Process " Splunk and CA Unicenter Cross Launching Into Systems Management Innovation ( 2006-05-02 , Redmonk )

Old News ;-)

[Jul 4, 2009] People Over Process " Splunk and CA Unicenter Cross Launching Into Systems Management Innovation

2006-05-02 | Redmonk

Pareto Principle Coverage for Systems Management

First, we see Unicenter+Splunk pitched as a two complimentary products. That certainly makes sense. As we outlined in our briefing notes on Splunk, they're specifically not about "higher level" systems management, like service tracking or even complex event handling. Splunk is squarely focused on log searching and the implied event indexing, while CA is more concerned with those higher level functions. So, you have the sysadmin tool (Splunk) and the management tool (CA):

Mell Estrada (CA): Well, from the CA perspective, we are very excited about the opportunity to work with Splunk because, like Michael [Baum, CEO of Splink] was saying, we have two very good technology sets. Splunk brings interesting indexing and search capabilities and CA brings up the large breadth of monitoring and management capabilities and bringing them together and integrating them in a - at least, at this point in a - what we call a partnership perspective, we are saying and letting the world know that we are taking the heavy investing out of the customer's hands; we, as proactive vendors in this space with world class technologies, are coming together in a proactive manner to help them manage their data and their data centers in a much more efficient manner.

There're two interesting points here:

1. the emphasis on addresses the issue of complexity in tool way rather than the usual systems management pixie dust
2. I suppose the "at least, at this point in a" is a tease about a possible acquisition?

And then to the Splunk perspective:

Michael Baum (Splunk): IT organizations [have] become a lot more sophisticated about dealing with that complexity and the problems they deal with as things get more complex and most IT organizations, large IT organizations today have done a really good job with help of products like Unicenter, at building a work flow, building a set of infrastructure around dealing with finding problems and alerting on those problems in the infrastructure. What I've seen in my past, and what we're trying to attack together with CA is, okay, now that we're really good at that, what about the problems that don't get solved quickly because it's not an obvious - there's a single component failure, or there's a piece of hardware down somewhere; now you have to go investigate.

The last part is the key: neither Unicenter nor Splunk will be a "complete" enterprise systems management solution. You need both the "dad/management" platform that does all the "boring" stuff like workflow, dashboards, etc., and the tool platform that's used for trouble shooting the internals and "low-level" issues. What's refreshing is seeing a member of The Big 4 accept that duality and doing something about it instead of focusing on the higher level issues and pixie-dusting away the low-level problems.

On Splunk's end, this is also a kinder, more enterprisey way of Splunk positioning itself with The Big 4, rather than saying "those folks make things more complex than they need to be." I can also imagine that Splunk devs (or, my dev-gut tells me, all of Splunk as everyone there seems "dev") are happy about this deal because it means they don't need to worry about writing and supporting dashboards, reports, ITIL driven workflows, and all that other stuff that can pushed off to CA now, and future partners I hope.

Contextual Launch & Back-end Integration

Michael Baum: The first piece is a integration at the user level with the NSM product, where you can launch a Splunk search on any of the components that you see in the NSM console. The second part is, we have tied into the Event API underneath Unicenter. And we are streaming events from the Event Database inside of Unicenter into the Splunk world so that they can be indexed and searched and navigated along with all of the other data that you might want to index with Splunk.

This is interesting as it shows a sort of "non-log" use of Splunk. It also gives Splunk equal footing with Unicenter as far as raw and, perhaps, even synthesized data: if Splunk is sucking data out of Unicenter, then Splunk theoretically has access to all the data that Unicenter does…meaning sysadmins could spend more time in Splunk than in Unicenter if they prefer Splunks's UI. That's just wild speculation. I'll have to check out a demo of the integration to know for sure.

Collaborative Systems Management

Michael Baum: [F]or Unicenter itself, which is a product that needs to be managed, certainly its useful to apply the notion of Splunk Base and intelligence about managing the Unicenter product itself but Unicenter - the particular product we are talking about here, the NSM product collects all kinds of data, right, there is all sorts of things that they are able to drive into their event database. So, you could imagine very deep connection with Splunk Base where you are looking at some data that's coming across an SNMP port; maybe it's a type of MIB information from a device - you have never seen it before, you don't really understand what it is. Well, there are probably a couple of thousand people around the world that know what it is and if they have taken the opportunity document that in the Wiki on Splunk Base, its something you can have direct access to.

Since I'm big time on Collaborative Systems Management, this is the most interesting aspect about the partnership to me. Indeed, we advised CA to build up more community involvement and help facilitate more user-driven content generation. I'd like to see CA take full advantage of this Splunk partnership and more directly integrate with Splunk Base. Part of that integration is getting involved in the growing Splunk Base community: for example, CA's internal IT could start uploading newly found events in Splunk Base and getting involved in describing those events. From the on- and off-line discussions I've had with CA folks, I know they're interested in doing more community building, and Splunk Base is a great place to start learning the ins and outs of working with and within a community.

Of course, it might be wise for Splunk to keep Splunk Base to itself, least CA "take over" that great idea. Unless that wouldn't be a problem down the line… Also, pulling in other Big 4 vendors would hedge against one vendor/user base dominating Splunk Base.

Recommended Links

SplunkBase - The Splunk Application Place

Slashdot | SplunkBase Brings IT Troubleshooting Wiki to the Masses

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: January 09, 2020