Welcome to timelion the timeseries expression interface for everything

Timelion. Timeline. Get it? Ok, enough with the puns. Timelion is the, clawing, gnashing, zebra killing, pluggable timeseries interface for everything. If your datastore can produce a timeseries, then you have all of the awesome power of Timelion at your disposal. Timelion lets you compare, combine and combobulate (not actually a word) datasets across multiple data sources, even entirely different technologies, all with the same easy-to-master expression syntax. While the beginning of this tutorial will focus on Elasticsearch, once you're rolling you'll discover you can use nearly everything you learn here with any datasource timelion supports.

Why start with elasticsearch? Well, you're using timelion, so we know you have Kibana, so you definitely have Elasticsearch. So the answer is: Because its easy. Timelion want everything to be easy. Ok, lets do this thing. If you're already familar with Timelion's syntax, Jump to the function reference, otherwise click the Next button in the lower right corner.

First time configuration

First you'll need to edit your timelion.json to tell timelion where to fetch data. If you're using logstash (and the default logstash-* pattern), you're already done. Otherwise, update the timefield and default_index parameters to match your environment. Look for this section:

    "es": {
      "timefield": "@timestamp",
      "default_index": "logstash-*",
      "allow_url_parameter": false
    },

You'll see some other parameters in there too, we won't be messing with them for now, but this file can be used to configure much of timelion's default behavior. That said, almost everything in this file can also be specified on-the-fly with the timelion expression syntax. More on that in a bit.

Could not validate elasticsearch config: {{es.invalidReason}}. Check timelion.json, restart Kibana, and try again. ({{es.invalidCount}})

Good news Elasticsearch is configured correctly!

Or at least, things look ok. I validated the elasticsearch url, default index and time field in your timelion.json and everything looks ok. Given your settings I found data between {{es.stats.min}} and {{es.stats.max}}. You're probably all set. If this doesn't look right, Click here for instructions on configuring the elasticsearch data source.

Intervals

You might already have one nice chart, but I'm going to operate on the assumption you don't for educational purposes. The input bar at the top has two inputs. On the left, is your expression, leave that alone for now, we'll get to it. On the right is the interval selector, which is currently set to {{state.interval}}. Looks good! Set it to auto. If timelion thinks your combination of time range and interval will produce too many data points it will throw an error. You can configure that limit in timelion.json

Time range

Now see that clock icon in the top right? Click it and select a time period that includes all or part of the time range in the first paragraph above. If you didn't before, you should now have a line chart with a count of your data points over time.

Elasticsearch querying in short

We're going to start off talking about the Elasticsearch datasource, because we've already validated that one works for you. Enter .es(*) in the expression input, if its not there already. Hit enter.

This said "hey elasticsearch, find everything in my default index". If you wanted to find a subset you might do something like .es(html) to count events matching html, or .es('user:bob AND bytes:>100') to find events with bob in the user field, and a bytes field that is greater than 100. Note that we surrounded our query in single quotes this time, because it has spaces. You can enter any lucene query string as the first argument to the .es() function.

Passing arguments

Timelion has a number of shortcuts for doing common things, one of which is that for simple arguments, ones that don't contain spaces or special characters, you don't need quotes. Many functions also have defaults, for example .es() and .es(*) do the same thing. Arguments also have names, so you don't have to remember their position, you can pass .es(index='logstash-*', q='*') to tell the elasticsearch data source "use * as the q (query) for the logstash-* index"

Beyond count

Counting events is all well and good, but the elasticsearch data source also supports any Elasticsearch metric that returns a single value. Min, max, avg, sum and cardinality are some of the most useful. Lets say you want a unique count of the src_ip field. You could do say, .es(*, metric='cardinality:src_ip'). To get the average of the bytes field you would run: .es(metric='avg:bytes').

Expressions and expressing yourself

Every timelion expression starts with a datasource function. From there, the sky is the limit and new functions can be appended, or "chained", to the data source to transform and augment it. From here we're going to assume you know something about your data. Feel free to replace the elasticsearch query with something more meaningful to you.

Up until now we've dealt with just the one chart. We're going to experiment, so add a few more. Click the Menu icon in the top right to expand the menu. Then click the Add Chart button.
.es(*) One expression
.es(*), .es(US) Two expressions. Two expressions on the same chart!
.es(*).color(#f66), .es(US).bars(1) Red expression. Let's colorize the first series red instead. Also, instead of lines for 2nd series, we'll have some bars, with a 1 pixel width.
.es(*).color(#f66).lines(fill=3), .es(US).bars(1).points(radius=3, weight=1) Wooo expressions. In the last example we used un-named arguments to color() and bars, which rely on the arguments position in a comma separated list. We can use named arugments to make expressions easier to read and arguments easier to remember.
(.es(*), .es(GB)).points() Also grouped expressions. Groups of expressions can be chained to functions as well. Both series will be shown as points instead of lines.

Data: Transform insert beat boxing

We can make our charts pretty all day, but its time for businessing. As an example exercise, we're going to figure out what percentage some subset of our data represents of the whole, over time. For example, what percentage of my web traffic comes from the US? Lets start with finding all events that contain US: .es('US'). Now, to find that ratio to the whole, we'd need to divide 'US' by everything, try this: .es('US').divide(.es()). Ah, not bad, but of course this provides us with a number between 0 and 1, lets correct that to a percentage: .es('US').divide(.es()).multiply(100). There, now we've divided all US traffic by all worldwide traffic, then multiplied the result by 100 to get a percentage.

Timelion has a number of built in arithmetic functions, such as sum, subtract, multiply and divide, many of which can take a series or a number. There are also other data transformation functions including movingaverage, abs and derivative. In addition there are other view transformation functions than the ones we learned on the previous page. See the function reference for the complete list of transforming, and drawing functions.

Now that you know the syntax, jump over to the Function Reference for detailed info on all of Timelions available functions.

Tip: You can always find this again by clicking the in the menu

Function reference Click a function for details and arguments or return to the tutorial.

.{{function.name}}() {{function.help}}
Argument Name Accepted Types Information
{{arg.name}} {{arg.types.join(', ')}} {{arg.help}}
This function does not accept any arguments. Well that's simple, isn't it?