Splunk - Search Optimization
Splunk already includes the optimization features, analyses and processes your searches for maximum efficiency. This efficiency is mainly achieved through the following two optimization goals −
Early Filtering − These optimizations filter the results very early so that the amount of data getting processed is reduced as early as possible during the search process. This early filter avoids unnecessary lookup and evaluation calculations for events that are not part of final search results.
Parallel Processing − The built-in optimizations can reorder search processing, so that as many commands as possible are run in parallel on the indexers before sending the search results to the search head for final processing.
Analysing Search Optimisations
Splunk has given us tools to analyse how the search optimization works. These tools help us figure out how the filter conditions are used and what is the sequence of these optimisation steps. It also gives us the cost of the various steps involved in the search operations.
Example
Consider a search operation to find the events which contain the words: fail, failed or password. When we put this search query in the search box, the built-in optimizers act automatically to decide the path of the search. We can verify how long the search took to return a specific number of search results and if needed can go on to check each and every step of the optimization along with the cost associated with it.
We follow the path of Search → Job → Inspect Job to get these details as shown below −
The next screen gives details of the optimization that has occurred for the above query. Here, we need to note the number of events and the time taken to return the result.
Turning Off Optimization
We can also turn off the in-built optimization and notice the difference in the time taken for the search result. The result may or may not be better than the in-built search. In case it is better, we may always choose this option of turning off the optimization for only this specific search.
In the below diagram, we use the No Optimization command presented as noop in the search query.
The next screen gives us the result of using no optimization. For this given query, the results come faster without using in-built optimizations.
