All Possible Settings to Configure in inputs.conf File in Your Splunk Instances

Splunk without data is nothing but a body without a soul. After all, we need Splunk to process the large unstructured machine data that is practically impossible for humans to handle manually. In the Data Pipeline phase, Data Input is the first phase to start with. The `inputs.conf` file is the configuration file that manages the Data Input process. This file directs your Splunk instance on how to receive the ingested data. Therefore, it's extremely important to understand the `inputs.conf` file in detail and learn about all the possible settings you can configure within it.

By the end of this post, you will have a solid understanding of all data input configurations Splunk is capable of handling in your production environment. Whether you are dealing with log files, network data, or other machine-generated data, configuring `inputs.conf` effectively will ensure that your Splunk deployment runs smoothly and efficiently.

Splunk Instances Where inputs.conf file Can be Configured

Before we do anything with the inputs.conf file, you should know where the file is located or where you should create the one if not created yet. And, we believe, you must know the Splunk instances or components you can configure the inputs.conf file. Let's explore together.

Location: The inputs.conf file resides in the `$SPLUNK_HOME/etc/system/local/` directory. If you need more granular control per app or environment, you might place it in `$SPLUNK_HOME/etc/apps/<AppName>/local/`.

The inputs.conf file can be configured on various Splunk instances where you need data to be ingested.

1. Splunk Indexer: You can configure data inputs on the indexer to receive data directly from sources or from Splunk Forwarders. The inputs.conf file on the indexer defines the input settings for the data sources it receives.

2. Splunk Heavy Forwarder: You can configure the inputs.conf file on a Heavy Forwarder to specify the data inputs and their settings. The Heavy Forwarder then processes and forwards the data to the indexers based on the configured rules.

3. Splunk Universal Forwarder: You can configure the inputs.conf file on the Universal Forwarder to define the data inputs and their settings. The Universal Forwarder then sends the data to the specified destination based on the configuration.

It's important to note that the specific Splunk instance where you configure the inputs.conf file depends on your Splunk architecture and data flow requirements. In a distributed Splunk deployment, you typically configure data inputs on the Forwarders (Universal or Heavy) and the Indexers. The Forwarders collect data from the sources and send it to the Indexers, while the Indexers receive and store the data for searching and analysis.

Types of Data Input Streams Splunk Supports

Once you know where the inputs.conf file resids on Splunk environment, we thought it is good to know what type of data input streams Splunk supports too before go deeper.

Splunk supports a wide variety of data input streams, allowing you to collect and index data from numerous sources. Understanding these data input types is crucial for configuring your `inputs.conf` file effectively. Here are the primary types of data input streams that Splunk supports:

1. File and Directory Monitoring

Splunk can monitor files and directories to index data as it gets written. This is particularly useful for log files and other continuously updated data sources.

Example Configuration:

[monitor:///var/log/syslog]
index = main
sourcetype = syslog

2. Network Inputs

Splunk can receive data over the network using various protocols.

[tcp://1514]
index = main
sourcetype = tcp_input

[udp://514]
index = main
sourcetype = udp_input

[http]
disabled = 0

3. Scripted Inputs

Scripted inputs enable you to run scripts to collect data from various sources, such as APIs or custom data sources.

Example Configuration:

[script:///path/to/script.sh]
interval = 300
index = main
sourcetype = scripted_input

4. Modular Inputs

Modular inputs allow for more complex data collection mechanisms by defining custom input types using Splunk's modular input framework.

Example Configuration:

[modular://modular_input]
interval = 60
index = main
sourcetype = modular_input

5. Windows Inputs

Splunk supports specific inputs for Windows environments, such as:

Windows Event Logs:

[WinEventLog://Application]
index = wineventlog
sourcetype = WinEventLog:Application

Windows Performance Monitoring:

[perfmon://CPU]
index = perfmon
sourcetype = Perfmon:CPU

6. Other Inputs

Syslog: For capturing syslog data from various network devices.

[udp://514]
index = syslog
sourcetype = syslog

HTTP/HTTPS: For data inputs using REST APIs.

[http://data_input_endpoint]
index = http_input
sourcetype = http_input

Example inputs.conf File

To better understand the structure and configuration options available in the inputs.conf file, let's take a closer look at an example file and explain each section in detail. Here's a sample inputs.conf file:

# Global settings
[default]
host = $decideOnStartup
sourcetype = default

# Monitor a file
[monitor:///var/log/myapp.log]
sourcetype = myapp
index = main
disabled = false

# Monitor a directory
[monitor:///var/log/nginx]
sourcetype = nginx
index = webserver
whitelist = \.log$
blacklist = \.old$
followTail = 0

# TCP input
[tcp://1514]
sourcetype = syslog
index = network
connection_host = dns

# UDP input
[udp://1514]
sourcetype = syslog
index = network
no_appending_timestamp = true

# Scripted input
[script:///path/to/script.sh]
interval = 300
sourcetype = custom
index = scripted
passAuth = admin

# HTTP Event Collector
[http]
disabled = 0
enableSSL = 1
port = 8088
outputgroup = default
useDeploymentServer = 0

# SSL settings
[SSL]
serverCert = /path/to/server.pem
password = password
rootCA = /path/to/ca.pem

# Data preprocessing
[props:my_sourcetype]
LINE_BREAKER = ([\r\n]+)

Let's go through each stanza and its purpose one after anoher:

1. Global settings:

- The [default] stanza defines global settings that apply to all inputs unless overridden by specific input stanzas.

- host = $decideOnStartup sets the host value for events to the hostname of the Splunk instance at startup.

- sourcetype = default sets the default source type for events that don't have a specific source type defined.

2. Monitor a file:

- The [monitor:///var/log/myapp.log] stanza configures Splunk to monitor the /var/log/myapp.log file.

- sourcetype = myapp assigns the myapp source type to events from this file.

- index = main specifies that the events should be stored in the main index.

- disabled = false ensures that the input is enabled.

3. Monitor a directory:

- The [monitor:///var/log/nginx] stanza configures Splunk to monitor the /var/log/nginx directory.

- sourcetype = nginx assigns the nginx source type to events from files in this directory.

- index = webserver specifies that the events should be stored in the webserver index.

- whitelist = \.log$ specifies a regular expression to include only files with a .log extension.

- blacklist = \.old$ specifies a regular expression to exclude files with a .old extension.

- followTail = 0 disables the followTail option, which means Splunk will read the entire file from the beginning.

4. TCP input:

- The [tcp://1514] stanza configures Splunk to listen on TCP port 1514 for incoming data.

- sourcetype = syslog assigns the syslog source type to events received on this port.

- index = network specifies that the events should be stored in the network index.

- connection_host = dns sets the host value for events to the DNS name of the sending host.

5. UDP input:

- The [udp://1514] stanza configures Splunk to listen on UDP port 1514 for incoming data.

- sourcetype = syslog assigns the syslog source type to events received on this port.

- index = network specifies that the events should be stored in the network index.

- no_appending_timestamp = true disables the automatic appending of timestamp and host information to the received events.

6. Scripted input:

- The [script:///path/to/script.sh] stanza configures Splunk to run the script.sh script as a scripted input.

- interval = 300 specifies that the script should be executed every 300 seconds (5 minutes).

- sourcetype = custom assigns the custom source type to events generated by the script.

- index = scripted specifies that the events should be stored in the scripted index.

- passAuth = admin passes the authentication information for the admin user to the script.

7. HTTP Event Collector:

- The [http] stanza configures the HTTP Event Collector (HEC) input.

- disabled = 0 enables the HEC input.

- enableSSL = 1 enables SSL for secure communication.

- port = 8088 specifies the port number for HEC to listen on.

- outputgroup = default sets the default output group for events received via HEC.

- useDeploymentServer = 0 disables the use of a deployment server for HEC configuration.

8. SSL settings:

- The [SSL] stanza configures SSL settings for inputs that support SSL.

- serverCert = /path/to/server.pem specifies the path to the server certificate file.

- password = password sets the password for the private key associated with the server certificate.

- rootCA = /path/to/ca.pem specifies the path to the root CA certificate file.

9. Data preprocessing:

- The [props:my_sourcetype] stanza defines data preprocessing rules for events with the my_sourcetype source type.

- LINE_BREAKER = ([\r\n]+) specifies a regular expression to determine how to split the incoming data into individual events based on line breaks.

This is not the end. There are plethora of options to configure. Don't ignore to check out this Splunk documentation for more granular details.

Remember to place the inputs.conf file in the appropriate directory ($SPLUNK_HOME/etc/system/local/ or $SPLUNK_HOME/etc/apps/<app_name>/local/) and restart Splunk for the changes to take effect.

We hope this article helps understand about the the location, Splunk instances where inputs.conf file to configure, and all possible settings to configure inputs.conf file in your Slunk deployment. 

That's all for now, we will cover more information about the Splunk in the up coming articles. Please keep visiting thesecmaster.com for more such technical information. Visit our social media page on Facebook, Instagram,  LinkedIn, Twitter, Telegram, Tumblr, & Medium and subscribe to receive information like this.