One line question, one line answer, no fluff. These are pulled from across the handbook to make recall fast. Use the filter to jump to a topic, or hit Hide answers to quiz yourself and click any question to reveal its answer.
Fundamentals
QWhat is machine data?
The text systems write as they run, such as logs, metrics, and traces.
QWhat does schema on read mean?
Splunk stores raw data and works out the structure at search time, not at ingest.
QWhat three things is Splunk always doing?
Collecting data, storing it in indexes, and searching it.
QWhat language searches Splunk?
The Search Processing Language, or SPL.
Data pipeline
QWhat are the four pipeline stages?
Input, parsing, indexing, and search.
QWhat metadata is set at input?
Source, host, and sourcetype.
QWhat happens at parsing?
The raw stream is broken into events, timestamps are found, and transforms run.
QWhich stage writes data to buckets?
Indexing.
QWhen are most field extractions applied?
At search time.
QWhy prefer search-time work over index-time?
Search-time settings are flexible and can change without re-indexing.
QWhich file controls inputs?
inputs.conf.
QWhich files control parsing?
props.conf and transforms.conf.
Components
QWhat does an indexer do?
Parses, indexes, stores, and searches data.
QWhat does a search head do?
Runs searches and dashboards by sending work to the indexers and merging results.
QWhat is distributed search?
A search head splitting a query across many indexers in parallel, then merging results.
QWhat is a search peer?
An indexer that a search head runs its searches against.
QWhat does the cluster manager do?
Coordinates an indexer cluster and tracks data copies. It does not index data.
QWhat does the deployer serve?
Apps and base config to search head cluster members.
QWhat does the deployment server serve?
Configuration to forwarders.
QWhat does the license manager track?
Daily ingest volume against your license.
QWhat is the Monitoring Console?
The built-in health dashboard for the whole deployment.
QWhat was the cluster manager called before?
The master node.
Topologies
QWhat are the three deployment shapes?
Single instance, distributed, and clustered.
QWhen do you use a single instance?
Labs, proofs of concept, and small or light workloads.
QWhat does a distributed non-clustered setup lack?
Automatic recovery when an indexer fails.
QWhat does clustering give you?
High availability through multiple data copies and synced search content.
QWhat is an SVA?
A Splunk Validated Architecture, a tested reference design.
QWhat does multi-site protect against?
The loss of a whole site or data center.
Forwarders
QWhat is a universal forwarder?
A small, light agent that collects and forwards data with almost no parsing.
QWhat is a heavy forwarder?
A full Splunk instance set to forward that can parse and route data.
QWhich forwarder is the default choice?
The universal forwarder.
QWhen do you use a heavy forwarder?
For API inputs or heavy content-based filtering and routing.
QWhich file sets where a forwarder sends data?
outputs.conf.
QWhat does auto load balancing do?
Spreads a forwarder's data across all indexers in its target group.
QWhat does useACK do?
Makes the forwarder wait for the indexer to confirm receipt before dropping data.
QWhat is an intermediate forwarder?
A forwarding tier between sources and indexers, often to cross a network boundary.
QDefault receiving port for forwarded data?
9997.
Deployment server
QWhat is a deployment client?
A forwarder that checks in and pulls its assigned apps.
QWhat is a server class?
A rule mapping a group of clients to a set of deployment apps.
QWhich file maps clients to apps?
serverclass.conf.
QWhich file points a forwarder at its deployment server?
deploymentclient.conf.
QWhere do deployment apps live on the server?
In etc/deployment-apps.
QDoes the deployment server manage cluster peers?
No. The cluster manager does that.
Inputs and parsing
QWhy prefer TCP over UDP for network inputs?
UDP can silently drop data under load.
QWhat is HEC?
The HTTP Event Collector, which lets apps send events over HTTP with a token.
QRecommended pattern for syslog?
Land it on a syslog server, write to files, and read the files with a forwarder.
QWhat does SHOULD_LINEMERGE control?
Whether Splunk merges multiple lines into one event.
QWhat is the most reliable event-breaking setting?
LINE_BREAKER.
QWhat does TIME_FORMAT set?
The exact format of the timestamp in the data.
QWhere must index-time parsing settings live?
On the parsing tier: indexers or heavy forwarders.
Indexer clustering
QWhat is replication factor?
How many total copies of each bucket the cluster keeps.
QWhat is search factor?
How many of the copies are searchable.
QCommon default replication and search factor?
Replication factor 3, search factor 2.
QHow many peers can a cluster lose safely?
Replication factor minus 1.
QCan search factor exceed replication factor?
No. A copy cannot be searchable if it does not exist.
QHow is shared config sent to peers?
The cluster manager pushes a bundle with apply cluster-bundle.
QHow does a search head find clustered peers?
It gets the peer list from the cluster manager.
Search head clustering
QWhat is the captain?
The member that coordinates the cluster and replicates changes.
QIs the captain a separate machine?
No, it is a role one member takes on.
QHow is the captain chosen?
By a majority vote among members.
QWhy an odd number of members?
So a majority can always form and two halves cannot both elect a captain.
QMinimum members for a real SHC?
Three.
QWhat command bootstraps the first captain?
splunk bootstrap shcluster-captain, run on one member.
QWhat command adds a search head to an SHC?
splunk init shcluster-config, then join the cluster. Not by editing conf files.
QWhen is a static captain used?
During an outage when the cluster cannot form a majority to elect one.
QShould the deployer be a cluster member?
No, it must sit outside the cluster.
Indexes and buckets
QWhat is a bucket?
A directory of events from a time span, plus the index files to search them.
QWhat are the bucket stages?
Hot, warm, cold, frozen, and thawed.
QWhich stage is being written to?
Hot.
QWhat happens at frozen by default?
The bucket is deleted unless you archive it.
QWhich file controls retention?
indexes.conf.
QWhich setting freezes data by age?
frozenTimePeriodInSecs.
QHow do you keep frozen data?
Set coldToFrozenDir or a coldToFrozenScript.
QWhy split data into separate indexes?
Access control and retention are set per index.
SmartStore
QWhat does SmartStore change?
It keeps the master copy of data in remote object storage and caches locally.
QWhy use SmartStore?
To scale storage and compute separately instead of adding indexers just for disk.
QWhat becomes the local disk?
A cache for the working set of data.
QRecommended network to the object store?
10 Gbps for good performance.
Config and precedence
QWhere do your changes go?
In a local folder, never in default.
QWhy never edit default?
Upgrades overwrite the default folder.
QGeneral precedence order, highest first?
System local, app local, app default, system default.
QWhat does btool show?
The final merged settings that Splunk will actually use.
QWhat does the --debug flag add to btool?
The exact file each winning setting came from.
QWhat defines a stanza?
A name in square brackets, with key = value pairs under it.
QHighest precedence in an indexer cluster peer?
Config pushed from the cluster manager to the peer-app location.
Sizing
QHow does Splunk scale?
Out, by adding machines to a tier, not up with one big server.
QWhat drives indexer count?
Daily ingest volume plus search load, sized for peak.
QWhat drives search head count?
Number of users and scheduled searches.
QShould you size from averages?
No, size for peak load.
QWhat measures storage speed?
IOPS, input and output operations per second.
Security
QWhich file sets how users log in?
authentication.conf.
QWhich file sets roles and access?
authorize.conf.
QWhat access model does Splunk use?
Role-based access control.
QHow do you limit who sees sensitive data?
Put it in its own index, since access is granted per index.
QHow do you hide secrets before storage?
Mask at index time with SEDCMD or transforms.
QShould the two clusters share a secret key?
No, use different keys for the indexer and search head clusters.
Operations and commands
QWhere is the splunk command?
In $SPLUNK_HOME/bin.
QDefault install path on Linux?
/opt/splunk, or /opt/splunkforwarder for a universal forwarder.
QHow do you start, stop, and restart?
./splunk start, stop, and restart.
QHow do you make Splunk start at boot?
./splunk enable boot-start.
QCommand to watch a file?
./splunk add monitor /path.
QMonitor versus oneshot?
Monitor keeps watching a live source; oneshot reads a file a single time.
QTurn on receiving on an indexer?
./splunk enable listen 9997.
QWhere do Splunk internal logs go?
The _internal index, including splunkd.log.
QCheck indexer cluster health from where?
The cluster manager, with ./splunk show cluster-status.
QWhere do you run add monitor?
On the machine that reads the data, usually a forwarder.