Understanding Pulse Opensearch Data

Pulse uses Opensearch (formerly Elasticsearch) to store and query high-frequency monitoring data. This article explains what data is stored in Opensearch, how it is organised, and how to manage it.

What Is Stored in Opensearch

Opensearch stores all the second-by-second monitoring data that Pulse captures from TM1 instances. This includes:

Index Type	Data Stored

Index Type	Data Stored
Server State	CPU, memory, disk, and TM1 service state — captured every second
Message Log	TM1 server log entries
Process History	TM1 TurboIntegrator process execution history
Chore History	TM1 chore execution history
Session Log	User session activity
User History	Historical record of TM1 user actions
Object History	History of changes to TM1 objects
Alerts	Record of triggered Pulse alerts

Index Naming and Frequency

Opensearch data is organised into indices. The frequency of index creation (daily, monthly, yearly) is controlled by pattern settings in Pulse.cfg under [Elastic]. For example:

ServerStateIndexPattern = yyyy-MM
MessageLogIndexPattern = yyyy-MM
ChoreProcessHistoryIndexPattern = yyyy

A yyyy-MM pattern creates one index per month. A yyyy pattern creates one index per year. More granular patterns (e.g. yyyy-MM-dd) create more indices but make individual index sizes smaller.

Managing Opensearch Index Size

As Opensearch stores second-by-second data, it can grow quickly. Manage its size using the Pulse Maintenance job. In Pulse.cfg:

[Maintenance]
ElasticHistoricYears = 1
ElasticHistoricMonths = 3
ElasticHistoricDays = 7

These settings tell Pulse how many years, months, and daily indices to retain. Indices older than these thresholds are deleted during the scheduled maintenance run.

Changing the Opensearch Data Drive

By default, Opensearch stores its data in <Pulse Install Directory>\opensearch\data. On environments with limited disk space on the Pulse drive, this can be moved to another drive. See Storing the Pulse Opensearch Data into Another Drive for steps.

Accessing Opensearch Data

Pulse Opensearch data is visualised through the Pulse Explorer (Opensearch Dashboards). To access it, navigate to:

http://<PulseServer>:8095

You can also query indices directly using the Opensearch REST API on port 8097.

Opensearch vs Relational Database

Storage	What It Holds	Best For

Storage	What It Holds	Best For
H2 / SQL Server / Postgres	Configuration, documentation, change tracking metadata, migration packages	Structured queries, Pulse web client features
Opensearch	Second-by-second monitoring data, logs, process history	Time-series queries, Pulse Explorer dashboards