ISAM Agent
ISAM Agent is an application that can be deployed to produce and maintain a replica of the system of record data on another system. It subscribes to the CDC topic, consumes change messages, and reproduces those changes in local SDMS data files.
The ISAM Agent is a .NET 10 application and can be deployed on Windows and Linux.
Startup
When ISAM Agent starts, it first loads the configuration it needs to run, opens its local state database, restores its saved node identity, and brings up its secure management endpoint. As part of that startup, it also reloads the last known replication state, including the snapshot the node was seeded from and the last saved CDC position. Before it begins reading live change traffic, the agent checks whether the local replica is still trustworthy enough to continue from where it left off.
If the local node is already hydrated, the startup check is straightforward. The agent verifies that all managed replica files are still present and readable, and it confirms that the saved CDC offset is still valid in Kafka. When those checks succeed, no snapshot restore is needed. The agent treats the local replica as current enough to resume, keeps the existing files in place, and prepares to continue from the stored CDC position rather than rebuilding the replica.
If missing or inconsistent data is detected, the agent switches into recovery mode. This happens when the node has never been seeded, when required replica files are missing or unreadable, when no saved CDC offset exists, or when the saved offset is no longer usable because it has fallen outside the range Kafka can still provide. In that case, the agent looks for the most recent recoverable counted snapshot. If the latest available snapshot is too old or otherwise invalid, it requests a new snapshot instead. Before restoring, it invalidates the current managed replica files, either deleting them or retaining renamed copies depending on configuration. It then restores the snapshot, records that snapshot as the new seed source, and saves the snapshot’s CDC offset as the new resume point.
After either path, the agent knows exactly where live replication must begin. If it resumed an already hydrated node, that is the previously saved CDC offset. If it restored from a snapshot, that is the CDC offset embedded in the snapshot, and the agent may first need to catch up from that point to the current end of the CDC stream. Once that starting position is established, the agent assigns the configured CDC topic and partition, positions itself at the chosen offset, and enters its main CDC processing loop. If the startup validation finds a non-recoverable problem, such as replica files that are locked or inaccessible in a way that prevents safe recovery, the agent stops instead of entering the loop.
Main Processing
Once the ISAM Agent enters its main processing loop, it attaches to the configured CDC topic and partition at the starting offset chosen during startup. From that point on, it waits for incoming change messages and handles them one at a time so replica updates stay orderly. If the agent started from a restored snapshot and had backlog to catch up on, it first replays those older CDC messages until it reaches the previously recorded end of the stream, then continues directly into normal live replication.
For each message, the agent reads the CDC payload, interprets the operation type, and applies the change to the corresponding local data file. Insert messages create records, update messages replace existing records, and delete messages remove records. Transaction boundary messages such as start and commit are recognized but do not cause any file changes on the replica. The agent uses the file mapping configuration to determine which local file should be updated and which key to use when locating the target record.
The loop also handles several out-of-sync conditions without stopping replication. If an insert arrives for a record that already exists, the agent logs a warning and switches to an update. If an update arrives for a record that does not exist, it logs a warning and switches to an insert. If a delete arrives for a record that is already missing, it logs a warning and skips the delete. When an update or delete finds that the current local record does not exactly match the “original” data captured in the CDC message, the agent still proceeds, but it records a warning that the local copy had already drifted.
After each message is applied, the agent saves the next CDC offset to its local state so it can resume from the correct place after a restart. Messages that are malformed or missing required record content are logged and skipped, allowing the agent to move on. If a message fails in a way that throws an actual processing error, the failure is logged, and the saved offset is not advanced for that message. The loop continues running until the agent is paused, stopped, or shut down, at which point consumption is canceled and the consumer is closed cleanly.
Shutdown
The ISAM Agent is usually started as a Windows Service or a Linux systemd service, so shutdown is initiated through the applicable service manager, for example, by stopping the Windows service or using systemctl stop. In those cases, the agent performs a graceful shutdown rather than treating the stop as a failure. Its status changes to Stopping, it stops accepting new work, and then it closes down its replay and management services in an orderly way. Once shutdown has completed normally, the status becomes Shutdown.
The same graceful path is used when shutdown is requested from the management interface. In the management UI, the Stop action sends a stop request to the agent, and the UI reflects the transition into Stopping and then Shutdown. The management API does the same thing through the stop endpoint, so automated administration tools can shut the agent down cleanly without logging on to the host. In all of these cases, the intent is to let the current in-flight work finish and then stop before taking another CDC message.
During shutdown, the agent stops waiting for new CDC traffic, cancels the replay loop, and closes the Kafka consumer cleanly. It does not perform a separate drain or final catch-up pass. Instead, it relies on the replication position it saves as messages are successfully processed. That means the next startup resumes from the last saved CDC position. In normal use, this prevents already-completed work from being lost and allows any not-yet-completed message to be retried after restart.
If the agent is started manually in a console, such as by a developer during testing or troubleshooting, pressing Ctrl+C triggers the same orderly shutdown sequence. That is primarily a manual-start scenario rather than the normal production model. For production deployments, the recommended approach is to stop the agent through Windows Service Control Manager, systemd, the management UI, or the management API so the process is given the opportunity to shut down cleanly.
Management UI
The ISAM Agent includes a built-in management UI that gives an administrator a secure, browser-based view of the agent’s health, activity, and configuration. Access is protected by the agent’s admin password, and once signed in, the administrator can move between status, logging, file, activity, snapshot, and administrative pages from a single dashboard. Because the UI is designed for day-to-day operational use, it focuses on visibility first, while still exposing a small set of management actions directly from the browser.
The main status page acts as the operator dashboard. It shows the current agent state, whether replication is paused, how many operations have been processed, how many errors have been recorded, when activity was last seen, which Kafka brokers and topics are in use, and which snapshot the node was seeded from. From that page, an administrator can refresh the view on a timer, pause or resume replication, and stop the agent. This makes it the primary screen for confirming that the replica is healthy and actively processing CDC traffic.
The UI also provides several pages for monitoring the replica in more detail. The log page shows recent log output in near real time, supports color-coded severity display, and allows the operator to pause the live log view, limit the number of entries shown, clear the local browser view, and download the visible log text. The file status page shows each managed replica file and whether it is present, along with details such as the local path, file size, record count, and creation date/time. The activity page provides a view of recent CDC messages from Kafka, with filtering, ordering, adjustable message counts, and payload inspection. For record-bearing messages, the operator can open the current and original record payloads and see byte-level differences highlighted, which is useful when investigating replication issues or data drift.
Snapshot handling is surfaced through its own page. The UI lists available snapshot files, identifies the newest snapshot, shows file sizes and timestamps, supports filtering and downloading the visible snapshot list, and allows older snapshots to be deleted. In the current implementation, the page includes a button for requesting a new snapshot, but that action is not yet wired up in the UI. Separate administrative pages allow for changing the admin password, adjusting logging levels by category, exporting the agent’s protected encryption key ring for secure backup, and reviewing configured file mappings. When enabled by configuration, the UI can also expose an authenticated host terminal in a separate browser window. If Swagger is enabled, the interface also includes a direct link to the API documentation.
Most of the functionality of the management UI is implemented by browser-side calls to the agent’s authenticated management API. The pages largely act as a client for endpoints that return status, file state, snapshot listings, recent Kafka activity, and log settings and control actions such as pause, resume, and stop. Live log streaming is delivered through the agent’s SignalR log endpoint, and the optional host terminal uses its authenticated WebSocket endpoint. Because of that design, the built-in UI is not the only way to manage or observe the agent: the same protected interfaces can be used to build custom management consoles, health checks, dashboards, and monitoring tools tailored to a customer’s environment.
Management API
The ISAM Agent exposes a management API on its administrative HTTPS endpoint. It is the same interface used by the built-in management UI, and it covers authentication, agent control, operational status, file inspection, activity inspection, log management, snapshot management, and key export. If Swagger is enabled in configuration, the API can also be explored through the protected Swagger UI.
At a high level, the API surface includes these groups of functions:
- Authentication
- Agent control
- Status and monitoring
- Additional management interfaces
Security
The management API (and therefore the management UI) is protected in several layers. First, it is hosted on the agent’s TLS-enabled administrative endpoint. The agent configures Kestrel with its local TLS certificate and will not start normally without the certificate and password needed to load it. In practice, this means management traffic is intended to run over HTTPS, and browser session cookies are marked Secure, so the browser will only send them over encrypted connections.
Second, nearly all management routes require authentication. The controller-based management endpoints are decorated with [Authorize], and the same authorization boundary also protects the browser UI pages, the SignalR log hub, and the WebSocket terminal. The main anonymous exception is the login endpoint, POST /api/auth/admin-token, which accepts the admin password and issues a session. Swagger, when enabled, is not public documentation; it is also behind the same authentication boundary.
The admin password itself is not stored in plain text. It is kept in the agent’s local encrypted store, and the agent decrypts it only to validate login attempts. That encrypted storage is backed by the agent’s Data Protection key ring. On Windows, the key ring can also be protected to the local machine with DPAPI, which means that even if the key files are copied elsewhere, they still may not be usable off host. The API includes an export function for that key ring specifically so it can be backed up securely. Because those keys protect access to locally stored secrets, the key export endpoint is highly sensitive and should be treated accordingly.
When login succeeds, the agent creates a signed JSON web token (JWT) with an expiry time controlled by AdminLoginDurationMinutes in configuration. The token contains an admin identity and is signed with an HMAC SHA-256 key derived locally at startup from agent-specific secret material, rather than from a hard-coded shared secret. On every protected request, the agent validates the token’s issuer, audience, signature, and lifetime, with only a small clock-skew allowance. If validation fails, the request is rejected.
The API supports two ways of presenting that JWT. Programmatic clients can send it in the standard Authorization: Bearer ... header. Browsers usually rely on the isamagent_auth cookie that is set at login time. That cookie is HttpOnly, Secure, and SameSite=Strict, which means browser scripts cannot read it directly; it is only sent over HTTPS, and it is less exposed to cross-site request scenarios. The authentication middleware will accept the bearer token directly, or, if no bearer token is present, it will read the JWT from that secure cookie instead.
Unauthorized handling is also split appropriately between browser and non-browser clients. For normal API callers, an invalid or missing token results in a 401 Unauthorized error. For browser-style GET and HEAD requests that ask for HTML, the agent redirects the user to the login page instead. That allows the UI to behave like a web application while keeping the API usable for scripts and custom tools.
There is also one special-case protection path for LaunchPad integration. The LaunchPad ticket endpoint is not anonymous: it requires an already valid bearer token, and it only works from loopback (localhost). It issues a short-lived, one-time login ticket, which is then redeemed locally to create the normal authenticated browser session. This limits that convenience path to trusted local use.
In short, the management API is protected by HTTPS transport, authenticated admin login, signed and expiring JWTs, secure browser cookies, authorization on all sensitive routes, and encrypted local secret storage. Custom management and monitoring tools can use the API safely, but they should do so as authenticated administrative clients and should protect returned tokens and exported key material as privileged secrets.
Configuration
The ISAM Agent supports two bootstrap modes: Configuration Server mode and local configuration mode. At startup it first decides which mode to use by looking for a ConfigServer URL, first on the command line (--config-server), then in environment variables (SIP_CONFIG_SERVER), and finally in appsettings.json (ConfigurationServerUrl). It resolves the profile name the same way, using --config-profile, SIP_CONFIG_PROFILE, or ProfileName, and defaults to the default profile if none is supplied. Those settings only tell the agent how to bootstrap itself; the agent’s actual operating configuration comes from either ConfigServer or the local profile files.
When a ConfigServer URL is available, the agent starts in ConfigServer mode. It opens its local bootstrap store, restores or creates its ConfigServer client credentials, registers with ConfigServer if needed, authenticates, and then downloads the profile’s MasterConfig.json and the server’s ServerConfig.ISAMAgent.json. If the local TLS certificate and password are not already present, it also requests those from ConfigServer so the management endpoint can run over HTTPS. Any configuration files successfully downloaded from ConfigServer are written to the local profile cache, so the node has a last-known-good copy available for future startups.
If ConfigServer is not being used, the agent runs in local configuration mode. In that case, it loads MasterConfig.json and ServerConfig.ISAMAgent.json directly from the local profile directory. On Windows that profile cache lives under %ProgramData%\Synergex\SIP\<profile>, and on Linux it lives under /etc/synergex/sip/<profile>. Those files may have been created automatically during an earlier successful ConfigServer bootstrap, or they can be placed and maintained there manually. In other words, a site that does not use ConfigServer can still run the agent entirely from local configuration files.
If ConfigServer is configured but unavailable, the agent falls back to those local cached files instead of failing immediately. It attempts to load any missing configuration from the cached MasterConfig.json and ServerConfig.ISAMAgent.json in the profile directory, which allows offline startup using the most recently downloaded settings. If the required local files are missing, unreadable, or incompatible, the fallback cannot succeed, and startup stops with a fatal configuration error. The same principle applies to the TLS materials: the agent can reuse locally installed certificate files, but if no valid certificate and password are available and they cannot be obtained from ConfigServer, the agent cannot continue.
Periodic Services
ISAM Agent supports Periodic Services which allows you to configure one or more recurring background tasks that run on a cron-style schedule alongside normal agent processing.