Auditable architecture makes relevant activity traceable: which person or service performed which action on what object, when, with what result and in which technical context? The answer must be reliable enough for incident response, internal control, customer assurance or a specific regulatory duty.
This does not mean storing every state change forever in an immutable system. Scope, integrity and retention must match the purpose and risk. An oversized audit trail can itself become a security and privacy liability.
This article covers engineering and security practice, not legal advice. Evidence and retention duties vary by sector, contract, data type and processing activity.
Distinguish audit trails, security logs and observability
The categories overlap but solve different problems:
| Data type | Primary purpose | Example |
|---|---|---|
| Audit trail | reconstruct a business or administrative action | role changed, contract approved |
| Security log | detect and investigate an attack or policy violation | failed login, suspicious export |
| Application log | diagnose a technical failure | timeout, exception, retry |
| Metric and trace | observe system behaviour and performance | latency, queue depth, request trace |
They may share infrastructure, but should not automatically share one schema and retention period. A debug log usually has a shorter useful life than a contractual approval record. The OWASP Logging Cheat Sheet explicitly notes that security, process, audit and transaction logs can have different purposes and content.
Start with an evidence matrix
Before selecting an event store, answer these questions for each evidence need:
- What specific purpose justifies collection?
- Which events are needed to answer it?
- Which fields are required rather than merely convenient?
- Who may write, read and export the data?
- How will tampering be prevented or detected?
- How quickly must investigation and export work?
- When does the purpose end and how will the data be deleted?
A matrix prevents “compliance” from becoming a vague reason for permanent full-fidelity logging.
| Purpose | Events | Integrity need | Retention | Approval |
|---|---|---|---|---|
| Account security | login, MFA change, recovery | high | documented from risk | security team |
| Admin actions | role, tenant and export changes | high | legal/contract review | dual control for export |
| Error analysis | technical failure and retry | medium | short, rotating | engineering |
| Business evidence | approval or status change | process-dependent | defined per process | process owner |
Concrete periods belong in the matrix but not as universal advice in an architecture article. They must follow applicable obligations and documented need.
Design a useful audit event
An audit event should provide enough context without copying the entire request or record. Typical fields include:
- stable event ID,
- event type and schema version,
- timestamp from a synchronised time source,
- human or machine actor,
- tenant and affected object,
- action and result,
- correlation or trace ID,
- source, environment and service,
- reason or ticket reference for particularly sensitive actions.
Keep secrets out of logs
Passwords, access tokens, session identifiers, private keys and full payment data do not belong in audit logs. Health information, message bodies and other sensitive personal data should not be copied as payload by default.
A stable internal identifier is often enough. Hashing or pseudonymisation can reduce risk but does not automatically make data anonymous. Free-text fields are especially risky because users and developers can place uncontrolled personal data or secrets in them.
Treat events as untrusted input
A crafted username containing line breaks must not create false log entries. Validate, normalise and encode event data for the target format. Limit field lengths and event types. Internal services can also be faulty or compromised.
A reference architecture for audit trails
A robust pipeline commonly has six layers:
- Producer: the application creates a typed event where the relevant action becomes final.
- Collection: a shared handler or collector validates schema, time and required fields.
- Transport: a queue or reliable delivery mechanism decouples the application and central store where the risk model permits it.
- Storage: a protected repository accepts events and supports defined search and export.
- Detection: rules and alerts identify critical patterns; an unread log does not provide detection.
- Lifecycle: archive, restriction and deletion follow the evidence matrix.
Not every product needs Kafka, a SIEM and WORM storage. A smaller application can start with a separate audit table, restrictive database roles, tested backups and monitored exports. The control level should match the potential impact and be tested.
Integrity requires more than “append-only”
An insert-only application API does not prove that a database administrator cannot alter records. Controls may include:
- separate roles for application, operations and audit review,
- no update or delete rights for the normal application identity,
- a copy in a separate security boundary,
- immutable or time-locked object storage for justified evidence,
- hash chaining or digital signatures for tamper detection,
- recorded and alerted access to audit data,
- regular restore and integrity tests.
The right combination follows the threat model and evidence requirement. “Immutable” should not be only a storage-product label; administrator paths, backups, keys and export procedures are part of the assessment.
Completeness and failure behaviour
The difficult part is often not storage but ensuring that each relevant business result creates the expected audit entry.
Transactions and the outbox pattern
If the business state commits but asynchronous log delivery fails, the trail has a gap. A transactional outbox can store the business change and pending event in the same database transaction. A worker then delivers it. Consumers must tolerate duplicates; a stable event ID supports idempotency.
Fail open or fail closed
What happens if the audit pipeline is unavailable?
- Blocking a high-risk administrative action may be appropriate.
- Blocking a public core function can harm availability.
This decision belongs in the threat model. Queue limits, local buffering, alerts and controlled degradation need to be defined before an incident.
Event sourcing is optional
Event sourcing uses domain events as the leading state record. It can provide a detailed business history but adds schema evolution, projections, replay, migration and privacy concerns. An audit trail can be built without event sourcing. For many SaaS products, a conventional state database with a focused audit log is the simpler architecture.
GDPR and retention
Audit logs can contain personal data such as user IDs, IP addresses and actions. The principles in GDPR Article 5 therefore apply, including purpose limitation, data minimisation, storage limitation, integrity and confidentiality. Article 32 requires technical and organisational measures appropriate to risk.
Practical consequences include:
- document purpose and legal basis per log category,
- collect only necessary identifiers,
- restrict log access more strongly than ordinary product access,
- record exports and review access where appropriate,
- implement fixed review and deletion processes,
- include backups and derived SIEM data in the lifecycle,
- prepare access, erasure and restriction workflows.
The Article 17 right to erasure is not absolute; statutory retention and legal claims can create exceptions. That still does not justify indefinite storage of all audit data.
Do not apply GoBD to every log
Germany's GoBD concern tax-relevant books, records and electronic documents. They do not automatically turn every application or security log into a retained accounting record. Where a system documents tax-relevant transactions, the responsible team and tax advisers should define the exact data set against the current Federal Ministry of Finance GoBD publication.
Test the audit capability
A dashboard full of log lines is not yet reliable evidence. Test regularly:
- Is every defined high-risk action captured?
- Are time, actor, object, outcome and correlation correct?
- Do retries create duplicates or gaps?
- Can input trigger log injection?
- Are secrets and unnecessary personal data excluded?
- Does monitoring detect a pipeline failure?
- Do search and export meet the required time?
- Are read and export operations controlled?
- Does deletion cover archives and backups?
- Can integrity still be verified after a restore?
A lean implementation sequence
- Select five to ten critical actions instead of “everything”.
- Create the evidence matrix and a versioned event schema.
- Implement a central audit handler with tests.
- Separate write, read and export permissions.
- Add alerts for pipeline failure and relevant security events.
- Define retention and deletion before production.
- Run a mock audit: another person reconstructs a real test case using only the intended evidence.
Expand into more processes or more complex storage and SIEM infrastructure only after this core works.
Conclusion
Auditable architecture is not synonymous with event sourcing, Data Vault or indefinite immutable logs. It is a demonstrable chain of deliberately selected events, reliable collection, protected integrity, controlled access and bounded retention.
Good teams begin with the question that an incident or audit must answer. That determines the event, protection, searchability and deletion point. The result provides evidence without becoming another data dump.
Read more
- Secure architecture for SaaS — threat modelling, access control and security by design
- Production-ready B2B SaaS — scaling and privacy in the wider architecture
- Scalable backend systems — messaging and operating models
- Architecture Sprint — a structured architecture review with fixed scope
