The Log Pipeline That Makes Federal Security Auditable
AWS-Native Security Operations Series | Part 3 of 6
How CloudTrail, CloudWatch, Kinesis, OpenSearch, and S3 work together to capture, process, and preserve the evidence your authorization depends on.
Detection tells you something happened. The log pipeline tells you what happened, when it happened, what led up to it, and what followed. In a federal environment, that distinction isn't academic. It's the difference between an incident you can investigate and report on, and an incident you can only acknowledge.
The FedRAMP question 'can you produce the logs from that date' is not hypothetical. We've had that conversation with customers and with assessors. The answer depends entirely on whether the log pipeline was built for durability or built for convenience.
Ours is built for durability.
CloudTrail: the API audit trail
Amazon CloudTrail captures API activity across every AWS service in the customer environment. Every API call, every management event, every data event, across every region and every account in an AWS Organization, with the identity of the caller, the time, the source IP, and the parameters passed.
CloudTrail is the foundational audit record for AWS environments. When an incident occurs, when an assessor asks for evidence, when an insider threat investigation requires reconstruction of activity over time, CloudTrail is the record that answers the question.
In our ConMon deployments, CloudTrail is enabled with multi-region logging, log file validation enabled for tamper detection, and S3 access logging on the CloudTrail bucket itself. We also enable CloudTrail Insights on high-activity accounts to detect unusual API call volumes that can indicate automated attack activity or compromised credentials operating at scale.
Amazon CloudWatch: operational telemetry
CloudWatch collects infrastructure metrics, application logs, and VPC Flow Logs, providing the operational telemetry layer that sits alongside CloudTrail's API audit trail.
VPC Flow Logs captured through CloudWatch give us network-level visibility: what traffic is flowing between resources, what connections are being made to external IPs, what ports are in use. Combined with GuardDuty's analysis of the same flow data, our analysts have both the raw evidence and the interpreted signal.
CloudWatch also provides the metric filters we use for CIS Benchmark alerting: unauthorized API calls, console login without MFA, root account usage, security group changes, network ACL changes, CloudTrail configuration changes, and other events that the CIS AWS Foundations Benchmark identifies as requiring immediate alert. These alerts feed into our SIEM as high-priority findings that bypass standard triage queues.
Amazon Kinesis Data Streams: the throughput layer
Log volume in an active federal AWS environment is substantial. CloudTrail alone in a multi-account organization can generate millions of events per day. CloudWatch adds infrastructure metrics and VPC Flow Log records on top. Without a capable ingestion layer, that volume overwhelms downstream systems or creates processing backlogs that delay detection.
Kinesis Data Streams is our throughput layer. It ingests the combined log feed in real time, handles volume spikes without dropping events, and decouples ingestion from processing so that a spike in CloudTrail events doesn't create a queue in the SIEM.
The decoupling matters operationally. If OpenSearch has a brief processing delay, logs don't get dropped. They're in Kinesis, waiting to be processed. If we need to replay a window of events for investigation or reprocessing, Kinesis retains the stream. That durability is built into the architecture, not added after an incident reveals the gap.
Amazon OpenSearch Service: the SIEM engine
OpenSearch is our SIEM engine. Log data flows from Kinesis into OpenSearch, where our custom detection rules, correlation logic, and analyst dashboards run.
OpenSearch gives us the search and analytics capability to query across billions of log events in near-real time. When an analyst needs to investigate a GuardDuty finding, they can pull every CloudTrail event associated with the relevant IAM identity in the prior 24 hours in seconds. When a ConMon scan surfaces a new finding, the analyst can check whether any related activity appeared in the logs before the finding was detected.
Our detection rules in OpenSearch are environment-specific, not generic templates applied uniformly across customers. The rules are tuned to each customer's baseline behavior, their expected API call patterns, their normal authentication sources, and their typical traffic flows. That specificity is what separates signal from alert fatigue. A rule that fires on every S3 GetObject call in a media company creates noise. The same rule tuned to fire on S3 GetObject calls to buckets with sensitive classifications from unexpected principals creates signal.
Amazon S3: the immutable evidence layer
S3 sits underneath the entire pipeline as the immutable evidence layer. It performs three distinct functions depending on customer requirements.
Immutable log archiving. Raw CloudTrail logs, VPC Flow Logs, and CloudWatch log exports are archived to S3 with Object Lock enabled in COMPLIANCE mode. Object Lock in COMPLIANCE mode means no one, including root account holders, can delete or modify the logs during the retention period. That's the tamper-proof record that satisfies auditor questions about log integrity.
ConMon evidence packages. Scan results, Inspector findings, Config compliance reports, and POA&M supporting evidence are stored in S3 as structured evidence packages. These packages are the artifacts that support monthly ConMon deliverables, annual assessment evidence requests, and incident investigation documentation.
Long-term retention. For customers with extended retention requirements, S3 Glacier handles archival storage where cost efficiency matters more than retrieval speed. Logs that need to be retained for three to seven years for compliance purposes don't need to sit in hot storage. Glacier keeps them accessible when needed at a fraction of the cost.
The log pipeline isn't a compliance checkbox. It's the evidence infrastructure that makes every other security operation defensible. Detection without auditable logs is an observation you can't prove. Detection with auditable logs is a record you can stand behind.
Why durability is a design requirement, not a nice-to-have
Every component of this pipeline is chosen partly for capability and partly for durability. Kinesis doesn't drop events under load. S3 Object Lock prevents tampering. CloudTrail log file validation detects if logs have been modified. Multi-region logging ensures activity in every region is captured, not just the primary region where most engineers look.
That durability is what makes the pipeline trustworthy for federal customers. An assessor asking for a specific date's API activity gets an answer. An incident investigation requiring reconstruction of activity over weeks gets the raw material it needs. A dispute about whether a control was operating during a specific window has an auditable record to consult.
We've worked with customers who had monitoring tools running and still couldn't answer forensic questions about past incidents because their log pipeline had gaps they didn't know about. That's not a tool problem. It's an architecture problem. The pipeline described here is how we make sure it doesn't happen to our customers.
Where FedRAMP 20x, DoW cATO, and OMB M-26-14 converge
FedRAMP 20x raises the bar from periodic compliance reporting to continuous, evidence-driven assurance. That shift makes the log pipeline more than a security operations capability. It becomes the evidence engine that supports machine-readable reporting, ongoing certification, collaborative continuous monitoring, and faster agency risk decisions.
In that model, CloudTrail, CloudWatch, Kinesis, OpenSearch, and S3 are not separate tools. They are the operational control plane for producing current security evidence. API activity, configuration changes, vulnerability signals, incident indicators, and response artifacts can be captured, correlated, retained, and turned into defensible proof without waiting for the next monthly or quarterly reporting cycle.
DoW cATO builds on the same principle from the mission side: cyber operations, assessment, training, active cyber defense, DevSecOps, and ongoing authorization have to operate from the same evidence base. The teams defending the environment, preparing for assessment, responding to findings, and improving workforce readiness should not be working from disconnected exports, stale screenshots, or manually assembled binders. They should be working from a live evidence pipeline that reflects what is actually happening inside the boundary.
OMB M-26-14 makes this architecture even more relevant because it reframes federal logging around two operational outcomes: Continuous Event Monitoring and Threat Hunting, Investigation, Response, and Forensics. In practical terms, that means agencies and their partners need logging infrastructure that can monitor activity in near real time, retain the right historical data, retrieve it quickly, and centralize it across cloud, third-party, IoT, OT, and mission systems when an investigation or response requires it.
This pipeline maps cleanly to those expectations. CloudTrail and CloudWatch provide the CEM signal. Kinesis keeps high-volume event streams moving without losing evidence. OpenSearch gives analysts the searchable, correlated view needed for investigation and response. S3 and Glacier provide the hot, warm, and cold retention layers needed for THIRF, forensics, audit support, and long-term defensibility.
.
FedRAMP 20x, DoW cATO, and OMB M-26-14 all point to the same operating model: stop treating compliance, authorization, logging, and defense as separate activities. Build the pipeline once, operate from it daily, prove from it continuously, and defend from it when it matters.
Next in the series: Part 4: Continuous Compliance, Not Periodic Compliance
https://lz.infusionpoints.com/awsnative
References
The following external references support the FedRAMP 20x, DoW cATO, OMB M-26-14, and AWS logging architecture discussed in this article.
Federal policy and authorization references.
- FedRAMP, “FedRAMP 20x.” https://www.fedramp.gov/20x/
- FedRAMP, “Approaching FedRAMP 20x Assessments.” https://www.fedramp.gov/2026/assessors/fedramp-assessments/20x/
- Office of Management and Budget, “M-26-14: Ensuring Effective and Efficient Agency Logging and Network Visibility to Defend Against Evolving Cyber Threats.” https://www.whitehouse.gov/wp-content/uploads/2026/05/M-26-14-Ensuring-Effective-and-Efficient-Agency-Logging-and-Network-Visibility-to-Defend-Against-Evolving-Cyber-Threats.pdf
- Cybersecurity and Infrastructure Security Agency, “Logging Reference Architecture.” https://www.cisa.gov/resources-tools/resources/logging-reference-architecture
AWS service and architecture references.
- AWS CloudTrail Documentation, “Validating CloudTrail log file integrity.” https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-validation-intro.html
- Amazon S3 Documentation, “Locking objects with Object Lock.” https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html
- Amazon Kinesis Data Streams Documentation. https://aws.amazon.com/documentation-overview/kinesis-data-streams/
- Amazon OpenSearch Service Documentation. https://aws.amazon.com/documentation-overview/opensearch-service/
- AWS Public Sector Blog, “Prepare for FedRAMP 20x with AWS automation and validation.” https://aws.amazon.com/blogs/publicsector/prepare-for-fedramp-20x-with-aws-automation-and-validation/
