Archived Logs - Long-Term Log Retention and Storage
Overview
Section titled “Overview”LogCentral automatically archives older logs to long-term storage, helping you maintain compliance requirements and access historical data while optimizing your storage costs. This article explains how the log archiving system works and how to manage your retention policies effectively.
Understanding Log Archiving
Section titled “Understanding Log Archiving”What is Log Archiving?
Section titled “What is Log Archiving?”Log archiving is the process of moving older logs from active storage (hot storage) to long-term storage (cold storage). This system allows you to:
- Maintain compliance with data retention requirements
- Access historical data when needed for audits or investigations
- Optimize storage costs by using different storage tiers
- Keep your active logs fast by reducing the volume of data in hot storage
How Automatic Archiving Works
Section titled “How Automatic Archiving Works”LogCentral handles log archiving automatically based on your retention policies. Logs move through different storage tiers as they age:
- Hot Storage: Recent logs that are actively searchable and quickly accessible
- Cold Storage: Archived logs stored for long-term retention
The system manages this transition automatically, so you don’t need to manually move logs between storage tiers.
Viewing Storage Statistics
Section titled “Viewing Storage Statistics”Location Storage Analytics
Section titled “Location Storage Analytics”You can view storage statistics and usage patterns for each location in your organization. Navigate to the location details page at /organizations/:orgId/locations/:locationId to access:
- Current storage usage across your locations
- Storage trends showing how your log volume changes over time
- Storage predictions to help you plan for future capacity needs
The storage chart provides visual insights into your log retention patterns and helps you understand how your storage is being utilized.
Understanding Storage Metrics
Section titled “Understanding Storage Metrics”When reviewing your storage analytics, you’ll see information about:
- Total storage used by the location
- Storage growth patterns
- Last log received timestamp
- Active platforms sending logs
This data helps you make informed decisions about retention policies and storage optimization.
Managing Retention Policies
Section titled “Managing Retention Policies”Setting Retention Periods
Section titled “Setting Retention Periods”Retention periods determine how long logs are kept in different storage tiers. Consider these factors when configuring retention:
- Compliance requirements: Industry regulations may mandate specific retention periods
- Operational needs: How far back do you typically need to search logs?
- Storage costs: Longer retention periods increase storage costs
- Data value: Some logs may have more long-term value than others
Location-Level Management
Section titled “Location-Level Management”Storage and retention are managed at the location level, allowing you to:
- Configure different retention policies for different locations
- Monitor storage usage per location
- Enable or disable locations based on your needs
- Delete locations when they’re no longer needed
Access the location details page to manage these settings for each of your log sources.
Accessing Archived Logs
Section titled “Accessing Archived Logs”Cold Search for Historical Data
Section titled “Cold Search for Historical Data”When you need to search archived logs, you’ll use the cold search functionality. Cold search is specifically designed to query logs in long-term storage.
Note: For detailed information about searching archived logs, refer to the Cold Search article. For searching recent logs in hot storage, see the Hot Search article.
When to Use Cold Search
Section titled “When to Use Cold Search”Use cold search when you need to:
- Investigate incidents that occurred outside your hot storage retention period
- Perform compliance audits requiring historical data
- Analyze long-term trends and patterns
- Retrieve logs for legal or regulatory purposes
Keep in mind that cold search may take longer than hot search since archived logs are stored in a different tier optimized for cost rather than speed.
Storage Cost Optimization
Section titled “Storage Cost Optimization”Best Practices
Section titled “Best Practices”To optimize your storage costs while maintaining necessary retention:
- Review your retention requirements regularly: Ensure you’re not keeping logs longer than necessary
- Monitor storage trends: Use the storage analytics to identify unexpected growth
- Configure appropriate retention periods: Balance compliance needs with cost considerations
- Clean up unused locations: Disable or delete locations that are no longer sending logs
Storage Predictions
Section titled “Storage Predictions”The storage chart includes prediction capabilities that help you:
- Anticipate future storage needs
- Plan for capacity increases
- Budget for storage costs
- Identify potential storage issues before they occur
Compliance and Data Retention
Section titled “Compliance and Data Retention”Meeting Regulatory Requirements
Section titled “Meeting Regulatory Requirements”Many industries have specific requirements for log retention:
- Financial services: Often require 7+ years of retention
- Healthcare: HIPAA compliance may mandate specific retention periods
- Government: Various regulations depending on sector and data type
- General business: Consider legal discovery requirements
Configure your retention policies to meet your specific compliance obligations.
Audit Trail
Section titled “Audit Trail”The system maintains audit logs of administrative actions, which can be valuable for:
- Compliance audits
- Security investigations
- Change tracking
- Accountability
Access audit logs through your organization’s administrative interface to review historical actions and changes.
Managing Multiple Locations
Section titled “Managing Multiple Locations”Organization-Wide Storage
Section titled “Organization-Wide Storage”When managing multiple locations across your organization:
- View aggregate statistics: See total storage across all locations
- Compare location usage: Identify which locations generate the most logs
- Standardize policies: Consider consistent retention policies across similar locations
- Monitor individually: Each location can be managed independently
Location Operations
Section titled “Location Operations”For each location, you can:
- View detailed storage statistics
- Monitor live log streaming
- Configure location-specific settings
- Enable or disable log collection
- Delete locations when no longer needed
Navigate to the location details page to access these management capabilities.
Getting Help
Section titled “Getting Help”If you need assistance with:
- Configuring retention policies: Contact your organization administrator
- Understanding storage costs: Review your subscription plan details
- Compliance requirements: Consult with your compliance team
- Technical issues: Reach out to LogCentral support
Remember that proper log retention and archiving are essential for both operational efficiency and regulatory compliance. Regular review of your storage analytics and retention policies helps ensure you’re maintaining the right balance for your organization’s needs.