Logging of Excessive Data

Description

Logging of Excessive Data is a security vulnerability where software logs an excessive amount of information, making log files difficult to process and potentially obscuring important security events. While logging is essential for security monitoring and forensic analysis, excessive logging can be counterproductive. Too much log data creates noise that hinders administrators' ability to detect anomalous conditions, provides cover for attackers by burying their activities in irrelevant entries, complicates debugging, and consumes significant system resources including disk space and CPU time.

Risk

Excessive logging creates multiple security and operational risks. Attackers can hide malicious activity within massive log volumes, knowing that administrators are unlikely to review every entry. Large log files consume disk space rapidly, potentially causing denial of service when storage is exhausted. Processing excessive logs consumes CPU resources, degrading system performance. During security incidents, forensic analysis becomes extremely difficult when relevant events are buried in noise. Real-time security monitoring systems may miss actual attacks while processing irrelevant data. Log retention policies become difficult to enforce with oversized files.

Solution

Implement appropriate logging levels for each environment—verbose debug logging in development, minimal essential logging in production. Suppress duplicate log messages by summarizing repeated entries (e.g., "last message repeated X times"). Configure maximum log file sizes with rotation and archival. Implement log sampling for high-volume events. Use structured logging to enable efficient filtering. Define clear policies for what should and shouldn't be logged at each severity level. Monitor log volume as a metric and alert on unusual spikes. Ensure logging configuration can be easily adjusted without code changes. Consider using separate log streams for different purposes (security events, debugging, audit trails).

Common Consequences

Impact	Details
Availability	Scope: Availability DoS: Resource Consumption - Excessive logging consumes disk space and CPU resources, degrading system performance.
Accountability	Scope: Accountability, Non-Repudiation Hide Activities - Large log volumes provide cover for attackers, hiding malicious activity in noise.
Integrity	Scope: Integrity Reduced Forensic Value - Oversized logs become impractical for security analysis and incident response.

Example Code

Vulnerable Code

// Vulnerable: Logging every request in detail
public class VulnerableRequestHandler {
    private static final Logger logger = Logger.getLogger(VulnerableRequestHandler.class);

    public void handleRequest(HttpServletRequest request) {
        // Vulnerable: Logs complete request details for every request
        logger.info("Request received from: " + request.getRemoteAddr());
        logger.info("Request method: " + request.getMethod());
        logger.info("Request URI: " + request.getRequestURI());
        logger.info("Query string: " + request.getQueryString());

        // Logs all headers
        Enumeration<String> headers = request.getHeaderNames();
        while (headers.hasMoreElements()) {
            String header = headers.nextElement();
            logger.info("Header " + header + ": " + request.getHeader(header));
        }

        // Logs all parameters
        for (Map.Entry<String, String[]> param : request.getParameterMap().entrySet()) {
            logger.info("Parameter " + param.getKey() + ": " + Arrays.toString(param.getValue()));
        }

        // In high-traffic environment, this generates gigabytes of logs
        processRequest(request);
    }
}

# Vulnerable: Logging in tight loops
import logging

logger = logging.getLogger(__name__)

def vulnerable_process_data(large_dataset):
    # Vulnerable: Logs every item in large dataset
    for i, item in enumerate(large_dataset):
        logger.debug(f"Processing item {i}: {item}")  # Millions of log entries
        logger.debug(f"Item details: {item.get_all_attributes()}")
        logger.debug(f"Item metadata: {item.metadata}")

        result = process_item(item)

        logger.debug(f"Processed item {i}, result: {result}")
        logger.debug(f"Memory usage after item {i}: {get_memory_usage()}")

    # For 1 million items, this creates 5+ million log entries

// Vulnerable: Excessive error logging
void vulnerable_network_handler(int sockfd) {
    char buffer[1024];
    ssize_t bytes;

    while ((bytes = recv(sockfd, buffer, sizeof(buffer), 0)) >= 0) {
        // Vulnerable: Logs every received packet
        syslog(LOG_INFO, "Received %zd bytes from socket %d", bytes, sockfd);
        syslog(LOG_DEBUG, "Buffer contents: %.*s", (int)bytes, buffer);

        if (bytes == 0) {
            // Vulnerable: Logs every empty read (common with non-blocking I/O)
            syslog(LOG_WARNING, "Empty read on socket %d", sockfd);
        }

        process_data(buffer, bytes);
    }

    // Vulnerable: Logs every error, including transient ones
    if (bytes < 0) {
        syslog(LOG_ERR, "Error reading socket %d: %s", sockfd, strerror(errno));
    }
}

// Vulnerable: Verbose middleware logging
const express = require('express');
const app = express();

// Vulnerable: Logs everything including sensitive data
app.use((req, res, next) => {
    console.log(`[${new Date().toISOString()}] ${req.method} ${req.url}`);
    console.log('Headers:', JSON.stringify(req.headers, null, 2));
    console.log('Body:', JSON.stringify(req.body, null, 2));
    console.log('Query:', JSON.stringify(req.query, null, 2));
    console.log('Cookies:', JSON.stringify(req.cookies, null, 2));

    // Capture response too
    const oldWrite = res.write;
    const oldEnd = res.end;
    const chunks = [];

    res.write = function(chunk) {
        chunks.push(chunk);
        oldWrite.apply(res, arguments);
    };

    res.end = function(chunk) {
        if (chunk) chunks.push(chunk);
        console.log('Response:', Buffer.concat(chunks).toString('utf8'));
        oldEnd.apply(res, arguments);
    };

    next();
});

Fixed Code

// Fixed: Appropriate logging with levels and summarization
public class FixedRequestHandler {
    private static final Logger logger = Logger.getLogger(FixedRequestHandler.class);
    private static final AtomicInteger requestCount = new AtomicInteger(0);
    private static final int LOG_SAMPLE_RATE = 100;

    public void handleRequest(HttpServletRequest request) {
        int count = requestCount.incrementAndGet();

        // Fixed: Only log summary periodically
        if (count % LOG_SAMPLE_RATE == 0) {
            logger.info("Processed " + count + " requests");
        }

        // Fixed: Debug level for detailed info (disabled in production)
        if (logger.isDebugEnabled()) {
            logger.debug("Request: " + request.getMethod() + " " + request.getRequestURI());
        }

        try {
            processRequest(request);
        } catch (Exception e) {
            // Fixed: Only log actual errors with relevant context
            logger.error("Request failed: " + request.getMethod() + " " + request.getRequestURI()
                        + " - " + e.getMessage());
        }
    }
}

# Fixed: Controlled logging with sampling and summarization
import logging
from collections import Counter

logger = logging.getLogger(__name__)

def fixed_process_data(large_dataset):
    total_items = len(large_dataset)
    processed_count = 0
    error_count = 0
    error_summary = Counter()

    # Fixed: Log start and end, not every item
    logger.info(f"Starting processing of {total_items} items")

    for i, item in enumerate(large_dataset):
        try:
            result = process_item(item)
            processed_count += 1

            # Fixed: Progress logging at intervals
            if (i + 1) % 10000 == 0:
                logger.info(f"Progress: {i + 1}/{total_items} items processed")

        except ProcessingError as e:
            error_count += 1
            error_summary[type(e).__name__] += 1

            # Fixed: Only log first few errors of each type
            if error_summary[type(e).__name__] <= 3:
                logger.warning(f"Processing error: {e}")

    # Fixed: Summary at end
    logger.info(f"Completed: {processed_count}/{total_items} successful, {error_count} errors")
    if error_summary:
        logger.warning(f"Error summary: {dict(error_summary)}")

// Fixed: Rate-limited error logging
#include <time.h>

static time_t last_empty_log = 0;
static int empty_count = 0;
static time_t last_error_log = 0;
static int error_count = 0;

void fixed_network_handler(int sockfd) {
    char buffer[1024];
    ssize_t bytes;

    while ((bytes = recv(sockfd, buffer, sizeof(buffer), 0)) >= 0) {
        if (bytes == 0) {
            empty_count++;
            time_t now = time(NULL);
            // Fixed: Rate limit empty read logging
            if (now - last_empty_log >= 60) {  // Log at most once per minute
                if (empty_count > 1) {
                    syslog(LOG_DEBUG, "Empty reads on socket %d: %d in last minute",
                           sockfd, empty_count);
                }
                last_empty_log = now;
                empty_count = 0;
            }
            continue;
        }

        process_data(buffer, bytes);
    }

    if (bytes < 0) {
        error_count++;
        time_t now = time(NULL);
        // Fixed: Rate limit error logging
        if (now - last_error_log >= 10) {  // Log at most every 10 seconds
            syslog(LOG_ERR, "Socket %d: %d errors, last: %s",
                   sockfd, error_count, strerror(errno));
            last_error_log = now;
            error_count = 0;
        }
    }
}

// Fixed: Minimal production logging
const express = require('express');
const app = express();

// Fixed: Configurable logging levels
const LOG_LEVEL = process.env.LOG_LEVEL || 'info';

app.use((req, res, next) => {
    const start = Date.now();

    res.on('finish', () => {
        const duration = Date.now() - start;

        // Fixed: Single line per request, essential info only
        const logData = {
            method: req.method,
            path: req.path,
            status: res.statusCode,
            duration: duration + 'ms'
        };

        // Fixed: Error level for problems, info for normal
        if (res.statusCode >= 500) {
            console.error('Request error:', JSON.stringify(logData));
        } else if (LOG_LEVEL === 'debug') {
            console.log('Request:', JSON.stringify(logData));
        }
    });

    next();
});

---

CVE Examples

• CVE-2007-0421: Server logged excessive data when receiving malformed headers, consuming disk space.
• CVE-2002-1154: Application failed to restrict update access, allowing attackers to fill error logs with excessive entries.

---

References

1. MITRE Corporation. "CWE-779: Logging of Excessive Data." https://cwe.mitre.org/data/definitions/779.html
2. OWASP. "Logging Cheat Sheet." https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
3. NIST. "Guide to Computer Security Log Management." SP 800-92.