logpare

Working with AI coding assistants, I kept running into the same wall: context windows are expensive real estate.

When debugging issues, I needed to share log files with Claude or GPT. But logs are incredibly verbose and repetitive. A 10MB log file might contain the same pattern repeated thousands of times with only small variations, timestamps, IDs, values.

The typical solutions, truncating logs or grepping for specific errors, felt imprecise. I was either losing important context or spending tokens on redundant information.

I wanted to keep the structure of log data without the repetition.

After researching log parsing algorithms, I discovered Drain, an algorithm that treats logs as a language and extracts patterns using a parse tree. Instead of seeing the same message 1000 times, you see the pattern once with a count.

logpare implements Drain in TypeScript with a CLI interface. You feed it logs, it outputs compressed patterns that preserve the meaningful structure while eliminating redundancy.

On typical server logs (nginx, application logs), logpare achieves 10-50x compression while preserving the patterns that matter for debugging.

• Algorithm selection matters.Drain's parse tree approach is specifically designed for log patterns, making it far more effective than generic compression.
• CLI-first design. Making it work seamlessly in terminal workflows (pipes, file arguments) was as important as the compression itself.
• MCP integration opens new possibilities. Adding an MCP server means Claude can use logpare directly, compressing logs before analyzing them.