Backend Import Process for Rules and Filters from Container Filesystem

[Kbayero]

Describe the feature

Implement an automated backend process to import correlation rules and filters from the container filesystem instead of requiring manual changesets. Currently, every rule or filter modification requires running a database changeset, which slows down development and lacks validation for rule format, allowing malformed rules to be persisted.

Use Case

• Developers can update rules by simply modifying files and rebuilding the image
• CI/CD pipelines can validate rule syntax before deployment
• Rule format errors are caught at import time, not at runtime
• Version control provides full history of rule changes via Git
• Faster iteration cycle for rule development and testing
• Rollback to previous rule versions by deploying previous image
• Consistent rule state across all deployments of the same version

Proposed Solution

1. Filesystem-based Rule Storage:

   • Store rules and filters as YAML files in container image
   • Organize by category: /rules/windows/, /rules/linux/, /filters/, etc.

2. Backend Import Service:

   • On startup, scan filesystem for rule/filter files
   • Validate format and schema before import
   • Import only new or modified rules
   • Log all import actions for audit trail

3. Validation Layer:

   • YAML schema validation for rule structure
   • Field validation (required fields, data types, regex patterns)
   • Dependency checking (referenced fields exist)
   • Reject invalid rules with descriptive error messages

4. Benefits:

   • No more changesets for rule updates
   • Git-based version control for rules
   • Pre-deployment validation in CI/CD
   • Atomic updates (all rules or none)

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change