prompt-optimize

Deterministic prompt compression for LLM applications. Reduce token count and API costs with zero network calls, zero model inference, and zero runtime dependencies.

---

Description

prompt-optimize applies a configurable pipeline of rule-based optimization passes to prompt text, producing a shorter prompt that conveys the same instructions to the model. Every transformation is deterministic: the same input always produces the same output. No API keys, no Python runtime, no external services.

The package targets a concrete problem: unnecessary tokens in LLM prompts cost real money at scale. A 500-token system prompt called 1 million times per day at $3.00/MTok costs $1.50/day. If 30% of those tokens are filler words, verbose phrasing, and redundant whitespace, that is over $160/year in waste from a single prompt.

prompt-optimize ships eight optimization passes organized into two safety tiers (safe and moderate), a protected-region system that ensures code blocks, template variables, URLs, quoted strings, and negation contexts are never modified, and a detailed report showing per-pass token savings.

---

Installation

npm install prompt-optimize

Requires Node.js >= 18.

---

Quick Start

import { optimize } from 'prompt-optimize';

const result = optimize(
  'Please just provide the answer in order to complete the task.'
);

console.log(result.optimized);
// "Provide the answer to complete the task."

console.log(result.report.savings);
// {
//   charactersBefore: 61,
//   charactersAfter: 41,
//   tokensBefore: 16,
//   tokensAfter: 11,
//   tokensSaved: 5,
//   percentage: 31.3
// }

---

Features

• Eight optimization passes covering whitespace normalization, comment stripping, filler word removal, verbose phrase replacement, preamble stripping, and redundancy merging.
• Three safety levels (safe, moderate, aggressive) controlling which passes run, with moderate as the default.
• Protected regions automatically detected and preserved: fenced code blocks, inline code, template variables ({{var}}, {var}, ${var}, %(var)s), URLs, quoted strings, HTML tags, and sentences containing negation words.
• Detailed optimization reports with per-pass breakdown, token savings, percentage reduction, and timing.
• Custom token counter support for plugging in accurate tokenizers (e.g., tiktoken, gpt-tokenizer).
• Reusable optimizer instances via createOptimizer() for applying the same configuration across multiple prompts.
• Zero runtime dependencies. Pure TypeScript, compiles to CommonJS.
• Full TypeScript support with exported types and declaration maps.

---

API Reference

optimize(input, options?)

Runs all applicable optimization passes on input and returns the optimized string plus a detailed report.

Parameters:

Parameter	Type	Description
input	string	The prompt text to optimize.
options	OptimizeOptions	Optional configuration (see Configuration).

Returns: OptimizationResult

import { optimize } from 'prompt-optimize';

const result = optimize('Please kindly summarize the document in order to save time.', {
  safety: 'moderate',
});

console.log(result.optimized);  // "Summarize the document to save time."
console.log(result.original);   // "Please kindly summarize the document in order to save time."
console.log(result.report);     // Full OptimizationReport

---

analyze(input, options?)

Runs the same pipeline as optimize() but returns only the OptimizationReport. Use this to preview potential savings without needing the optimized text wrapper.

Parameters:

Parameter	Type	Description
input	string	The prompt text to analyze.
options	OptimizeOptions	Optional configuration.

Returns: OptimizationReport

import { analyze } from 'prompt-optimize';

const report = analyze('Please just provide the answer in order to complete the task.');

console.log(report.savings.tokensSaved);  // 5
console.log(report.savings.percentage);   // 31.3
console.log(report.passes.filter(p => p.applied).map(p => p.name));
// ['Trailing Spaces', 'Whitespace Normalize', 'Filler Words', 'Verbose Phrases', ...]

---

createOptimizer(options)

Creates a reusable optimizer instance bound to a fixed set of options. Useful when optimizing multiple prompts with the same configuration.

Parameters:

Parameter	Type	Description
options	OptimizeOptions	Configuration to bind to the optimizer instance.

Returns: Optimizer

import { createOptimizer } from 'prompt-optimize';

const optimizer = createOptimizer({ safety: 'safe' });

const result1 = optimizer.optimize('First prompt with   extra   spaces.');
const result2 = optimizer.optimize('Second prompt with   extra   spaces.');
const report  = optimizer.analyze('Third prompt for analysis only.');

---

Exported Types

SafetyLevel

type SafetyLevel = 'safe' | 'moderate' | 'aggressive';

Controls which optimization passes are applied. Each level includes all passes from lower levels.

---

OptimizeOptions

interface OptimizeOptions {
  safety?: SafetyLevel;
  tokenCounter?: (text: string) => number;
}

Field	Type	Default	Description
safety	SafetyLevel	'moderate'	Controls which passes run.
tokenCounter	(text: string) => number	Math.ceil(text.length / 4)	Custom function for counting tokens.

---

OptimizationResult

interface OptimizationResult {
  optimized: string;
  original: string;
  report: OptimizationReport;
}

Field	Type	Description
optimized	string	The compressed prompt text.
original	string	The original, unmodified input.
report	OptimizationReport	Full report with savings and per-pass breakdown.

---

OptimizationReport

interface OptimizationReport {
  timestamp: string;
  durationMs: number;
  safetyLevel: SafetyLevel;
  savings: TokenSavings;
  passes: PassResult[];
}

Field	Type	Description
timestamp	string	ISO 8601 timestamp of when the optimization ran.
durationMs	number	Total duration of the optimization in milliseconds.
safetyLevel	SafetyLevel	The safety level that was applied.
savings	TokenSavings	Aggregate savings summary.
passes	PassResult[]	Per-pass breakdown of results.

---

TokenSavings

interface TokenSavings {
  charactersBefore: number;
  charactersAfter: number;
  tokensBefore: number;
  tokensAfter: number;
  tokensSaved: number;
  percentage: number;
}

Field	Type	Description
charactersBefore	number	Character count of the original text.
charactersAfter	number	Character count of the optimized text.
tokensBefore	number	Estimated token count of the original text.
tokensAfter	number	Estimated token count of the optimized text.
tokensSaved	number	Number of tokens saved (always >= 0).
percentage	number	Percentage of tokens saved (0--100).

---

PassResult

interface PassResult {
  id: string;
  name: string;
  safetyLevel: SafetyLevel;
  applied: boolean;
  skipReason?: string;
  tokensSaved: number;
  changeCount: number;
  durationMs: number;
}

Field	Type	Description
id	string	Unique pass identifier (e.g., 'filler-words').
name	string	Human-readable pass name (e.g., 'Filler Words').
safetyLevel	SafetyLevel	The safety tier this pass belongs to.
applied	boolean	Whether the pass made any changes.
skipReason	string | undefined	Reason the pass was skipped (e.g., 'safety-level').
tokensSaved	number	Tokens saved by this pass (always >= 0).
changeCount	number	Number of individual changes made.
durationMs	number	Time spent executing this pass in milliseconds.

---

ChangeEntry

interface ChangeEntry {
  passId: string;
  original: string;
  replacement: string;
  offset: number;
  line: number;
  column: number;
}

Represents a single text change made by an optimization pass.

---

Optimizer

interface Optimizer {
  optimize(input: string): OptimizationResult;
  analyze(input: string): OptimizationReport;
}

A reusable optimizer instance created by createOptimizer().

---

Configuration

Safety Levels

Level	Passes Included	Use Case
safe	trailing-spaces, empty-lines, whitespace-normalize, comment-strip	Automated pipelines with no human review. Only touches whitespace and inert content.
moderate	All safe passes + filler-words, verbose-phrases, preamble-strip, redundancy-merge	Production use with brief human review. Removes words and phrases that do not affect LLM behavior.
aggressive	All moderate passes + future high-aggression passes	Reserved for future structural transformation passes.

Custom Token Counter

The built-in token counter estimates tokens as Math.ceil(text.length / 4), which is accurate to within 10--15% for typical English text with BPE tokenizers. For exact counts, provide a custom counter:

import { optimize } from 'prompt-optimize';
import { encoding_for_model } from 'tiktoken';

const enc = encoding_for_model('gpt-4o');

const result = optimize('Please just provide the answer in order to complete the task.', {
  tokenCounter: (text) => enc.encode(text).length,
});

console.log(result.report.savings.tokensSaved); // Exact token savings

---

Optimization Passes

Safe Passes

These passes are guaranteed to preserve semantic meaning. They only modify whitespace and inert content.

ID	Name	Description
trailing-spaces	Trailing Spaces	Removes trailing spaces and tabs from each line.
empty-lines	Empty Lines	Collapses 4+ consecutive empty lines to 2. Removes leading and trailing blank lines.
whitespace-normalize	Whitespace Normalize	Normalizes line endings to \n. Collapses 3+ blank lines to 2. Collapses multiple mid-line spaces to one (preserving indentation).
comment-strip	Comment Strip	Removes HTML comments (<!-- -->), line-leading // comments, and line-leading # comments. Cleans up resulting blank lines.

Moderate Passes

These passes remove filler content that empirical testing shows has no effect on LLM output quality. Review recommended.

ID	Name	Description
filler-words	Filler Words	Removes standalone filler words: please, kindly, just, simply, basically, actually, certainly, of course, obviously, you know.
verbose-phrases	Verbose Phrases	Replaces wordy phrases with concise equivalents. See the phrase table below.
preamble-strip	Preamble Strip	Strips common AI preamble openers (e.g., "Certainly! Here is...", "I'd be happy to...", "You are a helpful assistant.").
redundancy-merge	Redundancy Merge	Detects near-duplicate sentences using Jaccard similarity (threshold > 0.85) and removes subsequent duplicates.

Verbose Phrase Replacements

Verbose Phrase	Replacement
in order to	to
due to the fact that	because
at this point in time	now
in the event that	if
for the purpose of	to
with regard to	about
in the process of	while
make sure to	ensure
it is important to note that	(removed)
it should be noted that	(removed)
please note that	(removed)
as previously mentioned	(removed)
as stated above	(removed)
needless to say	(removed)
at the end of the day	(removed)

---

Protected Regions

The following regions are automatically detected and never modified by any optimization pass, regardless of safety level:

Region Type	Pattern	Example
Fenced code blocks	``` ... ```	Code examples with language tags
Inline code	`...`	Inline code references
Template variables	{{var}}, {var}, ${var}, %(var)s	Placeholder values
URLs	https?://...	Links and endpoints
Quoted strings	"...", '...'	Exact values and names
HTML tags	<tag ...>	Structural markup
Negation contexts	Sentences containing: not, never, don't, must not, do not, cannot, shouldn't, won't, no	Safety-critical instructions

Negation protection is particularly important. A sentence like "Please do not include personal data" is left entirely intact to prevent any risk of disrupting the negation.

---

Error Handling

prompt-optimize is designed to be safe by default:

• Empty input: Passing an empty string returns it unchanged with zero savings reported.
• Already-optimal input: If no passes produce changes, the original text is returned with tokensSaved: 0 and percentage: 0.
• All-protected input: If the entire input consists of protected regions (e.g., a single code block), no modifications are made.
• Invalid safety level: The safety option accepts only 'safe', 'moderate', or 'aggressive'. Any other value will cause a runtime error when filtering passes.
• Token counter errors: If a custom tokenCounter function throws, the error propagates to the caller. Ensure your counter handles all string inputs.

All functions are synchronous and throw no errors for valid inputs. There are no async operations, no network calls, and no file I/O.

---

Advanced Usage

Analyzing Before Optimizing

Use analyze() to preview savings without generating the optimized text:

import { analyze } from 'prompt-optimize';

const report = analyze(systemPrompt, { safety: 'moderate' });

if (report.savings.percentage > 20) {
  console.log(`Prompt has ${report.savings.percentage}% waste — consider optimizing.`);
}

for (const pass of report.passes) {
  if (pass.applied) {
    console.log(`  ${pass.name}: saved ${pass.tokensSaved} tokens in ${pass.durationMs}ms`);
  }
}

Safety Level Comparison

Compare savings across safety levels to make an informed decision:

import { analyze } from 'prompt-optimize';

const prompt = loadPrompt();

const safeSavings = analyze(prompt, { safety: 'safe' }).savings;
const moderateSavings = analyze(prompt, { safety: 'moderate' }).savings;

console.log(`Safe:     ${safeSavings.tokensSaved} tokens (${safeSavings.percentage}%)`);
console.log(`Moderate: ${moderateSavings.tokensSaved} tokens (${moderateSavings.percentage}%)`);

Batch Optimization

Use createOptimizer() to process multiple prompts with the same configuration:

import { createOptimizer } from 'prompt-optimize';

const optimizer = createOptimizer({ safety: 'moderate' });

const prompts = loadAllPrompts();
let totalSaved = 0;

for (const [name, text] of Object.entries(prompts)) {
  const result = optimizer.optimize(text);
  totalSaved += result.report.savings.tokensSaved;
  console.log(`${name}: ${result.report.savings.percentage}% reduction`);
}

console.log(`Total tokens saved: ${totalSaved}`);

CI/CD Integration

Fail a build if any prompt exceeds a token waste threshold:

import { analyze } from 'prompt-optimize';

const MAX_WASTE_PERCENT = 15;

for (const promptFile of promptFiles) {
  const text = fs.readFileSync(promptFile, 'utf-8');
  const report = analyze(text, { safety: 'moderate' });

  if (report.savings.percentage > MAX_WASTE_PERCENT) {
    console.error(
      `${promptFile}: ${report.savings.percentage}% token waste exceeds ${MAX_WASTE_PERCENT}% threshold`
    );
    process.exit(1);
  }
}

Using with Accurate Tokenizers

For production cost accounting, plug in an accurate tokenizer:

import { createOptimizer } from 'prompt-optimize';
import { encode } from 'gpt-tokenizer';

const optimizer = createOptimizer({
  safety: 'moderate',
  tokenCounter: (text) => encode(text).length,
});

const result = optimizer.optimize(systemPrompt);
console.log(`Exact tokens saved: ${result.report.savings.tokensSaved}`);

---

TypeScript

prompt-optimize is written in TypeScript and ships with full type declarations (.d.ts) and declaration maps. All public types are exported from the package entry point:

import type {
  SafetyLevel,
  ChangeEntry,
  PassResult,
  TokenSavings,
  OptimizationReport,
  OptimizationResult,
  OptimizeOptions,
  Optimizer,
} from 'prompt-optimize';

The package compiles to ES2022 CommonJS with source maps enabled. TypeScript strict mode is enforced.

---

License

MIT