llmboss

getting started

Clone repo:

bun clone 'git+https://git.daemons.my/dab/llmboss.git'

Install backend dependencies:

cd llmboss/backend && bun install

Run server:

bun run llmboss/backend/index.js $PATH_TO_USERS $PATH_TO_AUTHTOKENS $PATH_TO_PROVIDERS

changelog

• Version 3.0.0
  • project name change and cleanup
  • added path_users arg to backend/index.js invocation
  • added simple login page
  • added simple websocket example
• Version 2.0.0
  • project name change and cleanup
  • added handle_request_validate logic for server auth protection
  • removed JSON encode/decode in message passing (req_bytes and resp_success_bytes passed directly)
• Version 1.0.0
  • now functions as a true proxy: passes unmodified request to provider and returns unmodified response
    • clients now use x-llmboss-provider HTTP Header to designate upstream provider
  • hostname now (optionally) provided by env var, (still) matches framerock default if unspecified
  • ⚠️ BACKWARD-INCOMPATIBLE CHANGES ⚠️
    • removed legacy endpoint, templating, streaming
• Version 0.2.0
  • improved templating logic