(https://github.githubassets.com) (https://avatars.githubusercontent.com) (https://github-cloud.s3.amazonaws.com) (https://user-images.githubusercontent.com/) (https://github.githubassets.com) (https://avatars.githubusercontent.com) (https://github.githubassets.com/assets/react-e27d1b3e03961e68.js) (https://github.githubassets.com/assets/react-compiler-runtime-4610bd6d3de9c049.js) (https://github.githubassets.com/assets/scheduler-58b860b049ca307c.js) (https://github.githubassets.com/assets/react-dom-e5fd46a22d5c4058.js) (https://github.githubassets.com/assets/react-dom-client-1b4a3ee065998cea.js) (https://github.githubassets.com/assets/react-is-e0b593954b4706d8.js) (https://github.githubassets.com/assets/react-jsx-runtime-4915cb0f5b3aff04.js) (https://github.githubassets.com/assets/react-reconciler-8e99e505c4429605.js) (../../github.githubassets.com/assets/light-99f877e9ddfc0e51.css) (../../github.githubassets.com/assets/light_high_contrast-48fdd0811afbab3c.css) (../../github.githubassets.com/assets/dark-79ad2ace604703b3.css) (../../github.githubassets.com/assets/dark_high_contrast-24484a076f02295f.css) (../../github.githubassets.com/assets/primer-primitives-ed9ca172356fd545.css) (../../github.githubassets.com/assets/primer-4136ede8b2650a2d.css) (../../github.githubassets.com/assets/global-54ba76e934a49d7c.css) (../../github.githubassets.com/assets/github-7338c07b3a07b8be.css) (../../github.githubassets.com/assets/repository-11ee8a031c040c1a.css) (../../github.githubassets.com/assets/code-2d56bdb0166c0238.css) (https://github.githubassets.com/assets/wp-runtime-cd44fa5e61606473.js) (https://github.githubassets.com/assets/app-foundation-69691b300d797200.js) (https://github.githubassets.com/assets/app-runtime-03eff074f84e6c35.js) (https://github.githubassets.com/assets/fetch-utilities-4b62182bc6e3c348.js) (https://github.githubassets.com/assets/ser-304cac05a866d0c2.js) (https://github.githubassets.com/assets/tp-34eaad11125a9e67.js) (../../github.githubassets.com/assets/app-runtime.c86f683a047bf1c6.module.css) (https://github.githubassets.com/assets/catalyst-044fa3a2acdaf0a9.js) (https://github.githubassets.com/assets/selector-observer-e88088f989b27670.js) (https://github.githubassets.com/assets/relative-time-element-fa0968675d128a19.js) (https://github.githubassets.com/assets/by-6b3c07383371bf58.js) (https://github.githubassets.com/assets/ja9-36f036c13c2e3a4c.js) (https://github.githubassets.com/assets/kwb-985b266c34021458.js) (https://github.githubassets.com/assets/r1-b075aab3204f32f9.js) (https://github.githubassets.com/assets/jz5-7c5d1669717041f8.js) (https://github.githubassets.com/assets/hj-5126390ff433704d.js) (https://github.githubassets.com/assets/j05-1b324859b9512e2d.js) (https://github.githubassets.com/assets/runtime-helpers-5ca1e1883536e092.js) (https://github.githubassets.com/assets/aria-live-76b18916d0c8ec4d.js) (https://github.githubassets.com/assets/hotkey-5109c77d7ac61078.js) (https://github.githubassets.com/assets/react-core-e3a2b2ca37b6ff4b.js) (https://github.githubassets.com/assets/a9-6602e6cd810d97c7.js) (https://github.githubassets.com/assets/ur-024971cb63acad59.js) (https://github.githubassets.com/assets/4t-364e19c63d726e9a.js) (https://github.githubassets.com/assets/6ny-c969fbae697a356f.js) (https://github.githubassets.com/assets/yhq-3a209035a30b1a09.js) (https://github.githubassets.com/assets/z1v-da5b33b877bfa358.js) (https://github.githubassets.com/assets/wo-8044e1bac6e0c328.js) (https://github.githubassets.com/assets/9yx-d8a7443ee1158ac6.js) (https://github.githubassets.com/assets/j0-41a4cf04cbd99503.js) (https://github.githubassets.com/assets/0p-81491ed0d9d666e2.js) (../../github.githubassets.com/assets/react-core.de6197eb1aa4cab1.module.css) (https://github.githubassets.com/assets/n9c-16e227ac48072d2b.js) (https://github.githubassets.com/assets/ic-991f04433f366e7b.js) (https://github.githubassets.com/assets/primer-react-2364f7b34e882f24.js) (https://github.githubassets.com/assets/ncx-9a860680a5f230b9.js) (https://github.githubassets.com/assets/j6-363f0ebf4f62f310.js) (https://github.githubassets.com/assets/t7l-bf6f21d7e7a43c35.js) (https://github.githubassets.com/assets/4u-3fb51fd5b58f1705.js) (https://github.githubassets.com/assets/ws-f1530f278b2b86e5.js) (https://github.githubassets.com/assets/s07-40c12c0b9c65754d.js) (https://github.githubassets.com/assets/n4u-6317ce1c31aed1db.js) (https://github.githubassets.com/assets/j8-bce747afaaaab486.js) (https://github.githubassets.com/assets/la-d2cd96f070e6b52a.js) (https://github.githubassets.com/assets/wb-7be2a2dbb2126c50.js) (https://github.githubassets.com/assets/z54-8ecb6d8609e3cc5a.js) (https://github.githubassets.com/assets/y6k-1b4caa3d041c2bb7.js) (https://github.githubassets.com/assets/80-9bdecb534530e940.js) (https://github.githubassets.com/assets/i3-37a6c0193f828eda.js) (https://github.githubassets.com/assets/gzu-b0e631d87d54c83d.js) (https://github.githubassets.com/assets/bi0-a78ba744687848c2.js) (https://github.githubassets.com/assets/on-898389fbf97246b6.js) (https://github.githubassets.com/assets/b1l-80083619107dbc39.js) (https://github.githubassets.com/assets/0l5-d2057e622f6442d6.js) (https://github.githubassets.com/assets/c2z-c10e51960850b47e.js) (https://github.githubassets.com/assets/f9-038758dcbfe08dab.js) (https://github.githubassets.com/assets/sp-741a59f220bc8fca.js) (https://github.githubassets.com/assets/0k-b7df0a74912a42bf.js) (https://github.githubassets.com/assets/og-72a377e4a14147a1.js) (https://github.githubassets.com/assets/b9-4fd36ab02c30bf52.js) (https://github.githubassets.com/assets/pe-0996411d3a882f4b.js) (https://github.githubassets.com/assets/dsj-cb1e4ddbce4f007b.js) (https://github.githubassets.com/assets/du-9695d8032ef894d8.js) (https://github.githubassets.com/assets/20x-9c793f5fbfb1a8eb.js) (https://github.githubassets.com/assets/ql0-d154b9935a526802.js) (https://github.githubassets.com/assets/wp-9f5235d335ab624b.js) (https://github.githubassets.com/assets/p0-917d47d17033694c.js) (https://github.githubassets.com/assets/uwi-80cf09633c6152b1.js) (https://github.githubassets.com/assets/th-5bd53302e6db9640.js) (https://github.githubassets.com/assets/7c-51b9e794c65c18dc.js) (https://github.githubassets.com/assets/6-07dff163e9c051a2.js) (https://github.githubassets.com/assets/je-2b451319b3493194.js) (https://github.githubassets.com/assets/cqu-ae855d304cc66202.js) (https://github.githubassets.com/assets/40-9352f323577b511f.js) (https://github.githubassets.com/assets/y8-03759649e1df028e.js) (https://github.githubassets.com/assets/xc-065e7847e4da513d.js) (https://github.githubassets.com/assets/7o-b3afe55b2b5952e7.js) (../../github.githubassets.com/assets/primer-react-css.9595615ef0f67f57.module.css) (../../github.githubassets.com/assets/ql0.bdd6be015c087bbc.module.css) (../../github.githubassets.com/assets/6.8132a6cad51a9393.module.css) (../../github.githubassets.com/assets/xc.877a51df9233c9c4.module.css) (../../github.githubassets.com/assets/7o.d5eb18fb1bdbeb26.module.css) (../../github.githubassets.com/assets/code-view.005a7d0f5c269051.module.css) (https://github.githubassets.com/assets/57-2b8bb2ff8d63201f.js) (../../github.githubassets.com/assets/primer-react-css.9595615ef0f67f57.module.css) (../../github.githubassets.com/assets/notifications-subscriptions-menu.ab64298eed7e9127.module.css) GitHub - Vibra-Ingenn/Janus: Janus is a API router for AI models written in Go and has a Vulkan Model runner · GitHub (repo_source) (https://github.githubassets.com/) (https://github.com/opensearch.xml) (GitHub) (https://github.com/fluidicon.png) (GitHub) (https://github.githubassets.com/assets/pinned-octocat-093da3e6fa40.svg) (https://github.githubassets.com/favicons/favicon.png) (../../github.githubassets.com/favicons/favicon.svg) (../manifest.json) (Janus.html#start-of-content) Skip to content (https://github.githubassets.com/assets/l7j-2ddd8b435dadae28.js) (https://github.githubassets.com/assets/keyboard-shortcuts-dialog-a4f5252e75daceae.js) (../../github.githubassets.com/assets/primer-react-css.9595615ef0f67f57.module.css) (../../github.githubassets.com/assets/keyboard-shortcuts-dialog.dba30dc12f524840.module.css) (https://github.githubassets.com/assets/pw-4f91498888595802.js) (https://github.githubassets.com/assets/app-install-banner-partial-5bd0c22a01b42e96.js) (../../github.githubassets.com/assets/primer-react-css.9595615ef0f67f57.module.css) (../../github.githubassets.com/assets/app-install-banner-partial.cf3db6e563824d82.module.css) (https://github.githubassets.com/assets/ak-e62e69106ade8d4b.js) (https://github.githubassets.com/assets/rwd-3f84ac06d286ef82.js) (https://github.githubassets.com/assets/fz5-23f28c0f388769c9.js) (https://github.githubassets.com/assets/jag-4f408bf2b7f99c87.js) (https://github.githubassets.com/assets/p1-76a75a6081059b19.js) (https://github.githubassets.com/assets/3d8-bbc4e3ce35fac150.js) (https://github.githubassets.com/assets/tua-88f94f65812643db.js) (https://github.githubassets.com/assets/t-2113ce3fc5dedb5c.js) (https://github.githubassets.com/assets/marketing-header-91f2b0b43d89f4ff.js) (../../github.githubassets.com/assets/primer-react-css.9595615ef0f67f57.module.css) (../../github.githubassets.com/assets/primer-react-brand-css.05b219cdb8e80c43.module.css) (../../github.githubassets.com/assets/t.f5c029d2b3105319.module.css) Navigation Menu (https://github.com/) (https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FVibra-Ingenn%2FJanus) Sign in Appearance settings Platform AI CODE CREATION (https://github.com/features/copilot) GitHub Copilot Write better code with AI (https://github.com/features/ai/github-app) GitHub Copilot app Direct agents from issue to merge (https://github.com/mcp) MCP Registry Integrate external tools DEVELOPER WORKFLOWS (https://github.com/features/actions) Actions Automate any workflow (https://github.com/features/codespaces) Codespaces Instant dev environments (https://github.com/features/issues) Issues Plan and track work (https://github.com/features/code-review) Code Review Manage code changes (https://github.com/features/code-quality) Code Quality Enforce quality at merge APPLICATION SECURITY (https://github.com/security/advanced-security) GitHub Advanced Security Find and fix vulnerabilities (https://github.com/security/advanced-security/code-security) Code security Secure your code as you build (https://github.com/security/advanced-security/secret-protection) Secret protection Stop leaks before they start EXPLORE (https://github.com/why-github) Why GitHub (https://docs.github.com) Documentation (https://github.blog) Blog (https://github.blog/changelog) Changelog (https://github.com/marketplace) Marketplace (https://github.com/features) View all features Solutions BY COMPANY SIZE (https://github.com/enterprise) Enterprises (https://github.com/team) Small and medium teams (https://github.com/enterprise/startups) Startups (https://github.com/solutions/industry/nonprofits) Nonprofits BY USE CASE (https://github.com/solutions/use-case/app-modernization) App Modernization (https://github.com/solutions/use-case/devsecops) DevSecOps (https://github.com/solutions/use-case/devops) DevOps (https://github.com/solutions/use-case/ci-cd) CI/CD (https://github.com/solutions/use-case) View all use cases BY INDUSTRY (https://github.com/solutions/industry/healthcare) Healthcare (https://github.com/solutions/industry/financial-services) Financial services (https://github.com/solutions/industry/manufacturing) Manufacturing (https://github.com/solutions/industry/government) Government (https://github.com/solutions/industry) View all industries (https://github.com/solutions) View all solutions Resources EXPLORE BY TOPIC (https://github.com/resources/articles?topic=ai) AI (https://github.com/resources/articles?topic=software-development) Software Development (https://github.com/resources/articles?topic=devops) DevOps (https://github.com/resources/articles?topic=security) Security (https://github.com/resources/articles) View all topics EXPLORE BY TYPE (https://github.com/customer-stories) Customer stories (https://github.com/resources/events) Events & webinars (https://github.com/resources/whitepapers) Ebooks & reports (https://github.com/solutions/executive-insights) Business insights (https://skills.github.com) GitHub Skills SUPPORT & SERVICES (https://docs.github.com) Documentation (https://support.github.com) Customer support (https://github.com/orgs/community/discussions) Community forum (https://github.com/trust-center) Trust center (https://github.com/partners) Partners (https://github.com/resources) View all resources Open Source COMMUNITY (https://github.com/open-source/sponsors) GitHub Sponsors Fund open source developers PROGRAMS (https://securitylab.github.com) Security Lab (https://maintainers.github.com) Maintainer Community (https://stars.github.com) GitHub Stars (https://archiveprogram.github.com) Archive Program REPOSITORIES (https://github.com/topics) Topics (https://github.com/trending) Trending (https://github.com/collections) Collections Enterprise ENTERPRISE SOLUTIONS (https://github.com/enterprise) Enterprise platform AI-powered developer platform AVAILABLE ADD-ONS (https://github.com/security/advanced-security) GitHub Advanced Security Enterprise-grade security features (https://github.com/features/copilot/copilot-business) Copilot for Business Enterprise-grade AI features (https://github.com/enterprise/premium-support) Premium Support Enterprise-grade 24/7 support (https://github.com/pricing) Pricing Search / (https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FVibra-Ingenn%2FJanus) Sign in (https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=Vibra-Ingenn%2FJanus) Sign up Appearance settings You signed in with another tab or window. (Janus.html) Reload to refresh your session. You signed out in another tab or window. (Janus.html) Reload to refresh your session. You switched accounts on another tab or window. (Janus.html) Reload to refresh your session. Dismiss alert (https://github.com/Vibra-Ingenn) Vibra-Ingenn / (Janus.html) Janus Public (https://github.com/login?return_to=%2FVibra-Ingenn%2FJanus) Notifications You must be signed in to change notification settings (https://github.com/login?return_to=%2FVibra-Ingenn%2FJanus) Fork (0) 0 (https://github.com/login?return_to=%2FVibra-Ingenn%2FJanus) Star (8) 8 (Janus.html) Code (Not available) (https://github.com/Vibra-Ingenn/Janus/issues) Issues (0) 0 (https://github.com/Vibra-Ingenn/Janus/pulls) Pull requests (0) 0 (https://github.com/Vibra-Ingenn/Janus/actions) Actions (Not available) (https://github.com/Vibra-Ingenn/Janus/projects) Projects (Not available) (https://github.com/Vibra-Ingenn/Janus/security) Security and quality (0) 0 (https://github.com/Vibra-Ingenn/Janus/pulse) Insights (Not available) Additional navigation options (Janus.html) Code (https://github.com/Vibra-Ingenn/Janus/issues) Issues (https://github.com/Vibra-Ingenn/Janus/pulls) Pull requests (https://github.com/Vibra-Ingenn/Janus/actions) Actions (https://github.com/Vibra-Ingenn/Janus/projects) Projects (https://github.com/Vibra-Ingenn/Janus/security) Security and quality (https://github.com/Vibra-Ingenn/Janus/pulse) Insights (Janus.html) main (https://github.com/Vibra-Ingenn/Janus/branches) Branches (https://github.com/Vibra-Ingenn/Janus/tags) Tags (https://github.com/Vibra-Ingenn/Janus/branches) (https://github.com/Vibra-Ingenn/Janus/tags) (Go to file) () Go to file Code Open more actions menu Latest commit History (https://github.com/Vibra-Ingenn/Janus/commits/main/) 2 Commits (https://github.com/Vibra-Ingenn/Janus/commits/main/) 2 Commits Folders and files Name Name (Last commit message) Last commit message (Last commit date) Last commit date (This path skips through empty directories) (https://github.com/Vibra-Ingenn/Janus/tree/main/.github/ISSUE_TEMPLATE) .github/ ISSUE_TEMPLATE (This path skips through empty directories) (https://github.com/Vibra-Ingenn/Janus/tree/main/.github/ISSUE_TEMPLATE) .github/ ISSUE_TEMPLATE (cmd) (https://github.com/Vibra-Ingenn/Janus/tree/main/cmd) cmd (cmd) (https://github.com/Vibra-Ingenn/Janus/tree/main/cmd) cmd (internal) (https://github.com/Vibra-Ingenn/Janus/tree/main/internal) internal (internal) (https://github.com/Vibra-Ingenn/Janus/tree/main/internal) internal (lib) (https://github.com/Vibra-Ingenn/Janus/tree/main/lib) lib (lib) (https://github.com/Vibra-Ingenn/Janus/tree/main/lib) lib (models) (https://github.com/Vibra-Ingenn/Janus/tree/main/models) models (models) (https://github.com/Vibra-Ingenn/Janus/tree/main/models) models (.env.example) (https://github.com/Vibra-Ingenn/Janus/blob/main/.env.example) .env.example (.env.example) (https://github.com/Vibra-Ingenn/Janus/blob/main/.env.example) .env.example (.gitignore) (https://github.com/Vibra-Ingenn/Janus/blob/main/.gitignore) .gitignore (.gitignore) (https://github.com/Vibra-Ingenn/Janus/blob/main/.gitignore) .gitignore (CODE_OF_CONDUCT.md) (https://github.com/Vibra-Ingenn/Janus/blob/main/CODE_OF_CONDUCT.md) CODE_OF_CONDUCT.md (CODE_OF_CONDUCT.md) (https://github.com/Vibra-Ingenn/Janus/blob/main/CODE_OF_CONDUCT.md) CODE_OF_CONDUCT.md (CONTRIBUTING.md) (https://github.com/Vibra-Ingenn/Janus/blob/main/CONTRIBUTING.md) CONTRIBUTING.md (CONTRIBUTING.md) (https://github.com/Vibra-Ingenn/Janus/blob/main/CONTRIBUTING.md) CONTRIBUTING.md (LICENSE) (https://github.com/Vibra-Ingenn/Janus/blob/main/LICENSE) LICENSE (LICENSE) (https://github.com/Vibra-Ingenn/Janus/blob/main/LICENSE) LICENSE (README.md) (https://github.com/Vibra-Ingenn/Janus/blob/main/README.md) README.md (README.md) (https://github.com/Vibra-Ingenn/Janus/blob/main/README.md) README.md (build.ps1) (https://github.com/Vibra-Ingenn/Janus/blob/main/build.ps1) build.ps1 (build.ps1) (https://github.com/Vibra-Ingenn/Janus/blob/main/build.ps1) build.ps1 (build.sh) (https://github.com/Vibra-Ingenn/Janus/blob/main/build.sh) build.sh (build.sh) (https://github.com/Vibra-Ingenn/Janus/blob/main/build.sh) build.sh (go.mod) (https://github.com/Vibra-Ingenn/Janus/blob/main/go.mod) go.mod (go.mod) (https://github.com/Vibra-Ingenn/Janus/blob/main/go.mod) go.mod (go.sum) (https://github.com/Vibra-Ingenn/Janus/blob/main/go.sum) go.sum (go.sum) (https://github.com/Vibra-Ingenn/Janus/blob/main/go.sum) go.sum (run.ps1) (https://github.com/Vibra-Ingenn/Janus/blob/main/run.ps1) run.ps1 (run.ps1) (https://github.com/Vibra-Ingenn/Janus/blob/main/run.ps1) run.ps1 (start-janus.ps1) (https://github.com/Vibra-Ingenn/Janus/blob/main/start-janus.ps1) start-janus.ps1 (start-janus.ps1) (https://github.com/Vibra-Ingenn/Janus/blob/main/start-janus.ps1) start-janus.ps1 (start.sh) (https://github.com/Vibra-Ingenn/Janus/blob/main/start.sh) start.sh (start.sh) (https://github.com/Vibra-Ingenn/Janus/blob/main/start.sh) start.sh View all files Repository files navigation (Janus.html#) README (Janus.html#) Code of conduct (Janus.html#) Contributing (Janus.html#) MIT license Moreitems Janus — Local LLM Server & OpenAI-Compatible API (Janus.html#janus--local-llm-server--openai-compatible-api) Janus is a single Go binary that runs .gguf models on your machine (GPU or CPU) and exposes an OpenAI-compatible API . No Python, no Docker, no Ollama required. Use it your way: call it from the command line (curl , PowerShell, scripts), wire it into Cursor / Cline / any OpenAI client — same local models, whatever workflow fits you. What you get (Janus.html#what-you-get) Local inference — llama.cpp via Vulkan (AMD / Intel / NVIDIA) or CPU fallback OpenAI-compatible API — /v1/chat/completions , /v1/models Hot-swap models — change .gguf without restarting Thinking model support — reasoning split into reasoning_content Chat template auto-detection — uses the template from GGUF metadata Zero dependencies — one .exe on Windows, no Python, no Docker Requirements (Janus.html#requirements) Platform What you need Windows (primary) Windows 10/11, (https://go.dev/dl/) Go 1.22+ , Vulkan-capable GPU recommended Linux Go 1.22+, Vulkan or CPU macOS Go 1.22+, CPU backend (Vulkan varies by hardware) Disk: plan for the model size (often 2–8 GB per model) plus ~50 MB for Janus + llama.dll. Quick start (Windows) (Janus.html#quick-start-windows) 1. Clone and build (Janus.html#1-clone-and-build) git clone https:// github.com / Vibra- Ingenn/ Janus.git cd Janus .\build.ps1 build.ps1 downloads pre-built llama.cpp Vulkan DLLs and compiles dist\janus.exe . 2. Download a model (Janus.html#2-download-a-model) Put a .gguf file in the models\ folder. Use the included downloader: go build - o dist\modelget.exe .\cmd\modelget .\dist\modelget.exe - repo meta- llama/ Llama- 3.2 - 3B- Instruct - file Llama- 3.2 - 3B- Instruct- Q8_0.gguf - out .\models\ Or download any GGUF from (https://huggingface.co/models?library=gguf) Hugging Face . 3. Configure (Janus.html#3-configure) copy .env.example .env Edit .env : INFERENCE_BACKEND = vulkan JANUS_MODEL_PATH = ./models/Llama-3.2-3B-Instruct-Q8_0.gguf JANUS_MAX_TOKENS = 4096 Variable Default Meaning INFERENCE_BACKEND vulkan vulkan , cpu , or openrouter JANUS_MODEL_PATH (required) Path to your .gguf file JANUS_GPU_LAYERS -1 -1 = all layers on GPU, 0 = CPU only JANUS_VRAM_CEILING_MB 9216 VRAM budget hint (MiB) JANUS_MAX_TOKENS 4096 Max tokens per reply JANUS_LISTEN_ADDR 127.0.0.1:8990 Bind address 4. Run (Janus.html#4-run) .\dist\janus.exe Opens (http://127.0.0.1:8990) http://127.0.0.1:8990 in your browser. 5. Verify (Janus.html#5-verify) curl http:// 127.0 .0.1 :8990 / health Quick start (Linux / macOS) (Janus.html#quick-start-linux--macos) git clone https://github.com/Vibra-Ingenn/Janus.git cd Janus go mod tidy go build -o dist/janus ./cmd/janus cp .env.example .env # edit .env — set JANUS_MODEL_PATH and INFERENCE_BACKEND=cpu if no Vulkan ./dist/janus On Linux you need libllama.so next to the binary or on LD_LIBRARY_PATH . OpenAI-compatible API (Janus.html#openai-compatible-api) curl http://127.0.0.1:8990/v1/chat/completions \ -H " Content-Type: application/json" \ -d ' { "model": "local", "messages": [{"role": "user", "content": "Hello!"}] }' Base URL: http://127.0.0.1:8990/v1 Endpoints (Janus.html#endpoints) Method Path Description GET /health Liveness check (?deep=true for details) GET /v1/models Model list POST /v1/chat/completions Chat (streaming supported) POST /models/load Hot-swap model GET /models/list Available .gguf files GET /engine/status VRAM and backend info Streaming (Janus.html#streaming) curl http://127.0.0.1:8990/v1/chat/completions \ -H " Content-Type: application/json" \ -d ' {"model":"local","stream":true,"messages":[{"role":"user","content":"Tell me a joke"}]}' Connect to Cursor / Cline / other clients (Janus.html#connect-to-cursor--cline--other-clients) Base URL: http://127.0.0.1:8990/v1 API Key: (leave blank) Pitfalls (learned the hard way) (Janus.html#pitfalls-learned-the-hard-way) Problem What's going on Fix "It built but my changes aren't there" On Windows, Go can't overwrite a running .exe . Stop all janus.exe in Task Manager, then rebuild. "Address already in use" A leftover process holds port 8990. Task Manager → end all janus.exe . Server starts, chat fails JANUS_MODEL_PATH wrong or no .gguf in models/ . Set the path in .env , put the file in models/ . "local engine failed to start" Missing llama.dll or GPU driver issue. Run .\build.ps1 . Update GPU drivers, or set INFERENCE_BACKEND=cpu . Wrong URL Default is http://127.0.0.1:8990 , not 8080. Bookmark 8990. First reply takes forever Model loading into VRAM — normal. Wait 10–60s; smaller quants (Q4 ) load faster. Project layout (Janus.html#project-layout) cmd/janus/ Main server (OpenAI-compatible API) cmd/modelget/ Hugging Face model downloader internal/engine/ llama.cpp Vulkan/CPU backend internal/bridge/ DLL loader and FFI bindings internal/singleton/ Single-instance guard models/ Put .gguf files here (not committed) dist/ janus.exe + llama.dll after build Development (Janus.html#development) Get-Process - Name " janus" - ErrorAction SilentlyContinue | Stop-Process - Force go test ./ ... go build - o dist\janus.exe .\cmd\janus .\dist\janus.exe Or just .\run.ps1 . For a full build including llama DLLs, use .\build.ps1 . Contributions welcome — see (https://github.com/Vibra-Ingenn/Janus/blob/main/CONTRIBUTING.md) CONTRIBUTING.md . License (Janus.html#license) MIT — see (https://github.com/Vibra-Ingenn/Janus/blob/main/LICENSE) LICENSE . About Janus is a API router for AI models written in Go and has a Vulkan Model runner Resources (Janus.html#readme-ov-file) Readme (Janus.html#MIT-1-ov-file) MIT license Code of conduct (Janus.html#coc-ov-file) Code of conduct Contributing (Janus.html#contributing-ov-file) Contributing (https://github.com/Vibra-Ingenn/Janus/activity) Activity Stars 8 stars Watchers 0 watching Forks (https://github.com/Vibra-Ingenn/Janus/forks) 0 forks (https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2FVibra-Ingenn%2FJanus&report=Vibra-Ingenn+%28user%29) Report repository Releases Packages Contributors Languages Footer (https://github.com) © 2026 GitHub, Inc. Footer navigation (https://docs.github.com/site-policy/github-terms/github-terms-of-service) Terms (https://docs.github.com/site-policy/privacy-policies/github-privacy-statement) Privacy (https://github.com/security) Security (https://www.githubstatus.com/) Status (https://github.community/) Community (https://docs.github.com/) Docs (https://support.github.com?tags=dotcom-footer) Contact Manage cookies Do not share my personal information You can’t perform that action at this time.