FauxPilot

FauxPilot is a self-hosted AI code completion server designed to run open-source language models on local hardware. It exposes an OpenAI-compatible API backed by NVIDIA Triton Inference Server and FasterTransformer, enabling development teams to serve code completions inside private infrastructure without transmitting source code to external cloud providers.

Why FauxPilot instead of Amazon CodeWhisperer

Amazon CodeWhisperer operates as an AWS-managed cloud service that streams editor keystrokes, active files, and code context to remote servers to generate suggestions. For organizations in regulated sectors, healthcare, or air-gapped environments, transmitting proprietary logic across the public internet violates compliance policies. FauxPilot replaces cloud transmission with local or private cluster execution. Source code and completions remain inside your perimeter, giving DevOps engineers full control over telemetry, network egress, and data access policies.

That architectural autonomy shifts operational responsibility to your internal team. Amazon CodeWhisperer requires no infrastructure upkeep, bundles built-in security scans, and connects directly into IDEs. FauxPilot provides only the raw completion backend. Deploying it requires dedicated NVIDIA GPUs, Docker container management, and manual endpoint configuration in developer editors. While CodeWhisperer offers turnkey convenience backed by AWS foundation models, FauxPilot grants complete data sovereignty at the cost of managing your own hardware and model weights.

What it does well

• Complete data isolation: code parsing and token generation stay within private networks, eliminating third-party retention risks.
• OpenAI-compatible API: exposes standard completion endpoints, allowing editors and extensions configured for OpenAI or Copilot to connect directly.
• Multi-GPU model parallelism: splits larger models across multiple NVIDIA cards via FasterTransformer, pooling distributed VRAM across available GPUs.
• Configurable model sizes: runs Salesforce CodeGen checkpoints from 350 million to 16 billion parameters to match available VRAM.
• Containerized deployment: uses Docker Compose to package Triton Inference Server and CUDA dependencies for reproducible Linux host setup.
• Predictable infrastructure costs: swaps recurring per-seat cloud fees for fixed on-premises compute expenses on existing hardware.