local-inference-lab/rtx6kpro
quality grade C, 60 out of 100RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink
- stars
- 1.2k
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Cluster scheduling, GPU sharing, serverless compute and the distributed execution layer.
Signals: gpu, cuda, distributed-computing, kubernetes, ray, slurm, serverless, hpc
649 results
RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink
WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime for cloud native, edge, and decentralized applications. It powers serverless apps, embedded functions, microservices, smart contracts, and IoT devices.
Open3D: A Modern Library for 3D Data Processing
:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and model-agnostic. Currently shipping Qwen3.8-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1×/2×/4x/8x cards.
SonicJS - The edge-native headless CMS for Cloudflare Workers. Sub-100ms response times, zero cold starts, TypeScript-first. Built on D1, R2, and Hono.
A flyweight in situ visualization and analysis runtime for multi-physics HPC simulations
Build, Validate, Route, Authenticate and Mock using OpenAPI
Expanding Serverless Framework beyond functions using the AWS CDK
Persist and reuse KV Cache to speedup your LLM.
Ultrafast serverless GPU inference, sandboxes, and background jobs
NVIDIA cuML: GPU-Accelerated Machine Learning
The developer platform for easily building reliable workflows with zero infrastructure for TypeScript & JavaScript
The AWS Serverless Application Model (AWS SAM) transform is a AWS CloudFormation macro that transforms SAM templates into CloudFormation templates.
A machine learning accelerator core designed for energy-efficient AI at the edge.
eBPF Observability - Distributed Tracing and Profiling
Enterprise Kubernetes management, accelerated. 🚀
Temporary rooms for Humans and AI Agents — run, supervise and steer Agent Tasks without a permanent workspace.
A modern Kubernetes GUI management desktop app for macOS & Windows. Multi-cluster support, real-time monitoring, AI assistant, terminal access, and more.
An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.
Amazon Q, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources
The Open Source Firebase Alternative with GraphQL.
KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes
Vercel Storage - Blob and Edge Config
24,543 repositories in the index in total.