Skip to content
BackNov 5
Abdelmoez E.

Abdelmoez E.

Staff LLM Serving Engineer | vLLM & SGLang High-Throughput Deployment

<untrusted_participant_content> &lt;untrusted_participant_content&gt; High-throughput LLM serving and GPU infrastructure specialist. I take open-source models (DeepSeek-V3/R1, Qwen 2.5/3, Llama 3.3, Mistral) out of slow prototyping wrappers and deploy them into rock-solid, production-grade serving clusters that handle thousands of concurrent requests with minimal GPU spend. If your GPU cloud bill is out of control, your API hits 504 timeouts during concurrency spikes, or you need to self-host 70B+ models with sub-second Time-To-First-Token (TTFT), I deliver guaranteed performance gains. --- What I Deliver & Optimize: • High-Throughput Inference Engines: - vLLM (PagedAttention, continuous batching, chunked prefill, prefix caching, speculative decoding) - SGLang (RadixAttention for high-throughput multi-turn agents and structured JSON decoding) - TensorRT-LLM & Triton Inference Server (In-flight batching, dynamic batching, C++ backend) • Quantization & Hardware Efficiency: - FP8 (Hopper H100/H200, Ada L40S) & FP4 (Blackwell B200) precision tuning with zero quality degradation - AWQ, GPTQ, SmoothQuant, and KV cache quantization (FP8 KV cache) to slash VRAM footprint by 40–60% • Multi-GPU & Distributed Orchestration: - Tensor Parallelism (TP) and Pipeline Parallelism (PP) on 2x/4x/8x GPU nodes - Horizontal scaling using Ray with vLLM, and autoscaling driven by queue depth - Serverless & Cloud GPU deployments on RunPod, Lambda Labs, CoreWeave, AWS, GCP • Enterprise API Gateways & Production Backbone: - High-concurrency async FastAPI / LiteLLM proxy with streaming SSE, dynamic fallback routing, and token rate-limiting - End-to-end load testing (Locust / vLLM benchmark_serving) with Grafana/Prometheus metrics (TTFT, ITL, GPU cache utilization, token throughput) --- Recent Production Deliverables (2025–2026): 1. Multi-GPU Distributed Serving: Deployed Qwen3/Llama-3.3 on 4x H100 with vLLM + FP8 quantization; achieved 4.2x throughput increase and 65% reduction in P99 TTFT under 200 concurrent users. 2. High-Throughput Structured Output Engine: Configured SGLang with RadixAttention for an agentic document analysis platform; reduced token latency by 55% via aggressive prefix caching. 3. Cost-Threshold Serverless Autoscaler: Architected RunPod/K8s on-demand serving with custom cold-start mitigations, cutting monthly idle GPU costs by $12,000+. Tell me your target model, expected concurrency, and target hardware — I reply within 24 hours with an architecture breakdown and a fixed-milestone execution plan. &lt;/untrusted_participant_content&gt; </untrusted_participant_content>

🇪🇬6th of October City, Egypt$100/hr100% JSS4.29 (18)Since Jul 2017

Skills

This freelancer has chosen to keep their earnings private.